Knowledge graph ontology constrained vegetation remote sensing classification method

Abstract

The invention discloses a knowledge graph ontology constrained vegetation remote sensing classification method. The method comprises the steps of data acquisition and data preprocessing, deep learning model selection and fusion of a knowledge graph ontology and a deep learning model. According to the method, the knowledge graph ontology and the FCN8s-ResNet 50 model are fused in the two stages of model training and classification post-processing. The result shows that 1) the classification accuracy of the deep learning model can be remarkably improved by fusing the mountain vegetation knowledge graph ontology in the two stages of deep learning model training and classification post-processing: the accuracy of the deep learning model constrained by the knowledge graph ontology is improved by 10%, and the overall classification accuracy of vegetation types reaches 89.2%; and 2) the deep learning model constrained by the mountain vegetation mapping knowledge domain ontology not only obtains excellent results in fine classification of vegetation types, but also improves the universality and interpretability of the model, so that powerful support is provided for wide-range remote sensing vegetation classification and application of the deep learning model.

Description

Technical Field

[0001] The present invention relates to the technical field of vegetation remote sensing classification, and in particular to a remote sensing classification method for vegetation community level constrained by a knowledge graph ontology. Background Art

[0002] Vegetation, as an important component of terrestrial ecosystems, has a significant impact on global material flows, energy flows, carbon balance and climate stability at different spatial and temporal scales. High-precision classification and rapid monitoring of mountain vegetation is an important basis for systematic research on the structure and ecological functions of mountain ecosystems, and provides necessary reference information for ecological restoration, climate change and carbon balance. Traditional vegetation classification and mapping are mainly based on field surveys, which are time-consuming and laborious. The development of remote sensing technology has not only solved the obstacles to field surveys caused by inconvenient transportation in mountainous environments, but also made mountain vegetation monitoring more real-time and effective.

[0003] At present, intelligent interpretation using high-resolution remote sensing images has become an important way to study vegetation cover, vegetation structure composition and its dynamic changes. Although the amount of remote sensing data acquired every day has reached the PB level, there are more and more sensors, the acquisition time is getting faster and faster, and there are more and more algorithms for remote sensing image target recognition and classification, vegetation classification research is still one of the most challenging problems in remote sensing science due to the spectral similarity between vegetation canopies. Although deep learning methods are widely used in vegetation classification tasks and have good performance in vegetation cover identification and dynamic monitoring, they perform poorly in vegetation fine classification, especially classification at the vegetation community level. At the same time, the existing deep learning models for vegetation classification are difficult to apply to large-scale tasks due to their low interpretability and universality, which limits the application and service capabilities of remote sensing big data. Summary of the invention

[0004] In response to the above-mentioned problems, the present invention aims to provide a vegetation remote sensing classification method constrained by a knowledge graph ontology, which can significantly improve the classification accuracy of the deep learning model by integrating the mountain vegetation knowledge graph ontology in the model training and classification post-processing stages; the deep learning model constrained by the mountain vegetation knowledge graph ontology not only achieves excellent results in the fine classification of vegetation types, but also greatly improves the universality and interpretability of the model, which will provide strong support for remote sensing vegetation classification and application on a larger scale.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows: a vegetation remote sensing classification method constrained by knowledge graph ontology, comprising the following steps: S1. Data acquisition and data preprocessing 1) Obtain digital surface model data of the study area and generate a digital elevation model with the same resolution by combining spatial resampling technology; 2) Obtain multi-temporal remote sensing and digital elevation model data of the study area, unify the projection coordinate system and crop accordingly according to the resolution, and pre-process the multi-temporal remote sensing and digital elevation model data to be consistent with the spatial range of the input data; 3) Obtain the vertical band spectrum of vegetation in the study area, and pre-process the vertical band spectrum of vegetation into a structured knowledge graph ontology that reflects the geographical distribution law of vegetation; 4) Obtain vegetation type map data and remote sensing images of the study area, obtain samples of each vegetation type in the study area based on the vegetation type map data and remote sensing images, and form a sample data set of vegetation types in the study area.

[0006] S2. Select a deep learning model Select a deep learning model that can simultaneously learn the features of remote sensing images from different sources and with different resolutions and has multiple channels.

[0007] S3. Integration of knowledge graph ontology and deep learning model in the training phase The preprocessed data in steps 1)-2) and 4) of step S1 are input into the deep learning model for training, and the structured knowledge graph ontology and the deep learning model are integrated in the model training stage to obtain a training model for vegetation classification.

[0008] S4. Fusion of knowledge graph ontology and deep learning model in post-classification processing The structured knowledge graph ontology and the deep learning model are integrated in the post-processing stage of vegetation classification to finally obtain the vegetation classification results.

[0009] Preferably, the sample data set preprocessed in step S1 is divided into a training set and a validation set, and trained using a deep learning model while integrating a structured knowledge graph to ultimately obtain a vegetation classification result.

[0010] Preferably, during the deep learning model training process, structured knowledge graph information is integrated, and a vegetation type is predicted by the deep learning model to compare the difference between the altitude range of the corresponding vertical zone in the knowledge graph ontology and the actual altitude.

[0011] Preferably, based on the loss function of interval overlap, the matching degree between the altitude distribution range of vegetation types in the knowledge graph ontology and the actual altitude is quantified. The specific process is as follows: and Respectively represent The lower and upper limits of the altitude range of the samples in the knowledge graph ontology, Indicates The true altitude of samples, then the interval overlap loss is in the form of formula (5-1):

[0012] in, is a small positive number used to prevent the denominator from being zero; Add this constraint loss to the original cross entropy loss to form a new composite loss function, as shown in formula (5-2): (5-2)

[0013] in, is the cross entropy loss for classification tasks, It is the weight coefficient that balances the classification loss and interval overlap loss.

[0014] Preferably, the prediction results of the deep learning model in step S3 are first classified by similarity using the deep learning model to obtain the similarity value of each category. , and then integrate the structured knowledge graph ontology to calculate the similarity and obtain a new predicted similarity distribution , and obtain the final vegetation classification result based on the new similarity.

[0015] Preferably, for similarity For prediction results that are less than the threshold, use DEM to obtain the altitude of the corresponding location , and obtain the vertical distribution information of all vegetation types in the knowledge graph ontology, and then calculate The closeness to the vertical distribution of each vegetation type is converted into a similarity distribution according to formulas (5-3) and (5-4): ,in represent The possible value range is Representative The distribution range of vegetation types in the knowledge graph ontology, Number of representative vegetation types:

[0016] (5-4)

[0017] The model predicts similarity Similarity to that calculated based on altitude Perform weighted fusion to generate a new predicted similarity distribution , as shown in formula (5-5):

[0018] in, and The model predicts similarity Similarity to that calculated based on altitude The weight of , the sum of the two is 1; Finally, based on the new similarity distribution , select the category with the highest similarity as the final model classification result.

[0019] The beneficial effect of the present invention is that the present invention combines the knowledge graph with the FCN8s-ResNet50 model to conduct four classification experiments, and the results show that: 1) Integrating the mountain vegetation knowledge graph ontology during model training and classification post-processing can significantly improve the classification accuracy of the deep learning model: the accuracy of the deep learning model constrained by the knowledge graph ontology is improved by 10%, and the overall classification accuracy of vegetation types at the community level reaches 89.2%.

[0020] 2) The deep learning model constrained by the mountain vegetation knowledge graph not only achieved excellent results in the fine classification of vegetation types, but also greatly improved the universality and interpretability of the model, which will provide strong support for remote sensing vegetation classification and application on a larger scale.

[0021] The integration of knowledge graphs and deep learning models is of great significance for achieving rapid extraction of vegetation information, rapid updating of vegetation maps, and real-time monitoring of vegetation coverage. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is an overview of the study area of ​​this invention.

[0023] Figure 2 This is the technical flow chart of Experiment 2 of the present invention (deep learning model integrating knowledge graph ontology during the training phase).

[0024] Figure 3 This is the technical flow chart of Experiment 3 of the present invention (integration of the deep learning model with the knowledge graph ontology in the post-classification processing stage).

[0025] Figure 4 This is the technical flow chart of Experiment 4 (integration of knowledge graph ontology in training phase and post-classification processing phase) of the present invention.

[0026] Figure 5 It is the confusion matrix of the classification model of the four comparative experiments of the present invention.

[0027] Figure 6 This is the training curve of the deep learning classification model in the four comparative experiments of the present invention.

[0028] Figure 7 The vegetation remote sensing classification results of the present invention integrated with the knowledge graph: (a) 1:250,000 vegetation type map interpreted by manual visual interpretation, (b) vegetation remote sensing classification results integrated with the knowledge graph ontology, (c) remote sensing image of the study area. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0030] See attached Figures 1 to 7 A vegetation remote sensing classification method constrained by a knowledge graph ontology is shown, comprising the following steps: S1. Data acquisition and data preprocessing 1) Obtain digital surface model data of the study area and generate a digital elevation model with the same resolution by combining spatial resampling technology; 2) Obtain multi-temporal remote sensing and digital elevation model data of the study area, unify the projection coordinate system and crop accordingly according to the resolution, and pre-process the multi-temporal remote sensing and digital elevation model data to be consistent with the spatial range of the input data; 3) Obtain the vertical belt spectrum of vegetation in the study area, and pre-process the vertical belt spectrum of vegetation into a structured knowledge graph ontology that reflects the geographical distribution law of vegetation; if the study area already has a knowledge graph of vertical belts of vegetation, it can be directly applied; 4) Obtain vegetation type map data and remote sensing images of the study area, obtain samples of each vegetation type in the study area based on the vegetation type map data and remote sensing images, and form a sample data set of vegetation types in the study area.

[0031] S2. Select a deep learning model Select a deep learning model that can simultaneously learn the features of remote sensing images from different sources and with different resolutions and has multiple channels.

[0032] S3. Integration of knowledge graph ontology and deep learning model in the training phase The preprocessed data in steps 1)-2) and 4) of step S1 are input into the deep learning model for training, and the structured knowledge graph ontology and the deep learning model are integrated in the model training stage to obtain a training model for vegetation classification.

[0033] S4. Fusion of knowledge graph ontology and deep learning model in post-classification processing The structured knowledge graph ontology and the deep learning model are integrated in the post-processing stage of vegetation classification to finally obtain the vegetation classification results.

[0034] The classification method of the present invention integrating knowledge graph in the training stage is as follows: The sample data set preprocessed in step S1 is divided into a training set and a validation set, and trained using a deep learning model, while integrating the structured knowledge graph to finally obtain the vegetation classification result. During the deep learning model training process, the structured knowledge graph information is integrated, and a vegetation type is predicted by the deep learning model, and the difference between the altitude range of the corresponding vertical zone in the knowledge graph ontology and the actual altitude is compared. Based on the loss function of interval overlap, the degree of matching between the altitude distribution range of the vegetation type in the knowledge graph ontology and the actual altitude is quantified. The specific process is as follows: and Respectively represent The lower and upper limits of the altitude range of the samples in the knowledge graph ontology, Indicates The true altitude of samples, then the interval overlap loss is in the form of formula (5-1):

[0035] in, is a small positive number used to prevent the denominator from being zero; Add this constraint loss to the original cross entropy loss to form a new composite loss function, as shown in formula (5-2): (5-2)

[0036] in, is the cross entropy loss for classification tasks, It is the weight coefficient that balances the classification loss and interval overlap loss.

[0037] The classification method of the present invention that integrates the knowledge graph in the post-classification processing stage is as follows: Use the deep learning model to first perform similarity classification on the prediction results of the deep learning model in step S3 to obtain the similarity value of each category , and then integrate the structured knowledge graph ontology to calculate the similarity and obtain a new predicted similarity distribution , and obtain the final vegetation classification result according to the new similarity. For prediction results that are less than a threshold (the threshold can be adjusted according to actual conditions), the altitude of the corresponding location is obtained using DEM. , and obtain the vertical distribution information of all vegetation types in the knowledge graph ontology, and then calculate The closeness to the vertical distribution of each vegetation type is converted into a similarity distribution according to formulas (5-3) and (5-4): ,in represent The possible value range is Representative The distribution range of vegetation types in the knowledge graph ontology, Number of representative vegetation types:

[0038] (5-4)

[0039] The model predicts similarity Similarity to that calculated based on altitude Perform weighted fusion to generate a new predicted similarity distribution , as shown in formula (5-5):

[0040] in, and The model predicts similarity Similarity to that calculated based on altitude The weight of , the sum of the two is 1; Finally, based on the new similarity distribution , select the category with the highest similarity as the final model classification result.

[0041] Example 1. Overview of the study area The study area is located in Taibai Mountain in the middle of Qinling Mountains. Taibai Mountain is one of the national nature reserves and the main peak of Qinling-Daba Mountains ( Figure 1 ). Taibai Mountain is 3767.2 meters above sea level and is a typical mountain in central China. It has rich and diverse vegetation types and landscapes and is an ideal study area for vegetation classification and mapping. In order to evaluate the universality of the model and avoid overfitting, this paper selected a training area and a test area outside the study area (about 20 kilometers away from the study area) to construct a deep learning model ( Figure 1 as shown).

[0042] 2. Vertical zones of vegetation in Taibai Mountain - distribution patterns of mountain vegetation The climate of Taibai Mountain is affected by the Mongolian cold air mass in winter and the Pacific subtropical high pressure belt in summer, forming a transitional climate. With the increase of altitude, the water and heat conditions change regularly, forming five different climate zones from bottom to top: warm temperate zone, temperate zone, cold temperate zone, cold zone, and alpine cold zone. The vegetation landscape also presents a clear vertical zoning pattern, namely the mountain vegetation vertical belts (MABs) corresponding to the climate zones. Due to the climate differences between the north and south slopes of Taibai Mountain, there are also obvious differences in the distribution of vegetation on the north and south slopes. The northern slope is a typical warm temperate vertical zone spectrum with Quercus variabilis as the base belt (1350m), above which are Quercus acutissima subbelt (1350-2100m), Quercus mongolica subbelt (2100-2400m), Betula albopicta subbelt (2400-2700m) and Betula sclerocarpa subbelt (2700-2900m). The base belt of the southern slope is a deciduous broad-leaved oak forest dominated by Quercus acutissima (800m), containing evergreen broad-leaved tree species such as Cyclobalanopsis glauca and Quercus rockii, and above which are Quercus variabilis subbelt (800m), Quercus acutissima subbelt (1400-2000m), Betula albopicta subbelt (2000-2600m) and Betula sclerocarpa subbelt (2600-2800m). Above the deciduous broad-leaved forest belt are the coniferous forest belt (2800-3450m) and the shrub meadow belt (above 3450m). The coniferous forest belt is divided from bottom to top into the Bashan fir forest subzone (2800-3100m) and the Taibai redwood forest subzone (above 3100m).

[0043] 3. Data and data preprocessing (1) Remote sensing data: The multi-source, multi-temporal and multi-resolution remote sensing data used in this invention include ZY3 and GF2 satellite data (winter, 2m resolution), and GF1 satellite data (summer and winter, 16m resolution). These multi-temporal and multi-spatial resolution remote sensing data enable the deep learning model to capture the seasonal variation characteristics of vegetation. The 10-meter resolution digital elevation model (DEM) data comes from the digital surface model (DSM) product of the ZY3 satellite data, which provides altitude information for vegetation types. To facilitate data input and standardized processing, the 2-meter resolution image is cropped to 224×224 pixel data, and the 16-meter resolution remote sensing image and DEM are cropped to 1 / 8 of the size of the 2-meter image (28×28 pixel data) to ensure the consistency of the spatial range of input data of different resolutions.

[0044] (2) Knowledge graph of vertical belts of vegetation in Taibai Mountain. As mentioned above, the vertical belt distribution law of vegetation on the north and south slopes of Taibai Mountain is important information for vegetation remote sensing classification, which helps to improve the accuracy and interpretability of classification. However, the traditional vegetation vertical belt spectrum exists in the form of unstructured text or charts, which is difficult to be used by structured models or processed by computers. Therefore, this application adopts the transformation into a structured knowledge graph. The knowledge graph of vertical belts of vegetation in Taibai Mountain is constructed with the support of ArcGIS and deep learning methods, combined with vegetation distribution laws, maps and remote sensing images, etc., to define and describe the composition, characteristics and relationships of vegetation in Taibai Mountain, reflect the geographical distribution pattern of vegetation in the region, and promote the application of knowledge such as vegetation distribution laws in various disciplines. The ontology of the vegetation knowledge graph has a four-level classification system: L0 level, natural or cultivated vegetation, non-vegetation; L1 level, vegetation type group; L2 level, vegetation type or subtype; L3 level, vegetation community group, community and subcommunity. The knowledge graph of Taibai Mountain vegetation vertical zones was constructed and stored using the Neo4j ontology editing tool and described using RDF and Web Ontology Language (OWL).

[0045] (3) Vegetation type map data. The vegetation type map data used in the present invention include the "1:250,000 vegetation type map of the Qinba Mountain Area" and the "1:1 million vegetation type map of China". Supported by the "China North-South Transition Zone Comprehensive Scientific Survey" project, the present invention uses multi-source, multi-temporal high-resolution image data such as GF-1 and GF-2 and related field survey samples and literature, based on the classification system of the original 1:1 million vegetation type map, and follows the process of vegetation major category → vegetation type group interpretation → vegetation community group, community, sub-community interpretation → vegetation type, vegetation subtype interpretation. A 1:250,000 vegetation type map covering the Qinba Mountain Area is compiled through visual interpretation using a combination of top-down and bottom-up classification methods. The "1:1 million vegetation type map of China" is obtained by scanning and digitization. It is the most detailed and accurate national vegetation type map to date, and can also provide assistance for the research of the present invention.

[0046] (4) Digital surface model (DSM) data. The 1:10,000 DSM data (resolution 10 m) of Taibai Mountain was generated based on ZY-3 satellite images and used to extract fine-scale slope, aspect, altitude and surface cover data. In addition, spatial resampling technology was used to make the DEM data consistent with the 16-meter resolution remote sensing data. The DEM data was also cropped to 28×28 pixel data.

[0047] 4. Classification method (1) Deep Learning Model The present invention adopts an improved multi-channel FCN8s-ResNet50 deep learning method. The advantage of this method is that the model is modified into a four-channel structure so as to learn and extract features from multi-source remote sensing data: GF2 or ZY3 images with a resolution of 2 meters in winter, GF1 images with a resolution of 16 meters in winter and summer, and resampled DEM data with a resolution of 16 meters are input into the above four channels respectively. Since the above four-channel data have different resolutions and contain different image features, the ResNet50 model is used to train ImageNet so as to extract features from each channel separately, rather than simply stacking different data bands together. In addition, compared with other deep network models, ResNet50 has the advantages of fewer required parameters and fewer computing resources while ensuring accuracy, so it is more suitable for the present invention.

[0048] The present invention selected 24,000 samples from the Taibai Mountain area, 3,000 samples of each vegetation type in 8 pure forests. The preprocessed data set was divided into a training set and a validation set, 15,000 samples were used for training, and 9,000 samples were used to verify the model performance.

[0049] (2) Triple conversion of Taibai Mountain vegetation vertical zone knowledge graph ontology The Taibai Mountain vegetation vertical belt knowledge graph ontology was exported from Neo4j using Python, represented as a relational triple table and an attribute triple table (Table 1), and stored as CSV data. The relational triple is represented by <head node-relationship-tail node>, such as <923-belongs-922>; the attribute triple is represented by <node-attribute-value>, such as <923-upper limit-1300 meters> (922 and 923 are the IDs of the ontology nodes). By structurally representing the triples of the vegetation knowledge graph ontology, the vertical distribution pattern of each vegetation type in Taibai Mountain can be introduced into the deep learning model.

[0050] Table 1 Examples of triples in the mountain vegetation knowledge graph ontology

[0051] (3) Integration of knowledge graph ontology and deep learning model In order to study the influence of the fusion of knowledge graph ontology and deep learning model on the classification results at different stages in the remote sensing classification process, four groups of comparative experiments are set up for analysis, as shown in Table 2. The Adam optimizer is used for model training, and the loss function is set to cross entropy loss, the batch size is set to 64, the initial learning rate is set to 1e-5, the learning rate decays exponentially with the training rounds, and the number of training rounds is 150 to balance the convergence speed and generalization ability of the model.

[0052] Table 2 Comparative experiment of pure forest deep learning model integrating knowledge graph ontology

[0053] Experiment 1: Classification model without knowledge graph fusion In this experiment, a vegetation classification method based on traditional remote sensing image features was adopted. Its core algorithm framework was based on the ResNet50 deep neural network model pre-trained on the large-scale ImageNet dataset.

[0054] Experiment 2: Classification model integrating knowledge graph in model training phase This experiment adopts the method of integrating knowledge graph ontology in the training stage, aiming to integrate the advantages of deep learning model and knowledge graph and improve the accuracy of the model in mountain vegetation classification. The specific implementation process is shown in Figure 2 .

[0055] The detailed implementation steps are as follows: First, a knowledge graph of mountain vegetation vertical zones is constructed and structured and exported in the form of triples (entity-relationship-entity). This knowledge information is parsed and integrated using programming languages ​​such as Python. The knowledge graph information is integrated into the training process of deep learning models (such as the improved ResNet50). For each prediction result of the model, the knowledge graph is queried in real time to obtain the vertical distribution band spectrum range corresponding to the predicted vegetation type. At the same time, according to the actual sample labels, the altitude range of the corresponding vegetation type is retrieved from the knowledge graph. Then the loss under the constraint conditions is designed and calculated, that is, the difference between the vertical distribution range of the vegetation type predicted by the model and the actual altitude label is compared.

[0056] The model prediction will get a vegetation type and compare its altitude range with the corresponding vertical zone in the body. Its true altitude In order to quantify the matching degree between the predicted interval and the actual altitude, a loss function based on interval overlap is designed as follows: and Respectively represent The lower and upper limits of the altitude range of the samples in the knowledge graph ontology. Indicates The true altitude of samples. Then a simple form of interval overlap loss can be designed as formula (5-1):

[0057] in, is a small positive number used to prevent the denominator from being zero. This loss function measures the proportion of the non-overlapping part between the predicted type's distribution interval and the true altitude to the predicted type's distribution interval. When the true altitude is within the distribution range of the ontology type, the loss is 0, otherwise the loss increases as it moves away from the interval.

[0058] This constrained loss is added to the original cross entropy loss to form a new composite loss function, and the model parameters are updated using the back propagation algorithm to optimize the model so that it can not only learn image features but also follow the prior knowledge in the knowledge graph to achieve more accurate and reasonable mountain vegetation classification.

[0059] The composite loss function is calculated as shown in formula (5-2): (5-2)

[0060] in, is the cross entropy loss for classification tasks, It is the weight coefficient that balances the classification loss and interval overlap loss.

[0061] Experiment 3: Classification model integrating knowledge graph ontology in the post-classification processing stage In Experiment 2, a neural network classification model with ontology constraints in the training phase was established for vegetation classification. However, different embedding methods and positions of knowledge graphs may have different effects and optimization effects on the neural network model. Therefore, this experiment integrates the knowledge graph ontology ( Figure 3 as shown).

[0062] The specific steps of integrating the knowledge graph ontology during post-classification processing are as follows: First, the remote sensing images are classified using the deep learning model, and the similarity of each category is output, which is recorded as Then a threshold (such as 0.5) is set in the model, and for all categories whose predicted similarity is lower than this threshold, the model's prediction is considered to be uncertain.

[0063] For the prediction results whose similarity is lower than the threshold, get the altitude corresponding to the location , and find the vertical distribution information of all vegetation types in the knowledge graph, and then calculate The closeness to the vertical distribution of each vegetation type is converted into a similarity distribution according to formulas (5-3) and (5-4): ,in represent The possible value range is Representative The distribution range of vegetation types in the knowledge graph ontology, Represents the number of vegetation types.

[0064]

[0065] (5-4)

[0066] The model predicts similarity Similarity to that calculated based on altitude Perform weighted fusion to generate a new predicted similarity distribution , as shown in formula (5-5), where and The model predicts similarity Similarity to that calculated based on altitude The weight of , the sum of the two is 1;

[0067]

[0068] Finally, based on the new similarity , and select the category with the highest similarity as the final model prediction result. This method also utilizes the domain expert knowledge of the knowledge graph, which helps to improve the accuracy of vegetation type classification in complex environments.

[0069] Experiment 4: Classification model integrating knowledge graph in both model training and classification post-processing The optimization methods of the neural network model in Experiment 2 and Experiment 3 are integrated, and the knowledge graph ontology is integrated in both the model training stage and the classification post-processing stage to verify the final optimization effect of the knowledge graph ontology on the model. Figure 4 shown.

[0070] Accuracy Assessment When comparing and analyzing the model accuracy of the four experiments, the accuracy, recall, F1-Score and overall accuracy were calculated according to the confusion matrix to evaluate the performance of the model on the validation set. The final accuracy evaluation of vegetation remote sensing classification was verified by random point sampling. The specific method is as follows: 10 random samplings were performed in the vegetation classification results, 100 points were randomly sampled each time, and invalid points on the boundary of the study area were removed; the vegetation types of the valid points in the "1:250,000 Qinba Mountain Vegetation Type Map" and the predicted vegetation type map were obtained to obtain the sampling results; then the accuracy statistical indicators were calculated. In this invention, the overall accuracy is used to evaluate the quality of the classification results. The overall accuracy is the proportion of correctly classified samples to the total number of samples. The calculation formula is: Overall accuracy = (number of correctly classified samples / total number of samples) × 100%.

[0071] Study Results 1. Prediction results and accuracy of Experiment 1 Table 3 Accuracy evaluation of training models without knowledge graph ontology integration

[0072] Depend on Figure 5 As shown in (a) and Table 3, Experiment 1, as a benchmark experiment, has an overall accuracy of about 70.6%. The accuracy of different categories is between 0.63 and 0.83, among which the shrub and meadow have higher accuracy, both of which are 0.83, while the accuracy of the Taibai Redwood forest is the lowest, which is 0.63; the recall of different categories is between 0.59 and 0.82, among which the recall of the Pinus tabulaeformis forest is the highest, which is 0.82, while the recall of the shrub is the lowest, which is 0.67; the F1 scores of different categories are between 0.61 and 0.79, among which the meadow has the highest F1 score, which is 0.79, while the Armand pine forest has the lowest F1 score, which is 0.61.

[0073] 2. Prediction results and accuracy of Experiment 2 like Figure 5 As shown in (b) and Table 4, the overall accuracy of Experiment 2 is 75.3%, which is about 4.7% higher than the baseline experiment. Overall, the performance of this group of models has improved in most categories, and the overall average accuracy, recall, and F1 score have all increased by about 4%-5%. The accuracy of the sharp-toothed oak, Taibai redwood forest, and Armand pine forest, which had poor classification results in Experiment 1, has been significantly improved. The accuracy and recall of the sharp-toothed oak forest and Armand pine forest have increased by nearly 10%, and the F1 score has also been significantly improved.

[0074] Table 4 Accuracy evaluation of classification model integrating knowledge graph in training phase

[0075] 3. Prediction results and accuracy of Experiment 3 like Figure 5 As shown in (c) and Table 5, the overall accuracy of Experiment 3 is about 76.1%, which is about 5.5% higher than the baseline experiment. As can be seen from Table 5, the accuracy of the cork oak forest has been significantly improved, increasing by about 10%. Vegetation types with poor baseline results, such as Taibai redwood forest and Bashan fir forest, have also been significantly improved. In addition, the recall rate of shrubs has been greatly improved, but the accuracy has decreased slightly, which may be because the model sacrifices a certain degree of classification accuracy while ensuring that more shrub samples are detected.

[0076] Table 5 Accuracy evaluation of classification model integrating knowledge graph in post-classification processing stage

[0077] 4. Prediction results and accuracy of Experiment 4 Table 6 Accuracy evaluation of classification models that integrate knowledge graph ontology in both training and post-classification processing stages

[0078] like Figure 5 As shown in (d) and Table 6, the overall accuracy of Experiment 4 is about 81.5%, which is about 10.9% higher than the baseline experiment, and the improvement effect is significant. Except for shrubs and meadows, the accuracy and recall of other vegetation types have improved to varying degrees, showing stronger classification capabilities overall, and it can be seen from the confusion matrix that the confusion phenomenon has been significantly reduced. Although the accuracy of shrubs and meadows has slightly decreased, the recall rates have increased significantly, from 0.67 and 0.75 to 0.88, respectively, indicating that the model also performs better in identifying this type.

[0079] Comparative analysis of prediction results and accuracy of four groups of experiments The training process of the four groups of experiments is as follows Figure 6 As shown, by comparing the results of the four experiments (Table 7), the following conclusions can be drawn: There were significant differences in vegetation type recognition in Experiment 1. Shrubs and meadows had better classification results due to their unique remote sensing image features, while the classification results of Quercus acutissima, Larix taibaiensis, and Pinus armandii forests were poor due to their similar distribution areas and image features to other vegetation types. The overall accuracy of the model was 70.64%.

[0080] After integrating the knowledge graph ontology in the model training phase, the overall accuracy of Experiment 2 was significantly improved, especially for types with poor classification results such as Quercus acutissima, Larix taibaiensis, and Pinus armandii forests. The introduction of ontology knowledge helps the model understand and distinguish these vegetation types with similar image features and similar distribution. When the remote sensing image features are similar, the different altitudes of different vegetation distributions in the vertical band spectrum can make more accurate judgments. The overall accuracy of the model is 75.25%.

[0081] In Experiment 3, after integrating the knowledge graph ontology in the post-classification processing stage, the accuracy of some categories such as cork oak forests was significantly improved. This may be because the cork oak forest is at the bottom of the vertical band spectrum. When it is misclassified and the distribution pattern is significantly different from the vertical band spectrum, the knowledge graph ontology can be used to identify and correct it in the post-processing stage. The overall accuracy of the model is 76.08%.

[0082] After the knowledge graph ontology was integrated in both the training stage and the post-classification processing stage of Experiment 4, the overall accuracy was greatly improved, and the recognition effect of almost all vegetation types was significantly improved. This proves that the integration of the knowledge graph ontology and the deep learning model in the model training and decision-making process can greatly reduce the misclassification caused by the similarity of remote sensing image features, and can effectively use geographic knowledge such as vertical band spectrum to solve the problem of vegetation type identification in complex environments, and improve the interpretability and universality of the model. The overall accuracy of the model is 81.51%.

[0083] In general, the application of knowledge graph ontology in different stages of vegetation type classification tasks (model training and classification post-processing) can significantly improve performance, especially in solving the confusion problem between vegetation types with similar image features. This is because knowledge graphs can provide rich semantic and contextual information, making up for the shortcomings of relying solely on remote sensing image features for classification, thereby helping the model to better understand and distinguish different vegetation types.

[0084] Table 7 Prediction results and accuracy evaluation of four groups of experiments

[0085] Remote sensing classification results of vegetation types in Taibai Mountain based on Experiment 4 In order to further verify the effectiveness of the deep learning model constrained by knowledge graph ontology in vegetation classification, the deep learning model in Experiment 4 was used for vegetation remote sensing classification in Taibai Mountain area. The classification results are shown in Figure 2. Figure 7 As shown in (b) Figure 7 (a) is a 1:250,000 vegetation type map interpreted by manual visual interpretation; (b) is the vegetation remote sensing classification result integrated with the knowledge graph ontology; (c) is the remote sensing image of the study area). After 10 random samplings (100 points each time), 22 invalid points on the boundary of the study area were removed, and the number of valid results was 978. Among them, 872 results were predicted correctly, and the overall accuracy of the final vegetation classification results in the Taibai Mountain study area was 89.2%.

[0086] The principle of the present invention is: the present invention integrates the mountain vegetation knowledge graph ontology with the FCN8s-ResNet50 model to achieve high-precision classification at the vegetation community level. Under the constraints of the knowledge graph ontology, the accuracy of the deep learning model increased by 10%, and the overall classification accuracy of vegetation types reached 89.2%. At the same time, the model established in the training area also achieved high accuracy in the classification of the experimental area, indicating that the deep learning model constrained by the knowledge graph ontology has good universality, further verifying the strong potential and practical value of the knowledge graph in the classification of vegetation remote sensing images.

[0087] Experimental results show that the vegetation remote sensing classification method constrained by knowledge graph ontology not only achieves excellent results in fine classification of vegetation types, significantly improves the accuracy and stability of vegetation type recognition, but also greatly improves the universality and interpretability of the model by introducing deeper semantic information and geographic distribution knowledge, providing strong support for remote sensing applications of large-scale vegetation classification. Combining knowledge graphs with deep learning methods can achieve rapid extraction of vegetation information, updating of vegetation maps, and real-time monitoring of vegetation coverage.

[0088] The basic principles, main features and advantages of the present invention are shown and described above. The present invention may also have various changes and improvements, which all fall within the scope of the present invention claimed for protection.

Claims

1. A vegetation remote sensing classification method constrained by knowledge graph ontology, characterized in that: The following steps are involved: S1. Data acquisition and data preprocessing 1) Obtain digital surface model data of the study area and generate a digital elevation model with the same resolution by combining spatial resampling technology; 2) Obtain multi-temporal remote sensing and digital elevation model data of the study area, unify the projection coordinate system and crop accordingly according to the resolution, and pre-process the multi-temporal remote sensing and digital elevation model data to be consistent with the spatial range of the input data; 3) Obtain the vertical band spectrum of vegetation in the study area, and pre-process the vertical band spectrum of vegetation into a structured knowledge graph ontology that reflects the geographical distribution law of vegetation; 4) Obtain vegetation type map data and remote sensing images of the study area, obtain samples of each vegetation type in the study area based on the vegetation type map data and remote sensing images, and form a sample data set of the study vegetation type; S2. Select a deep learning model Select a deep learning model that can simultaneously learn the features of remote sensing images from different sources and resolutions and has multiple channels; S3. Integration of knowledge graph ontology and deep learning model in the training phase The data pre-processed in steps 1)-2) and 4) in step S1 are input into the deep learning model for training, and the structured knowledge graph ontology and the deep learning model are integrated in the model training stage to obtain a training model for vegetation classification; S4. Fusion of knowledge graph ontology and deep learning model in post-classification processing The structured knowledge graph ontology and the deep learning model are integrated in the post-processing stage of vegetation classification to finally obtain the vegetation classification results.

2. According to claim 1, a vegetation remote sensing classification method constrained by knowledge graph ontology is characterized by: The sample data set preprocessed in step S1 is divided into a training set and a validation set, and trained using a deep learning model while integrating a structured knowledge graph to finally obtain the vegetation classification result.

3. According to claim 2, a vegetation remote sensing classification method constrained by knowledge graph ontology is characterized by: During the deep learning model training process, structured knowledge graph information is integrated, and a vegetation type is predicted by the deep learning model to compare the difference between the altitude range of the corresponding vertical zone in the knowledge graph ontology and the actual altitude.

4. According to claim 3, a vegetation remote sensing classification method constrained by knowledge graph ontology is characterized by: Based on the interval overlap loss function, the matching degree between the altitude distribution range of vegetation types in the knowledge graph ontology and the actual altitude is quantified. The specific process is as follows: and Respectively represent The lower and upper limits of the altitude range of the samples in the knowledge graph ontology, Indicates The true altitude of samples, then the interval overlap loss is in the form of formula (5-1): ； in, is a small positive number used to prevent the denominator from being zero; Add this constraint loss to the original cross entropy loss to form a new composite loss function, as shown in formula (5-2): （5-2）； in, is the cross entropy loss for classification tasks, It is the weight coefficient that balances the classification loss and interval overlap loss.

5. According to claim 4, a vegetation remote sensing classification method constrained by knowledge graph ontology is characterized by: Use the deep learning model to first perform similarity classification on the prediction results of the deep learning model in step S3 to obtain the similarity value of each category , and then integrate the structured knowledge graph ontology to calculate the similarity and obtain a new predicted similarity distribution , and obtain the final vegetation classification result based on the new similarity.

6. The vegetation remote sensing classification method constrained by knowledge graph ontology according to claim 5 is characterized by: For similarity For prediction results that are less than the threshold, use DEM to obtain the altitude of the corresponding location , and obtain the vertical distribution information of all vegetation types in the knowledge graph ontology, and then calculate The closeness to the vertical distribution of each vegetation type is converted into a similarity distribution according to formulas ((5-3)) and (5-4) ,in represent The possible value range is Representative The distribution range of vegetation types in the knowledge graph ontology, Number of representative vegetation types: ； (5-4)； The model predicts similarity Similarity to that calculated based on altitude Perform weighted fusion to generate a new predicted similarity distribution , as shown in formula (5-5): ； in, and The model predicts similarity Similarity to that calculated based on altitude The weight of , the sum of the two is 1; Finally, based on the new similarity distribution , select the category with the highest similarity as the final model classification result.

Images