Method and device for accurately generating interpretable mud flat drawing
By applying a random forest classifier and a concise rule set generation method in the tidal flat map, the problems of uninterpretation of the black box algorithm and excessive feature dimensions in the existing technology are solved, and the accurate classification and accuracy of the tidal flat are achieved, providing a scientific basis for the management of tidal flats.
Patent Information
- Application Number
- CN202411779382.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-05
- Publication Date
- 2025-05-06
AI Technical Summary
In the existing mudflat mapping technology, the uninterpretationality and high feature dimensions of the black box algorithm lead to unsatisfactory classification results, and it is difficult to accurately identify the boundaries of mudflats.
Using a random forest-based machine learning method, we use low tide image maps, collect surface reflectivity data of pixel points of mudflat land objects, train a classification model, generate a concise rule set, and evaluate feature weights through interpretation rate and accuracy rate, and filter out decision rules to achieve interpretable mudflat maps.
The accurate classification of tidal flats is achieved, the feature dimension is reduced, the drawing accuracy is improved, and the overfitting phenomenon is reduced, providing a scientific basis for the management, protection and repair of tidal flats.
Smart Images
Figure CN119942168A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a tidal flat mapping technology, and in particular to a method and a device for accurately generating interpretable tidal flat mapping. Background Art
[0002] Tidal flats are the main type of land use formed by tidal action. As a protective barrier for the coastline, they can effectively reduce the impact of waves on land and even reduce the impact of extreme weather events such as storm surges on human activities. Accurate mapping of tidal flats is crucial to the management, protection and restoration of tidal flat ecosystems.
[0003] At present, the mapping of tidal flats is concentrated on fuzzy black box algorithms, while machine learning has been widely used in tidal flat mapping. In the operation, all spectral feature data are input into the model, and the classification right is entirely handed over to a black box. This is not conducive to recognizing the response of spectral features to tidal flats and using fewer features to achieve accurate mapping. Alternatively, the feature dimension must be reduced before inputting data, which will also incorrectly filter out features that have high weights in the classification but are redundant with other features.
[0004] Although there are studies at home and abroad that do not use black box algorithms and directly consider the effects of tides, focusing on the dynamic changes of tidal fluctuations on submerged areas and determining the scope of tidal flats by pixel submergence frequency, this method will mistakenly classify vegetation such as mangroves and salt marshes covering the tidal flats as tidal flats, and the classification is relatively limited. Alternatively, the tidal flat index is calculated based on a small area, which is a complex process and is only valid for specific images and specific areas, and cannot be applied to large-scale areas.
[0005] There are still few high-resolution studies on tidal flats, so the amount of information obtained for classified tidal flats is small, and the classification effect is still not ideal. In order to more accurately understand the classification information of tidal flats, it is necessary to use high-resolution images.
[0006] There is a lack of research on the interpretability of black-box classification algorithm models. Most studies explain the importance of a single feature in the model to the classification results, but are limited to surface explanations. However, the specific role of each feature and its threshold composition rules still needs further exploration. Summary of the invention
[0007] The purpose of the present invention is to overcome the deficiencies of the above-mentioned prior art, provide a method for accurately generating interpretable tidal flat mapping, solve the problem of insufficient spectral recognition of tidal flat classification, and make up for the unexplainable problem of black box classification algorithm.
[0008] To achieve the above object, the technical solution of the present invention is:
[0009] In a first aspect, the present invention provides a method for accurately generating interpretable tidal flat mapping, comprising:
[0010] Obtain low tide imagery;
[0011] Based on the acquired low tide image, the surface reflectance data of a number of beach feature pixel points are collected to obtain the beach feature pixel sample data;
[0012] Inputting the tidal flat feature pixel sample data into a preset random forest classifier to train the classification data and generate a classification model, wherein the classification model generates a concise rule set by extracting rules, merging rules, and pruning features; each rule in the concise rule set consists of a feature and a threshold;
[0013] The explanation rate of each rule in the concise rule set is counted, the accuracy of each rule classification is calculated, the explanation rate and accuracy are combined to evaluate the weight of each feature, and the decision rule is screened out; the decision rule includes subset rules composed of a single feature and a threshold;
[0014] The subset rules in the decision rules are split to provide a detailed view of the decision process for each feature and its corresponding threshold, explaining the role of each feature in accurately classifying the tidal flat.
[0015] Optionally, the classification model generates a concise rule set by extracting rules, merging rules, and pruning features, including:
[0016] Using the trained random forest model, the original set of rules is extracted from each decision tree. Each rule is a path from the root node to the leaf node of the decision tree.
[0017] Merge the rule set of one tree in the random forest with that of another tree, extract the rules whose classification accuracy reaches the threshold and delete the duplicate rules to form a new rule set, which is then merged with the rule set of the next tree, and iterate each tree;
[0018] The rules whose accuracy reaches the threshold are retained, and the sample sets that meet the rules whose accuracy reaches the threshold are deleted. The remaining sample sets are classified with the remaining rules, and the process is repeated. The features that affect the classification results are pruned and deleted to generate a concise rule set.
[0019] Optionally, the low tide images obtained are high-quality remote sensing low tide images of the tide stations and each period to ensure that the low tide images match the tide stations.
[0020] Optionally, the collecting of surface reflectance data of a plurality of beach feature pixel points based on the acquired low tide image to obtain beach feature pixel sample data includes:
[0021] Based on the acquired low tide image map, the high spectral reflectance data of n tidal flat feature pixel points are collected at different spatial positions and different vegetation coverage conditions using the visual interpretation method, where n is a positive integer; for the areas visually interpreted as tidal flats, only pure pixels that do not contain other land type pixels are collected and extracted as tidal flat feature pixel sample data.
[0022] Optionally, the explanation rate and accuracy are combined to evaluate the weight of each feature and filter out decision rules, including:
[0023] Condition 1: Extract rules with explanation rate > N% and accuracy > M% as temporary important rules;
[0024] Condition 2: Calculate the weight of each feature by combining the explanation rate and accuracy, and select the features with the top P% weights;
[0025] The features that simultaneously meet the above two conditions and their corresponding thresholds are extracted as decision rules; wherein N, M and P are all natural numbers.
[0026] Optionally, N is 50, M is 95, and P is 10.
[0027] Optionally, the weight of each feature calculated by combining the explanation rate and the accuracy is:
[0028] Farure Weight=Precision*Interpretation Rate
[0029] Where Precision is the accuracy of rule classification, and Interpretation Rate is the interpretation rate.
[0030] Optionally, a detailed view of the decision process is provided for each feature and its corresponding threshold, explaining the role of each feature in accurately classifying the tidal flat, including:
[0031] Feature importance sorting: Sort the features in the rule according to their weights;
[0032] Feature combination analysis: according to the order of feature importance, features are gradually superimposed and combined;
[0033] Sample classification visualization: Draw the distribution of correctly classified and incorrectly classified samples for each feature combination;
[0034] Feature sensitivity analysis: displays the constraint results and sensitivity of each feature to different ground object pixels.
[0035] In a second aspect, the present invention provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0036] Memory, used to store computer programs;
[0037] The processor is used to implement the steps of the method for accurately generating an interpretable tidal flat mapping as described in any of the above items when executing the program stored in the memory.
[0038] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the method for accurately generating an interpretable tidal flat map as described in any one of the above are implemented.
[0039] Compared with the prior art, the present invention has the following beneficial effects:
[0040] This application uses machine learning based on remote sensing spectral features to accurately generate an interpretable tidal flat mapping method, which can avoid repeated training of classification models, reduce the complexity of the classification process, deepen the understanding and recognition of the impact of spectra on tidal flat classification, and reveal some current implicit knowledge. Through this understanding, it is possible to accurately extract feature data related to the classification of tidal flats, reduce the dimension of the feature data input to the classification algorithm model, improve the accuracy of mapping tidal flats, and reduce the overfitting phenomenon caused by high-latitude data. It provides a scientific basis for relevant departments to overcome field measurements, quickly and accurately map tidal flats, manage, protect and restore tidal flat ecosystems, and protect microorganisms, fish, migratory birds and other marine organisms or plants and animals that use tidal flats as their habitats, and maintain the health, sustainability and carbon sequestration capacity of tidal flat ecosystems. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0043] Figure 1 A flowchart of a method for accurately generating interpretable tidal flat mapping provided in an embodiment of the present application;
[0044] Figure 2 To visualize the constraint results and sensitivity diagram of each feature on different ground object pixels;
[0045] Figure 3 A schematic diagram of the composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0046] Example:
[0047] To make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0048] See also Figure 1 As shown, the method for accurately generating interpretable tidal flat mapping provided by this embodiment mainly includes the following steps:
[0049] 110. Obtain low tide images
[0050] Specifically, in this step, the nearest tidal station and high-quality remote sensing low tide images of each period are obtained to ensure that the low tide images match the tidal stations. This step can ensure that the available images are low tide water level images, and maximize the exposed area of the tidal flats.
[0051] 120. Collecting surface reflectance data of a number of beach feature pixel points based on the acquired low tide image to obtain beach feature pixel sample data;
[0052] Specifically, in this step, based on the low tide image, the hyperspectral reflectance data of n tidal flat pixel points are collected by visual interpretation, and the tidal flats in the north and south regions are selected, including tidal flats with only bare beaches, mangroves, salt marshes, reeds and other vegetation. In other words, the sample data collected are selected to be distributed in complex areas with vegetation such as salt marshes and mangroves and single bare beach areas without vegetation, so that it can be applied to different tidal flats.
[0053] For the area interpreted as a tidal flat by visual observation, only pure pixels without other land-type pixels are collected and extracted as tidal flat ground feature pixel sample data. Feature values are constructed, including single band, band ratio, band addition, band subtraction, band normalization, texture features, and features of vegetation, soil, salinity and water-related indexes, and other band transformation forms.
[0054] 130. Input the pixel sample data of the tidal flat features into the preset random forest classifier to train the classification data and generate a classification model. The classification model generates a concise rule set by extracting rules, merging rules and pruning features; each rule in the concise rule set consists of features and thresholds. The features include single band, band ratio, band addition, band subtraction, band normalization, texture features, and features related to vegetation, soil, salinity and water indexes.
[0055] Since the random forest classifier is insensitive to dimensions and extreme values, the sample data of the tidal flat features pixel is input into the random forest training classification data to generate a classification model with more decision trees and complicated decision rules. The classification model is pruned through three steps: rule, rule merging, and feature pruning to generate a concise, accurate, and human-readable rule set. The classification model process simplifies the steps of drawing tidal flats. When using other places to classify tidal flats again, there is no need to re-extract the sample reflectance data and re-input the classification model to generate the results. The tidal flat area can be directly drawn using this concise rule set.
[0056] 140. Count the explanation rate of each rule in the concise rule set, and calculate the accuracy of each rule classification. Combine the explanation rate and accuracy to evaluate the weight of each feature rule and screen out the decision rule; the decision rule contains subset rules consisting of a single feature and a threshold. For example, rule A>0.5, B<0.6, A is the feature, >0.5 is the threshold, and A>0.5 is the subset rule in the decision rule.
[0057] In this way, by counting the explanation rate of each rule (the number of explained samples), calculating the accuracy of each rule's classification, combining the explanation rate and accuracy to evaluate the weight of each feature, the decision rules are screened out, and the complexity of the rules after the simplified model is effectively reduced, which facilitates the interpretation and recognition of the specific impact of the spectrum on the classified tidal flats, and effectively reduces the characteristics of the classified tidal flats when there is no obvious change in accuracy.
[0058] 150. Splitting the subset rules in the decision rules provides a detailed view of the decision process for each feature and its corresponding threshold, explaining the role of each feature in accurately classifying the tidal flat.
[0059] In this way, by splitting the subset rules in the key rules, a detailed view of the decision process is provided for each feature and its corresponding threshold, explaining the role of each feature in accurately classifying the tidal flats, thereby visualizing the constraint results and sensitivity of each feature to different ground feature pixels, such as Figure 2 As shown in the figure, taking a certain bay as an example, a is the land use result map, bi is the map of the constraints on landform pixels by the subset rule superposition combination, and j is the extracted tidal flat map. The correct and incorrect sample maps of each combination classification can be flexibly drawn, so there is no need for experts with a lot of relevant knowledge to explain the mechanism of drawing tidal flats by this rule set. Through visualization, the area where each subset rule restricts different ground pixels is vividly displayed, so that everyone can easily observe and understand the process and mechanism of drawing tidal flats.
[0060] In a specific embodiment, the classification model generates a rule set by extracting rules, merging rules, and pruning features, including:
[0061] Using the trained random forest model, the original rule set is extracted from each decision tree. Each rule is a path from the root node to the leaf node of the decision tree.
[0062] Merge the rule set of one tree in the random forest with that of another tree, extract the rules whose classification accuracy reaches the threshold and delete the duplicate rules to form a new rule set, which is then merged with the rule set of the next tree, and iterate each tree;
[0063] Thus, through the above two operation steps, the hierarchical structure of the decision tree is removed on the basis of tree integration, and then all possible rule sets for feature segmentation are extracted, and similar rules between decision trees are merged, and finally the rules or features with higher accuracy are retained. The above operation steps consider all possible rules, deepen the association between decision trees, reduce the errors or noise of a single tree, improve the generalization ability of recognition rules, and avoid overfitting.
[0064] The rules whose accuracy reaches the threshold are retained, the sample sets that meet the classification of the rules are deleted, the remaining sample sets are classified with the remaining rules, and the iteration is performed, and the feature rules that affect the classification results are pruned and deleted.
[0065] Since the decisions in the previously trained classification model contain a large number of redundant and overfitted rules, the rule pruning process removes most of the useless rules through the above steps, leaving only the rules with higher classification accuracy. Therefore, the accuracy of the simplified rules does not change significantly compared to the original complex decision rules, and still maintains a high accuracy.
[0066] In a specific embodiment, the step 140 includes:
[0067] Extract rules with interpretation rate > 50% and precision > 95% from the rule set as temporary important rules:
[0068] Where: TP is the number of true positives, which means the number of samples correctly classified as positive examples by the classifier;
[0069] FP is the number of false positives, which indicates the number of negative samples that are misclassified as positive samples by the classifier.
[0070]
[0071] Where Count satisfy : The number of samples that meet the rule, Count total : The total number of samples.
[0072] Next, calculate the feature weight (Farure Weight) and select the top 10% of features:
[0073] Farure Weight=Precision*Interpretation Rate
[0074] Where Precision is the probability of correct classification of the rule, Interpretation Rate is the interpretation rate
[0075] Finally, the features that satisfy both of the above conditions and their corresponding thresholds are extracted as decision rules.
[0076] In this way, through the above operations, the complexity of the rules after simplifying the model is effectively reduced, which facilitates the interpretation and understanding of the specific impact of spectral characteristics on tidal flat classification. While maintaining the classification accuracy, the number of features required for classification is reduced, and the interpretability and visualization capabilities of the model are improved.
[0077] In a specific embodiment, the detailed view of the decision process is provided for each feature and its corresponding threshold, explaining the role of each feature in accurately classifying the tidal flat, including:
[0078] Feature importance sorting: Sort the features in the rule according to their weights;
[0079] Feature combination analysis: according to the order of feature importance, features are gradually superimposed and combined;
[0080] Sample classification visualization: Draw the distribution of correctly classified and incorrectly classified samples for each feature combination;
[0081] Feature sensitivity analysis: displays the constraint results and sensitivity of each feature to different ground object pixels.
[0082] Thus, through the above operations, the ability of non-professionals to understand the classification process is improved, which promotes a deep understanding of the physical meaning of decision features, helps to reveal some undiscovered potential knowledge, and provides an intuitive basis for further optimizing the classification algorithm.
[0083] In summary, this application accurately generates an interpretable tidal flat mapping method based on machine learning of remote sensing spectral features, which can avoid repeated training of classification models, reduce the complexity of the classification process, deepen the understanding and recognition of the impact of spectra on tidal flat classification, and reveal some current implicit knowledge. Through this understanding, it is possible to accurately extract feature data related to the classification of tidal flats, reduce the dimension of the feature data input to the classification algorithm model, improve the accuracy of mapping tidal flats, and reduce the overfitting phenomenon caused by high-latitude data. It provides a scientific basis for relevant departments to overcome field measurements, quickly and accurately map tidal flats, manage, protect and restore tidal flat ecosystems, and protect microorganisms, fish, migratory birds and other marine organisms or plants and animals that use tidal flats as their habitats, and maintain the health, sustainability and carbon sequestration capacity of tidal flat ecosystems.
[0084] like Figure 3 As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114; the memory 113 is used to store computer programs; the processor 111 is used to implement the steps of the method for accurately generating an interpretable tidal flat mapping method provided by any of the aforementioned method embodiments when executing the program stored in the memory 113.
[0085] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for accurately generating an interpretable tidal flat mapping method as provided in any of the aforementioned method embodiments are implemented.
[0086] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0087] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for accurately generating interpretable tidal flat mapping, characterized in that: include: Obtain low tide imagery; Based on the acquired low tide image, the surface reflectance data of a number of beach feature pixel points are collected to obtain the beach feature pixel sample data; Inputting the tidal flat feature pixel sample data into a preset random forest classifier to train the classification data and generate a classification model, wherein the classification model generates a concise rule set by extracting rules, merging rules, and pruning features; each rule in the concise rule set consists of a feature and a threshold; The explanation rate of each rule in the concise rule set is counted, the accuracy of each rule classification is calculated, the explanation rate and accuracy are combined to evaluate the weight of each feature, and the decision rule is screened out; the decision rule includes subset rules composed of a single feature and a threshold; The subset rules in the decision rules are split to provide a detailed view of the decision process for each feature and its corresponding threshold, explaining the role of each feature in accurately classifying the tidal flat.
2. The method for accurately generating interpretable tidal flat mapping according to claim 1, characterized in that: The classification model generates a concise rule set by extracting rules, merging rules, and pruning features, including: Using the trained random forest model, the original set of rules is extracted from each decision tree. Each rule is a path from the root node to the leaf node of the decision tree. Merge the rule set of one tree in the random forest with that of another tree, extract the rules whose classification accuracy reaches the threshold and delete the duplicate rules to form a new rule set, which is then merged with the rule set of the next tree, and iterate each tree; The rules whose accuracy reaches the threshold are retained, and the sample sets that meet the rules whose accuracy reaches the threshold are deleted. The remaining sample sets are classified with the remaining rules, and the process is repeated. The features that affect the classification results are pruned and deleted to generate a concise rule set.
3. The method for accurately generating interpretable tidal flat mapping according to claim 1, characterized in that: The low tide images obtained are high-quality remote sensing low tide images of tide stations and each period to ensure that the low tide images match the tide stations.
4. The method for accurately generating interpretable tidal flat mapping according to claim 1, characterized in that: The surface reflectance data of a plurality of beach feature pixel points are collected based on the acquired low tide image to obtain beach feature pixel sample data, including: Based on the acquired low tide image map, the high spectral reflectance data of n tidal flat feature pixel points are collected at different spatial positions and different vegetation coverage conditions using the visual interpretation method, where n is a positive integer; for the areas visually interpreted as tidal flats, only pure pixels that do not contain other land type pixels are collected and extracted as tidal flat feature pixel sample data.
5. The method for accurately generating interpretable tidal flat mapping according to claim 1, characterized in that: The explanation rate and accuracy are combined to evaluate the weight of each feature and select decision rules, including: Condition 1: Extract rules with explanation rate > N% and accuracy > M% as temporary important rules; Condition 2: Calculate the weight of each feature by combining the explanation rate and accuracy, and select the features with the top P% weights; The features that simultaneously meet the above two conditions and their corresponding thresholds are extracted as decision rules; wherein N, M and P are all natural numbers.
6. The method for accurately generating interpretable tidal flat mapping according to claim 5, characterized in that: N is 50, M is 95, and P is 10.
7. The method for accurately generating interpretable tidal flat mapping according to claim 5, characterized in that: The weight of each feature is calculated by combining the explanation rate and accuracy: Farure Weight=Precision*Interpretation Rate Where Precision is the accuracy of rule classification, and Interpretation Rate is the interpretation rate.
8. The method for accurately generating interpretable tidal flat mapping according to claim 1, characterized in that: The paper provides a detailed view of the decision-making process for each feature and its corresponding threshold, explaining the role of each feature in accurately classifying mudflats, including: Feature importance sorting: Sort the features in the rule according to their weights; Feature combination analysis: according to the order of feature importance, features are gradually superimposed and combined; Sample classification visualization: Draw the distribution of correctly classified and incorrectly classified samples for each feature combination; Feature sensitivity analysis: displays the constraint results and sensitivity of each feature to different ground object pixels.
9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the steps of the method for accurately generating an interpretable tidal flat mapping as described in any one of claims 1 to 8 when executing the program stored in the memory.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that When the computer program is executed by a processor, the steps of the method for accurately generating an interpretable tidal flat mapping as described in any one of claims 1 to 8 are implemented.