Method for evaluating feature importance of basin runoff prediction based on structural effect consistency

Abstract

This invention discloses a method for evaluating the importance of features in watershed runoff prediction based on structural effect consistency. This invention relates to the field of hydrological prediction technology and includes the following steps: acquiring multi-source feature data of monitored watershed runoff, constructing a TabNet model, optimizing hyperparameter combinations to improve model performance, extracting feature mask weights, and calculating the first importance evaluation value at the model structure level; using a SHAP interpreter to calculate the contribution value of features to the prediction results, obtaining the second importance evaluation value at the prediction output effect level; sorting the first and second importance evaluation values ​​in descending order to obtain the structure rank and effect rank, and then calculating the consistency index; generating a fusion importance evaluation value for each feature data through geometric fusion calculation, effectively integrating model structure and feature effects, and providing a reliable feature importance evaluation for watershed runoff prediction.

Description

Technical Field

[0001] This invention relates to the field of hydrological prediction technology, specifically to a method for assessing the importance of watershed runoff prediction features based on the consistency of structural effects. Background Technology

[0002] In the field of watershed runoff forecasting, machine learning and deep learning-based methods have gradually become a research hotspot. Existing techniques typically rely on building complex models to process multi-source feature data to improve the accuracy of runoff rate prediction. Common methods include random forests, support vector machines, and neural networks. These models, by learning from diverse features such as historical watershed runoff data and meteorological data, can achieve a certain degree of prediction of future runoff. However, despite the good results achieved by existing techniques in terms of prediction accuracy, there are still many shortcomings in the evaluation of feature importance. Existing technologies often use a single method or metric to evaluate the importance of features, which may result in incomplete and inaccurate evaluations. For example, while mainstream models such as Random Forest, XGBoost, and Deep Neural Networks have strong predictive capabilities, their interpretability is poor. The TabNet model selects features at different decision steps through an attention mask mechanism, which has a certain degree of structural interpretability. SHAP provides the ability to quantify the contribution of features from the perspective of prediction results, but the TabNet model cannot characterize the direction and magnitude of feature changes in the output. SHAP is sensitive to feature correlation and does not reflect the internal structural information of the model. Therefore, there is an urgent need for a new method that can integrate the importance and effect contribution strength of structurally selected features in order to obtain a more robust and reliable feature ranking system.

[0003] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a method for assessing the importance of watershed runoff prediction features based on the consistency of structural effects, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for assessing the importance of watershed runoff prediction features based on the consistency of structural effects, comprising the following steps: The original multi-source feature data and corresponding runoff rates of the monitored watershed runoff over several previous time periods are obtained to form a training dataset. A TabNet model is constructed, and the original multi-source feature data is used as input samples and the corresponding runoff rates are used as labels to pre-train the TabNet model. Based on the performance parameters of the TabNet model during pre-training, the hyperparameters of the TabNet model are optimized to obtain the optimal hyperparameter combination. The TabNet prediction model is trained based on the optimal hyperparameter combination. During the training process, the feature mask weights generated for each input sample in each decision step are extracted. Based on the feature mask weights of all input samples in all decision steps, the first importance evaluation value of each feature data at the model structure level is calculated. For the trained TabNet prediction model, the SHAP interpreter is used to calculate the SHAP contribution value of each feature data in each input sample to the prediction result on the training dataset. Based on all the calculated SHAP contribution values, the second importance evaluation value of each feature data in the predicted output effect is calculated. The various feature data are sorted in descending order based on the first importance evaluation value and the second importance evaluation value, respectively, to obtain the structural rank and effect rank corresponding to each feature data. The consistency index of each feature data is calculated based on the structural rank and effect rank. According to the first importance evaluation value, the second importance evaluation value and the corresponding consistency index, the fusion importance evaluation value of each feature data is generated through geometric fusion calculation.

[0006] Furthermore, the multi-source feature data specifically includes: daily average air pressure, daily maximum temperature, daily minimum temperature, daily precipitation, daily solar radiation received, vegetation index, population density, daily evapotranspiration and sediment content within the monitoring basin; The collected original multi-source feature data are preprocessed to obtain the training dataset. The preprocessing includes missing value removal and data standardization. Specifically, missing value removal involves deleting feature data groups with missing data. Data standardization is performed on the remaining data after missing values ​​have been removed. Specifically, for any type of feature data, the feature data is normalized based on the maximum and minimum values ​​of the collected feature data. The logic behind pre-training the TabNet model is as follows: Several pre-training data sets are randomly selected from the training dataset. Each pre-training data set contains a set of multi-source feature data and corresponding runoff rate data. These pre-training data sets are used as training data for pre-training the TabNet model.

[0007] Furthermore, the performance parameter specifically refers to the root mean square error between the predicted runoff rate output by the TabNet model and the actual runoff rate during the pre-training process. The specific logic for optimizing the hyperparameters of the TabNet model is as follows: The hyperparameters of the TabNet model include learning rate, batch size, decision steps, feature representation dimension, and regularization parameter. Based on the reference hyperparameter data of previous TabNet model training, the value range of each hyperparameter is determined. Using the value range of each hyperparameter as a constraint, a data point is randomly selected within the value range of each hyperparameter and combined to form a candidate hyperparameter group. Multiple rounds of selection are performed to form several candidate hyperparameter groups. Based on the pre-trained data set, the TabNet model under different candidate hyperparameter sets is pre-trained. The root mean square error obtained by cross-validation during the pre-training process is obtained. Based on the root mean square error value, a global search is performed on the candidate hyperparameter sets of the TabNet model. The objective function is defined with minimizing the root mean square error as the optimization objective.

[0008] Furthermore, the objective function is specifically expressed as follows: In the formula, For the first Objective function values ​​for each candidate hyperparameter set. Indicates the first In cross-validation, based on the first The root mean square error is calculated from the candidate hyperparameter groups, where k is the index of the cross-validation fold. The total number of folds in cross-validation, where Indexes for candidate hyperparameter groups; Using the hyperparameters in the candidate hyperparameter group as the optimization object and the objective function as the fitness function, the fitness function value of each candidate hyperparameter group is determined by the optimization algorithm, and the candidate hyperparameter group corresponding to the minimum fitness function value is taken as the optimal hyperparameter combination.

[0009] Furthermore, the hyperparameters of the TabNet model are set based on the optimal hyperparameter combination, and the TabNet model is trained on the training dataset. The method for obtaining the feature mask weights generated for each input sample in each decision step is as follows: The TabNet model, through its sequential attention mechanism, outputs a feature selection mask corresponding to the input sample and the current decision step for each input sample (i.e., the input multi-source feature data). This mask is specifically a vector with a length equal to the number of input multi-source feature data categories, where each element's value is between 0 and 1, representing the importance of the corresponding feature data being selected and used for calculation in the current decision step. The feature mask weights generated for each input sample in each decision step are specifically represented as follows: Where i is the input sample index and j is the decision step index. This is the index of the types of feature data in the input sample, i.e. It represents the feature mask weight of the p-th feature data for the i-th input sample in the j-th decision step.

[0010] Furthermore, the specific formula used to calculate the first importance evaluation value of each feature data at the model structure level is as follows: In the formula, This is the first importance evaluation value for the p-th feature data. To input the total number of samples, This represents the total number of steps in the decision-making process. For each feature in each input sample, the SHAP contribution value to the prediction result is calculated using the SHAP interpreter.

[0011] Furthermore, the second importance evaluation value of each feature data in predicting the output effect is specifically represented by averaging the absolute values ​​of the feature SHAP values ​​of each feature data in each input sample. The specific formula used for calculation is as follows: In the formula, In the formula for the second importance evaluation value of the p-th feature data, This is the second importance evaluation value for the p-th feature data. Let be the feature SHAP value of the p-th feature data in the i-th input sample; The specific formulas used to obtain the structural rank and effect rank corresponding to each type of feature data are as follows: In the formula, The structural rank ranking of the p-th feature data. For the effect rank ranking of the p-th feature data, This is a ranking function used to sort feature data.

[0012] Furthermore, the specific formula used to calculate the consistency index of each feature based on the structural rank and effect rank is as follows: In the formula, Let p be the consistency index of the p-th feature data. Total number of feature data categories; Based on the consistency index of each feature data, the fusion importance evaluation value of each feature is generated through geometric fusion calculation. The specific logic behind this is as follows: In the formula, This represents the fusion importance evaluation value of the p-th feature data; Based on the fusion importance evaluation value of each feature data, the feature data are classified into levels according to a preset proportion range and visualized to complete the assessment of the importance of watershed runoff prediction features. The specific logic behind classifying the various feature data into levels is as follows: Based on the fusion importance evaluation value of various feature data, the feature data of different types are sorted in descending order to form an importance sequence. The top 30% of the feature data in this sequence is regarded as the first importance level, the top 30% to the top 80% of the feature data is regarded as the second importance level, and the remaining feature data is regarded as the third importance level. The importance of the feature data decreases sequentially from the importance level.

[0013] Compared with the prior art, the beneficial effects of the present invention are: To address the limitation of existing feature importance assessment methods to a single dimension, a dual evaluation mechanism is proposed. This mechanism assesses feature importance from both the model structure and prediction output effect levels. At the model structure level, the scheme extracts feature mask weights based on the TabNet model and quantifies the contribution of features to the model structure by starting from the model's internal decision-making steps using these mask weights. At the prediction output effect level, the scheme calculates the contribution of features to the prediction results based on the SHAP interpreter, revealing the impact of features on the actual runoff prediction effect. By conducting dual analysis at both the structural and effect levels, the importance of features can be assessed more comprehensively, avoiding the bias caused by a single evaluation dimension in traditional methods. This makes the feature assessment results more reliable, reflecting both the logic of model structure selection and the significance of the predictive effect. It provides a scientific basis for subsequent model optimization and feature selection, and improves the accuracy of watershed runoff prediction. Furthermore, the feature importance assessment method based on geometric fusion calculation solves the problem of the lack of an effective fusion mechanism between different evaluation indicators in the existing technology. It fully considers the comprehensive performance of features at the structural and effect levels. The geometric fusion calculation method has high robustness and high accuracy, and can provide more accurate feature importance assessment results for watershed runoff prediction models. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the overall method flow of the present invention; Figure 2 A color-mapped distribution of the consistency index and fusion importance evaluation values; Figure 3Calculate the grouping chart to integrate the importance evaluation values. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0016] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0017] Example: Please see Figure 1 The present invention provides a technical solution: A method for assessing the importance of watershed runoff prediction features based on the consistency of structural effects, comprising the following steps: Step 1: Obtain the original multi-source feature data and corresponding runoff rates of the monitored watershed runoff over several previous time periods to form a training dataset, construct a TabNet model, and pre-train the TabNet model using the original multi-source feature data as input samples and the corresponding runoff rates as labels.

[0018] The multi-source feature data specifically includes: daily average air pressure, daily maximum temperature, daily minimum temperature, daily precipitation, daily solar radiation received, vegetation index, population density, daily evapotranspiration and sediment content within the monitoring basin; The specific method for determining the monitoring basin scope is as follows: based on topography and water system distribution, the basin boundary is delineated using topographic data and a digital elevation model. Hydrological models, such as SWAT or HEC-HMS, are used to analyze the hydrological characteristics within the basin to determine its boundary scope. Combined with GIS geographic information system technology, the characteristics of rivers, lakes, and topographic relief within the basin are analyzed to ensure that the monitoring area can comprehensively cover the hydrological processes within the basin. At the same time, relevant water resource management policies and environmental protection requirements are considered to ensure that the basin scope meets the needs of management and monitoring.

[0019] The multi-source feature data listed above can be obtained through the following methods: using data released by meteorological departments, such as the China Meteorological Administration and local meteorological observatories, to obtain meteorological data such as daily average air pressure, daily maximum temperature, daily minimum temperature, daily precipitation, and daily received solar radiation; Using remote sensing data, such as MODIS and Landsat satellite imagery, and remote sensing data processing software, such as ENVI and QGIS, the vegetation index of the monitored watershed is calculated, usually based on the reflectance of the red and near-infrared bands. By obtaining statistical data from the National Bureau of Statistics or local statistical bureaus, relevant statistical yearbooks or databases are acquired, and the total population and area within the monitored watershed are calculated to obtain the population density. Daily evapotranspiration is calculated by monitoring evapotranspiration at meteorological stations or by using meteorological data models, such as the Penman-Monteith formula combined with meteorological data, or by obtaining direct measurement data through meteorological services. By setting up water quality monitoring points within the watershed and regularly collecting water samples for laboratory analysis to determine sediment content, remote sensing technology can also be used to estimate sediment concentration.

[0020] The collected original multi-source feature data are preprocessed to obtain the training dataset. The preprocessing includes missing value removal and data standardization. Specifically, missing value removal involves deleting feature data groups with missing data. Data standardization is performed on the remaining data after missing values ​​have been removed. Specifically, for any type of feature data, the feature data is normalized based on the maximum and minimum values ​​of the collected feature data. Normalization preprocessing is a conventional existing technique and will not be elaborated here. Missing values ​​can lead to biased calculation results, especially in statistical analysis or machine learning. Missing values ​​may be misinterpreted as zero or other default values, interfering with the accuracy of the results. Removing missing values ​​can simplify the dataset, retain more complete and effective features, reduce model complexity, and improve training efficiency. Different features may have different units or dimensions. Standardization can eliminate unit differences and make feature data comparable. For many machine learning algorithms, normalization can accelerate the convergence speed of the model and avoid training oscillations caused by excessive differences in feature scale. The logic behind pre-training the TabNet model is as follows: Several pre-training data sets are randomly selected from the training dataset. Generally, 30% to 50% of the data sets in the training dataset are selected as pre-training data sets. Each pre-training data set contains a set of multi-source feature data and corresponding runoff rate data. The pre-training data sets are used as training data for pre-training the TabNet model. Step 2: Based on the performance parameters of the TabNet model during pre-training, optimize the hyperparameters of the TabNet model to obtain the optimal hyperparameter combination. Train the TabNet prediction model based on the optimal hyperparameter combination. During training, extract the feature mask weights generated for each input sample in each decision step. Based on the feature mask weights of all input samples in all decision steps, calculate the first importance evaluation value of each feature data at the model structure level.

[0021] The performance parameter specifically refers to the root mean square error between the predicted runoff rate output by the TabNet model and the actual runoff rate during the pre-training process; calculating the root mean square error is a conventional technique, and the specific calculation method will not be elaborated here.

[0022] The specific logic for optimizing the hyperparameters of the TabNet model is as follows: The hyperparameters of the TabNet model include the learning rate, batch size, decision steps, feature representation dimension, and regularization parameter. Based on the reference hyperparameter data of previous TabNet model training, specifically by consulting publicly available academic papers, technical reports, or research articles, which usually list the hyperparameters used to train the TabNet model for similar problems, such as runoff prediction, time series prediction, or classification problems, the range of values ​​for each hyperparameter is determined. Using the range of values ​​for each hyperparameter as a constraint, a data point is randomly selected within the range of values ​​for each hyperparameter and combined to form a candidate hyperparameter group. Multiple rounds of selection are performed to form several candidate hyperparameter groups. Based on the pre-trained data set, the TabNet model under different candidate hyperparameter sets is pre-trained. The root mean square error obtained by cross-validation during the pre-training process is obtained. Based on the root mean square error value, a global search is performed on the candidate hyperparameter sets of the TabNet model. The objective function is defined with minimizing the root mean square error as the optimization objective.

[0023] Cross-validation is a common method for evaluating the robustness of a model. It involves splitting the dataset into... The model is trained and validated on each subset separately, which can avoid overfitting the model to a specific training set or test set. In this scheme, the performance of the candidate hyperparameter group is evaluated by cross-validation, and the average root mean square error is used as the metric to determine the optimal hyperparameter group. The objective function is specifically expressed as follows: In the formula, For the first Objective function values ​​for each candidate hyperparameter set. Indicates the first In cross-validation, based on the first The root mean square error is calculated from the candidate hyperparameter groups, where k is the index of the cross-validation fold. The total number of folds in cross-validation, where Indexes for candidate hyperparameter groups; It should be noted that, regarding the first Objective function values ​​of candidate hyperparameter sets ,when Smaller, meaning the candidate hyperparameter set A low root mean square error (RMSE) across all cross-validation tradeoffs indicates good model prediction performance, a smaller difference between predicted and true values, and that the model fits the data more accurately, exhibits better generalization ability, and maintains good performance on unseen data. A larger mean square error indicates that the hyperparameter set exhibits a high root mean square error during cross-validation, suggesting that the model's prediction error is large. Using the hyperparameters in the candidate hyperparameter group as the optimization object and the objective function as the fitness function, the fitness function value of each candidate hyperparameter group is determined by the optimization algorithm, and the candidate hyperparameter group corresponding to the minimum fitness function value is taken as the optimal hyperparameter combination.

[0024] Specifically, a genetic algorithm is used as the optimization algorithm to perform global optimization. A candidate hyperparameter group is treated as an individual, and the hyperparameters in the candidate hyperparameter group are treated as genes. The optimal hyperparameter combination is determined through iterative operations of selection, crossover, and mutation. Since the genetic algorithm is a conventional optimization algorithm, the specific iterative optimization steps will not be elaborated here.

[0025] The hyperparameters of the TabNet model are set based on the optimal combination of hyperparameters, and the TabNet model is trained on the training dataset. The method for obtaining the feature mask weights generated for each input sample in each decision step is as follows: The TabNet model, through its sequential attention mechanism, outputs a feature selection mask corresponding to the input sample and the current decision step for each input sample (i.e., the input multi-source feature data). This mask is specifically a vector with a length equal to the number of input multi-source feature data categories, where each element's value is between 0 and 1, representing the importance of the corresponding feature data being selected and used for calculation in the current decision step. The feature mask weights generated for each input sample in each decision step are specifically represented as follows: Where i is the input sample index and j is the decision step index. This is the index of the types of feature data in the input sample, i.e. Let represent the feature mask weight of the p-th feature data for the i-th input sample in the j-th decision step, where The closer the value is to 1, the more critical the contribution of the feature to the prediction of the current sample in this decision step. In practice, by registering a forward propagation hook in the trained TabNet model or by directly calling a specific interface provided by the model, such as the forward_masks method, the mask weights generated by the model in each decision step when processing all input samples can be obtained.

[0026] The specific formula used to calculate the first importance evaluation value of each feature data at the model structure level is as follows: In the formula, This is the first importance evaluation value for the p-th feature data. To input the total number of samples, This represents the total number of steps in the decision-making process.

[0027] It should be noted that, Used to represent the first importance evaluation value of the p-th feature data. A larger value indicates that the feature significantly influences the prediction results when the model makes predictions, and the model tends to use this feature in different situations. A larger value also indicates a greater influence. The value indicates that the model relies heavily on this feature when making decisions, suggesting that the feature has strong information content and discriminative power in the dataset, and can effectively help the model identify patterns and rules.

[0028] Inner Summation The calculation is performed on the 1st decision step in all decision steps for each input sample. The sum of the mask weights of the feature data represents the degree of contribution of the feature to the sample in different decision steps, taking into account the performance of the feature in multiple decision steps.

[0029] The outermost layer summation calculates all samples for the first... The sum of the importance contributions of each feature data point can be used to obtain the overall performance of that feature across the entire dataset by summing the feature mask weights of all samples. Using this formula, for the th The feature data is comprehensively evaluated using mask weights across all samples and decision steps. The feature mask weight for each sample in each decision step reflects the contribution of the feature at each step. This contribution was summarized, thus demonstrating the first The structural importance of various feature data in the entire dataset and model decision-making process.

[0030] Step 3: For the trained TabNet prediction model, use the SHAP interpreter to calculate the SHAP contribution value of each feature data in each input sample to the prediction result on the training dataset. Based on the calculated SHAP contribution values, calculate the second importance evaluation value of each feature data at the level of prediction output effect.

[0031] For each feature in each input sample to contribute to the prediction result, the SHAP value of each feature data in each input sample is calculated using the SHAP interpreter. This includes: using shap.KernelExplainer to estimate the feature SHAP value of each feature data in each sample; based on the trained TabNet model, after the model training is completed, selecting the expected prediction value output by the trained TabNet model when the input sample is selected as the background reference data; and using KernelExplainer to calculate the SHAP value of each feature data in the input sample.

[0032] The SHAP value is used to quantify the contribution of each feature to the model's prediction results. By calculating the SHAP value, the model's predictions can be made more interpretable, helping to understand the role of each feature in the prediction, thereby improving the model's transparency. The second importance rating of each feature data point in the predicted output effect is specifically represented by averaging the absolute values ​​of the feature SHAP values ​​of each feature data point in each input sample. The specific formula used for calculation is as follows: In the formula, This is the second importance evaluation value for the p-th feature data. Let be the feature SHAP value of the p-th feature data in the i-th input sample; It should be noted that when The larger the value, the more it indicates that the first... The more significant the influence of a feature on the prediction results across all input samples, the stronger its discriminative power, enabling the model to make reasonable predictions across multiple samples. Calculated the first The degree of influence of a feature on the overall prediction of the model can reflect the stability and importance of the feature in different samples. Even if the SHAP value of some feature data is negative, it will be reflected by its absolute value, thus emphasizing the important role of the feature in the prediction process. The formula is derived by taking the absolute value of the feature SHAP value for each type of feature data and then averaging it. In fact, in quantifying features The influence of a feature across all input samples is considered, regardless of its positive or negative contribution; only the magnitude of the contribution is taken into account. This allows for a more comprehensive evaluation of the role of a feature in the model's predictions.

[0033] Step 4: Sort the various feature data in descending order based on the first importance evaluation value and the second importance evaluation value respectively to obtain the structural rank and effect rank corresponding to each feature data. Calculate the consistency index of each feature data based on the structural rank and effect rank. Based on the first importance evaluation value, the second importance evaluation value and the corresponding consistency index, generate the fusion importance evaluation value of each feature data through geometric fusion calculation.

[0034] The specific formulas used to obtain the structural rank and effect rank corresponding to each type of feature data are as follows: In the formula, The structural rank ranking of the p-th feature data. For the effect rank ranking of the p-th feature data, This is a ranking function used to sort feature data.

[0035] The specific formula used to calculate the consistency index of each feature based on the aforementioned structural rank and effect rank is as follows: In the formula, Let p be the consistency index of the p-th feature data. Total number of feature data categories; It should be noted that the consistency index of the p-th feature data The value is in the range [0,1], when When, it indicates that the structural rank and the effect rank are completely consistent; when When the two are completely inconsistent, it indicates that they are not at all. If the importance assessment of a feature under different methods, such as the importance of structure and effect, is highly consistent, it indicates that the feature is very stable in the model and has relatively reliable importance.

[0036] Conversely, if the consistency index is low, it indicates that there are significant differences in the assessment of the importance of the feature, which may mean that the role of the feature in the model is not clear or consistent enough. in The calculation focuses on the absolute difference between structural rank and effect rank. This difference reflects the relative change in the importance of the same feature under different evaluation criteria. This absolute difference is calculated by dividing the total number of feature data categories. This achieves the standardization of differences; This formula converts discrepancies into a consistency index; the smaller the discrepancy, the more consistent the assessment of the importance of the feature in terms of structure and effect. The closer the value is to 1, the less likely it is to be negative.

[0037] By combining structural rank and effect rank to assess feature consistency, a more comprehensive feature analysis can be provided. This approach helps identify which features are relatively stable in the model and which feature effects may vary depending on the model or data.

[0038] Based on the consistency index of each feature data, the fusion importance evaluation value of each feature is generated through geometric fusion calculation. The specific logic behind this is as follows: In the formula, This represents the fusion importance evaluation value of the p-th feature data; It should be noted that when A larger value indicates that the feature has higher importance after comprehensively considering both structural importance and SHAP importance. Specifically, fusion importance reflects the relative influence of the feature in the model prediction and takes into account the consistency of feature evaluation. This means that the feature is considered important under different evaluation methods, and its importance is highly consistent, thus making it more reliable; The geometric mean is used to combine two importance values, effectively balancing their influence, especially when these values ​​have large scale differences. The geometric mean generally reflects the correlation between two values ​​better than the arithmetic mean, and in some cases, it better reflects the characteristics of the data. Specifically, it works by... It partially reflects the calculation method of the geometric mean; in Some are obtained through the consistency index The geometric mean results were weighted and adjusted when the consistency index was... When the consistency index approaches 1, the weight of this part is close to 1, indicating that the importance of the feature is highly consistent under the two evaluation methods, thus enhancing its overall influence. When the weight is lower, the weight is reduced accordingly, indicating that the importance of features varies greatly under different methods, thus reducing their impact on the evaluation of fusion importance.

[0039] Based on the fusion importance evaluation value of each feature data, the feature data are classified into levels according to a preset proportion range and visualized to complete the assessment of the importance of watershed runoff prediction features. The specific logic behind classifying the various feature data into levels is as follows: Based on the fusion importance evaluation value of various feature data, the feature data of different types are sorted in descending order to form an importance sequence. The top 30% of the feature data in this sequence is regarded as the first importance level, the top 30% to the top 80% of the feature data is regarded as the second importance level, and the remaining feature data is regarded as the third importance level. The importance of the feature data decreases sequentially from the importance level.

[0040] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0041] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0042] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0043] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for assessing the importance of watershed runoff prediction features based on the consistency of structural effects, characterized in that, The specific steps include: The original multi-source feature data and corresponding runoff rates of the monitored watershed runoff over several previous time periods are obtained to form a training dataset. A TabNet model is constructed, and the original multi-source feature data is used as input samples and the corresponding runoff rates are used as labels to pre-train the TabNet model. Based on the performance parameters of the TabNet model during pre-training, the hyperparameters of the TabNet model are optimized to obtain the optimal hyperparameter combination. The TabNet prediction model is trained based on the optimal hyperparameter combination. During the training process, the feature mask weights generated for each input sample in each decision step are extracted. Based on the feature mask weights of all input samples in all decision steps, the first importance evaluation value of each feature data at the model structure level is calculated. For the trained TabNet prediction model, the SHAP interpreter is used to calculate the SHAP contribution value of each feature data in each input sample to the prediction result on the training dataset. Based on all the calculated SHAP contribution values, the second importance evaluation value of each feature data in the predicted output effect is calculated. The various feature data are sorted in descending order based on the first importance evaluation value and the second importance evaluation value, respectively, to obtain the structural rank and effect rank corresponding to each feature data. The consistency index of each feature data is calculated based on the structural rank and effect rank. According to the first importance evaluation value, the second importance evaluation value and the corresponding consistency index, the fusion importance evaluation value of each feature data is generated through geometric fusion calculation.

2. The method for assessing the importance of watershed runoff prediction features based on structural effect consistency as described in claim 1, characterized in that: The multi-source feature data specifically includes: daily average air pressure, daily maximum temperature, daily minimum temperature, daily precipitation, daily solar radiation received, vegetation index, population density, daily evapotranspiration and sediment content within the monitoring basin; The collected original multi-source feature data are preprocessed to obtain the training dataset. The preprocessing includes missing value removal and data standardization. Specifically, missing value removal involves deleting feature data groups with missing data. Data standardization is performed on the remaining data after missing values ​​have been removed. Specifically, for any type of feature data, the feature data is normalized based on the maximum and minimum values ​​of the collected feature data. The logic behind pre-training the TabNet model is as follows: Several pre-training data sets are randomly selected from the training dataset. Each pre-training data set contains a set of multi-source feature data and corresponding runoff rate data. These pre-training data sets are used as training data for pre-training the TabNet model.

3. The method for assessing the importance of watershed runoff prediction features based on structural effect consistency as described in claim 2, characterized in that: The performance parameter specifically refers to the root mean square error between the predicted runoff rate output by the TabNet model and the actual runoff rate during the pre-training process. The specific logic for optimizing the hyperparameters of the TabNet model is as follows: The hyperparameters of the TabNet model include learning rate, batch size, decision steps, feature representation dimension, and regularization parameter. Based on the reference hyperparameter data of previous TabNet model training, the value range of each hyperparameter is determined. Using the value range of each hyperparameter as a constraint, a data point is randomly selected within the value range of each hyperparameter and combined to form a candidate hyperparameter group. Multiple rounds of selection are performed to form several candidate hyperparameter groups. Based on the pre-trained data set, the TabNet model under different candidate hyperparameter sets is pre-trained. The root mean square error obtained by cross-validation during the pre-training process is obtained. Based on the root mean square error value, a global search is performed on the candidate hyperparameter sets of the TabNet model. The objective function is defined with minimizing the root mean square error as the optimization objective.

4. The method for assessing the importance of watershed runoff prediction features based on structural effect consistency as described in claim 3, characterized in that: The objective function is specifically expressed as follows: In the formula, For the first Objective function values ​​for each candidate hyperparameter set. Indicates the first In cross-validation, based on the first The root mean square error is calculated from the candidate hyperparameter groups, where k is the index of the cross-validation fold. The total number of folds in cross-validation, where Indexes for candidate hyperparameter groups; Using the hyperparameters in the candidate hyperparameter group as the optimization object and the objective function as the fitness function, the fitness function value of each candidate hyperparameter group is determined by the optimization algorithm, and the candidate hyperparameter group corresponding to the minimum fitness function value is taken as the optimal hyperparameter combination.

5. The method for assessing the importance of watershed runoff prediction features based on structural effect consistency according to claim 4, characterized in that: The hyperparameters of the TabNet model are set based on the optimal combination of hyperparameters, and the TabNet model is trained on the training dataset. The method for obtaining the feature mask weights generated for each input sample in each decision step is as follows: The TabNet model, through its sequential attention mechanism, outputs a feature selection mask corresponding to the input sample and the current decision step for each input sample (i.e., the input multi-source feature data). This mask is specifically a vector with a length equal to the number of input multi-source feature data categories, where each element's value is between 0 and 1, representing the importance of the corresponding feature data being selected and used for calculation in the current decision step. The feature mask weights generated for each input sample in each decision step are specifically represented as follows: Where i is the input sample index and j is the decision step index. This is the index of the types of feature data in the input sample, i.e. It represents the feature mask weight of the p-th feature data for the i-th input sample in the j-th decision step.

6. The method for assessing the importance of watershed runoff prediction features based on structural effect consistency as described in claim 5, characterized in that: The specific formula used to calculate the first importance evaluation value of each feature data at the model structure level is as follows: In the formula, This is the first importance evaluation value for the p-th feature data. To input the total number of samples, This represents the total number of steps in the decision-making process. For each feature in each input sample, the SHAP contribution value to the prediction result is calculated using the SHAP interpreter.

7. The method for assessing the importance of watershed runoff prediction features based on structural effect consistency as described in claim 6, characterized in that: The second importance rating of each feature data point in the predicted output effect is specifically represented by averaging the absolute values ​​of the feature SHAP values ​​of each feature data point in each input sample. The specific formula used for calculation is as follows: In the formula, This is the second importance evaluation value for the p-th feature data. Let be the feature SHAP value of the p-th feature data in the i-th input sample; The specific formulas used to obtain the structural rank and effect rank corresponding to each type of feature data are as follows: In the formula, The structural rank ranking of the p-th feature data. For the effect rank ranking of the p-th feature data, This is a ranking function used to sort feature data.

8. The method for assessing the importance of watershed runoff prediction features based on structural effect consistency as described in claim 7, characterized in that: The specific formula used to calculate the consistency index of each feature based on the aforementioned structural rank and effect rank is as follows: In the formula, Let p be the consistency index of the p-th feature data. Total number of feature data categories; Based on the consistency index of each feature data, the fusion importance evaluation value of each feature is generated through geometric fusion calculation. The specific logic behind this is as follows: In the formula, This represents the fusion importance evaluation value of the p-th feature data; Based on the fusion importance evaluation value of each feature data, the feature data are classified into levels according to a preset proportion range and visualized to complete the assessment of the importance of watershed runoff prediction features. The specific logic behind classifying the various feature data into levels is as follows: Based on the fusion importance evaluation value of various feature data, the feature data of different types are sorted in descending order to form an importance sequence. The top 30% of the feature data in this sequence is regarded as the first importance level, the top 30% to the top 80% of the feature data is regarded as the second importance level, and the remaining feature data is regarded as the third importance level. The importance of the feature data decreases sequentially from the importance level.

Images