Power transmission and transformation construction area soil quality evaluation method based on XGBoost and network analysis

Abstract

The invention relates to a power transmission and transformation construction area soil quality evaluation method based on XGBoost and network analysis. The method comprises the following steps: soil sample collection and index detection; constructing a minimum data set MDS; index scoring and weight determination; calculating a soil quality index SQI; and performing model evaluation and grade division: performing evaluation and optimization on soil quality index models constructed by different methods through a sensitivity index SI and an efficiency ratio ER, and performing grade division on the soil quality of the construction area according to the finally selected soil quality index. The problems that in the prior art, the subjectivity of a model is high, the nonlinear relation cannot be effectively processed, and the topography and landform of a power transmission and transformation construction area are complex can be solved.

Description

Technical Field

[0001] This application relates to the field of soil quality in power transmission and transformation construction areas, and in particular to a method for evaluating soil quality in power transmission and transformation construction areas based on XGBoost and network analysis. This method integrates machine learning, complex network analysis and traditional analysis methods, and is applicable to environmental monitoring, soil remediation assessment and sustainable land management in power construction projects. Background Technology

[0002] Soil quality is a core element for maintaining sustainable development and ensuring ecosystem security. Soils in power transmission and transformation construction areas are susceptible to erosion and have fragile ecosystems due to construction disturbances, topography, and climate conditions; therefore, accurate quality assessment is crucial.

[0003] Currently, most soil quality assessment methods are based on the Soil Quality Index (SQI). The core of this approach lies in selecting key indicators from numerous soil indicators to form a minimal dataset (MDS) and determining their weights. Existing technologies mainly suffer from the following drawbacks:

[0004] Limitations of traditional statistical methods: Principal component analysis (PCA), a widely used method, is based on linear assumptions and struggles to capture the complex nonlinear relationships between soil indicators, resulting in poor applicability in highly heterogeneous soils in power transmission and transformation construction areas. Furthermore, PCA may overestimate indicators with high variability but minor ecological functions.

[0005] Highly subjective: Relying on expert opinions to determine indicators and weights is highly subjective and lacks unified objective standards, resulting in the reproducibility of evaluation results and making it difficult to compare different studies.

[0006] Insufficient model sensitivity: The commonly used linear scoring function is not sensitive enough to changes in soil quality and cannot accurately reflect the subtle but critical changes in soil quality in the power transmission and transformation construction area.

[0007] The model has weak generalization ability: traditional static models are difficult to adapt to the soil quality evaluation needs of different types of power transmission and transformation projects, different construction stages and different geographical regions, and lack universality and flexibility.

[0008] Insufficient consideration of construction disturbance characteristics: The existing evaluation system fails to closely integrate the impact of power transmission and transformation construction activities (such as compaction, mixing, and pollution) on specific soil properties, resulting in evaluation indicators that are not specific enough and cannot accurately reflect the true soil degradation caused by construction. Summary of the Invention

[0009] The purpose of this application is to provide a method for evaluating soil quality in power transmission and transformation construction areas based on XGBoost and network analysis, so as to solve the problems of strong model subjectivity, inability to effectively handle nonlinear relationships, and complex topography in power transmission and transformation construction areas in the existing technology.

[0010] To achieve the above objectives, this application provides the following technical solution:

[0011] This application provides a method for evaluating soil quality in power transmission and transformation construction areas based on XGBoost and network analysis, including the following steps:

[0012] S1. Soil Sample Collection and Index Detection: Sampling points were set up in the power transmission and transformation construction area to collect soil samples and detect physical and chemical indicators to form the total dataset TDS;

[0013] S2. Construction of Minimal Dataset (MDS): Principal Component Analysis (PCA), Network Analysis (NA), and XGBoost machine learning methods are used to select a subset of core indicators from the total dataset, which is the minimum dataset.

[0014] S3. Indicator Scoring and Weight Determination: The selected core indicators are scored using linear or non-linear scoring functions, and weights are assigned to each core indicator using a weight determination method corresponding to each screening method.

[0015] S4. Soil Quality Index (SQI) Calculation: Based on the scores and weights of the core indicators, the soil quality index is calculated by weighted summation or arithmetic mean.

[0016] S5. Model Evaluation and Grading: Soil quality index models constructed by different methods are evaluated and optimized using the sensitivity index (SI) and efficiency ratio (ER), and the soil quality of the construction area is graded according to the finally selected soil quality index.

[0017] In step S1, the soil parameters detected specifically include:

[0018] Physical properties: Soil bulk density (BD) and capillary water holding capacity (CMC);

[0019] Chemical indicators: pH value, soil organic matter (SOM), total nitrogen (TN), available phosphorus (AP), and available potassium (AK).

[0020] In step S2, the specific steps for constructing the minimum dataset using principal component analysis include:

[0021] S2.1. Perform principal component analysis on all indicators in the total dataset and retain the principal components with eigenvalues ​​greater than 1;

[0022] S2.2. For each principal component, select the index with the highest absolute factor loading as a candidate index;

[0023] S2.3. Perform Spearman correlation analysis on the candidate indicators. If the absolute value of the correlation coefficient between two candidate indicators is greater than or equal to 0.6, retain one of them to form the principal component analysis minimum dataset PCA-MDS.

[0024] In step S2, the specific steps for constructing the minimum dataset using network analysis include:

[0025] S2.1. Construct a network graph based on the Spearman correlation coefficient matrix of all soil parameters, where nodes represent indicators and edges represent significant correlations (p < 0.05).

[0026] S2.2. Use the community detection algorithm to divide the network into modules, and divide the indicators into different functional modules;

[0027] S2.3. Calculate the eigenvector centrality of each node. In each module, select indicators whose eigenvector centrality value is greater than or equal to 0.75 and whose value is within 90% to 100% of the maximum value of that module.

[0028] S2.4. Perform correlation analysis on the selected indicators to remove redundancy, and finally form the minimum dataset for network analysis NA-MDS.

[0029] In step S2, the specific steps for constructing the minimum dataset using the XGBoost machine learning method include:

[0030] S2.1. Use a preliminary SQI (e.g., an SQI calculated based on a linear score of all indicators with equal weights) or a key soil function composite index as the target variable.

[0031] S2.2. Train the TDS using the XGBoost regression model and optimize the model hyperparameters (e.g., n_estimators, max_depth, learning_rate) through cross-validation.

[0032] S2.3. Extract the feature importance scores (based on Gain values) of each indicator from the trained model.

[0033] S2.4. Sort the indicators in descending order of importance score and calculate the cumulative importance contribution rate. Select the top few indicators with a cumulative contribution rate that reaches a preset threshold (e.g., 95%) to form XGBoost-MDS.

[0034] In step S3, the standardized scoring of the indicators uses a linear or nonlinear function:

[0035] Linear scoring function: For benefit-type indicators, an ascending function is used. For cost-related indicators, a descending order function is used:

[0036] Nonlinear scoring function:

[0037] Using S-curve functions: , where a=1, \(x_0\) is the sample mean of the indicator, b=-2.5 for benefit-type indicators, and b=2.5 for cost-type indicators.

[0038] In step S3, the weight determination method includes:

[0039] For the indicators selected by PCA, their weights Based on the factor loading of this index on its principal component Calculation of the proportion of the total factor loadings of all selected indicators: ;

[0040] For the indicators selected by network analysis, their weights Based on the eigenvector centrality of this indicator Calculation of the proportion of the sum of the eigenvector centralities of all selected indicators: ;

[0041] For the metrics selected by the XGBoost method, they are first sorted according to their gain values, and then the weights are calculated using the rank-sum ratio method: , where N is the total number of indicators and k is the ranking of the indicators.

[0042] In step S4, the formula for calculating the soil quality index is:

[0043] Weighted Soil Quality Index: ,in Let i be the weight of the i-th indicator. The score for the i-th indicator;

[0044] Unweighted Soil Quality Index: , where n is the number of indicators.

[0045] In step S5, the specific method for model evaluation is as follows:

[0046] Calculate the sensitivity index:

[0047] Computational efficiency ratio: , where K is the number of soil parameters that are significantly correlated with the soil quality index, and N is the total number of soil parameters.

[0048] In step S5, the specific method for determining the soil quality grade is as follows:

[0049] The final selected soil quality index values ​​were divided into five levels using the equal-interval method:

[0050] Grade I (Excellent): SQI > 0.8

[0051] Grade II (Good): 0.6 < SQI ≤ 0.8

[0052] Level III (Medium): 0.4 < SQI ≤ 0.6

[0053] Grade IV (Poor): 0.2 < SQI ≤ 0.4

[0054] Grade V (range): SQI ≤ 0.2.

[0055] Compared with existing technologies, the beneficial effects of this invention are: methodological integration and innovation: For the first time, principal component analysis (PCA), network analysis (NA), and XGBoost machine learning are applied in parallel to the construction of soil MDS in power transmission and transformation construction areas. This comprehensively utilizes the advantages of traditional statistics, complex systems theory, and cutting-edge artificial intelligence algorithms, and achieves complementarity and verification of index selection.

[0056] A targeted evaluation system: closely focusing on typical problems such as soil compaction, nutrient loss, and structural damage caused by power transmission and transformation construction activities, highly relevant core indicators such as BD, CMC, SOM, AP, and AK were selected to ensure the relevance and sensitivity of the evaluation.

[0057] Dynamically optimized modeling process: Through internal validation and optimization of multiple models using SI and ER, the final evaluation model is always in the optimal state, enhancing the robustness and adaptability of the method. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is an overall flowchart of the mountainous soil quality evaluation method based on XGBoost and network analysis provided in the embodiments of the present invention.

[0060] Figure 2 This is a detailed technical roadmap of constructing a minimum dataset (MDS) using three methods (PCA, NA, XGBoost) in this embodiment of the invention.

[0061] Figure 3 This is a flowchart illustrating the model evaluation and soil quality grade determination process in this embodiment of the invention. Detailed Implementation

[0062] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. It should be noted that similar reference numerals and letters in the following drawings indicate similar items; therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0063] The terms “comprising,” “including,” or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0064] The terms “first,” “second,” etc., are used only to distinguish one entity or operation from another, and should not be construed as indicating or implying relative importance, nor as requiring or implying any such actual relationship or order between these entities or operations.

[0065] like Figure 1 and Figure 2 As shown in the embodiment of this application, the following embodiment is provided: the disturbance range of the power transmission and transformation construction area is divided into several sub-regions according to the grid method. In each sub-region, the surface soil at a depth of 0-20cm is collected using the five-point method and mixed into a representative sample.

[0066] Soil index testing

[0067] Seven indicators were tested on each sample, specifically including:

[0068] Physical properties: Soil bulk density (BD), capillary water holding capacity (CMC)

[0069] Chemical indicators: pH, soil organic matter (SOM), total nitrogen (TN), available phosphorus (AP), available potassium (AK)

[0070] All tests were conducted in accordance with national or industry standard methods.

[0071] Soil bulk density: determined using the ring sampler method.

[0072] Capillary water holding capacity: determined by immersion using the ring cutter method.

[0073] pH value: determined by potentiometric method (soil-to-water ratio 1:2.5)

[0074] Soil organic matter: determined by potassium dichromate oxidation-external heating method.

[0075] Total nitrogen: determined by the Kjeldahl method.

[0076] Available phosphorus: determined by sodium bicarbonate extraction-molybdenum antimony colorimetric method.

[0077] Available potassium: determined by ammonium acetate extraction-flame photometry.

[0078] Constructing a Minimal Dataset (MDS)

[0079] S1. PCA-MDS: Principal component analysis was performed on the 7 indicators, resulting in 3 principal components with eigenvalues ​​> 1, and the cumulative variance contribution rate was 82.5%. The indicator with the highest loading was selected from each principal component, and after correlation testing, the SOM, AP, and AK indicators were finally determined to constitute the PCA-MDS.

[0080] S2. NA-MDS Construction: A soil index correlation network was constructed using Gephi software (significance level p<0.05). Modular analysis of the network identified two functional modules: a physical property module and a chemical property module. Based on eigenvector centrality (≥0.75) and correlation analysis, three core indicators—SOM, BD, and AK—were ultimately selected to constitute the NA-MDS.

[0081] S3. XGBoost-MDS Construction: Based on TDS-NL-W-SQI PCA Using the xgboost package in Python as the target variable, the XGBoost-MDS was trained. The optimized hyperparameters were: n_estimators=150, max_depth=4, learning_rate=0.1, gamma=0, colsample_bytree=0.8, min_child_weight=2, subsample=0.7. Based on feature importance (gain value), indicators with a cumulative contribution rate >95% were selected, ultimately determining SOM, AP, AK, and pH as the four indicators constituting the XGBoost-MDS.

[0082] Soil quality index calculation and model optimization (corresponding to claims 8 and 9)

[0083] S1. Calculation: According to the formula It calculates soil quality indices for each MDS model under different scoring and weighting combinations. For example, it can generate multiple model results such as MDS-LW-SQI_PCA, MDS-NL-W-SQI_NA, and MDS-NL-W-SQI_XGBoost.

[0084] S2. Preferred Result: Calculate the sensitivity index (SI) and efficiency ratio (ER) for all candidate models. In this embodiment, calculations on 120 samples show that the model based on XGBoost-MDS and nonlinear weighting (MDS-NL-W-SQI_XGBoost) has the best performance, with an SI of 18.6 and an ER of 85.7%. Therefore, this model is selected as the final evaluation model.

[0085] Soil quality classification and application

[0086] S1. Grading: The SQI values ​​of each evaluation unit calculated by the optimal model (MDS-NL-W-SQI_XGBoost) are graded according to the standard.

[0087] S2. Results and Applications: The final evaluation results show that the soil quality in the tower foundation excavation area and material storage area is mostly Grade IV (poor) and Grade V (very poor), which is highly consistent with the actual situation of heavy machinery compaction and material leakage during construction; the soil quality in the construction access road area is mostly Grade III (medium) and Grade IV (poor); the soil quality in the traction area and areas not significantly disturbed is mostly Grade II (good) and Grade III (medium). Based on these evaluation results, targeted soil restoration recommendations can be made: for Grade IV and V areas, implement enhanced remediation measures such as topsoil improvement and organic fertilizer application; for Grade III areas, implement measures such as vegetation restoration and soil structure improvement; for Grade I and II areas, implement protective measures to prevent further degradation.

[0088] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for evaluating soil quality in power transmission and transformation construction areas based on XGBoost and network analysis, characterized in that, Includes the following steps: S1. Soil Sample Collection and Index Detection: Sampling points were set up in the power transmission and transformation construction area to collect soil samples and detect physical and chemical indicators to form the total dataset TDS; S2. Construction of Minimal Dataset (MDS): Principal Component Analysis (PCA), Network Analysis (NA), and XGBoost machine learning methods are used to select a subset of core indicators from the total dataset, which is the minimum dataset. S3. Indicator Scoring and Weight Determination: The selected core indicators are scored using linear or non-linear scoring functions, and weights are assigned to each core indicator using a weight determination method corresponding to each screening method. S4. Soil Quality Index (SQI) Calculation: Based on the scores and weights of the core indicators, the soil quality index is calculated by weighted summation or arithmetic mean. S5. Model Evaluation and Grading: Soil quality index models constructed by different methods are evaluated and optimized using the sensitivity index (SI) and efficiency ratio (ER), and the soil quality of the construction area is graded according to the finally selected soil quality index.

2. The method according to claim 1, characterized in that, In step S1, the soil parameters detected specifically include: Physical properties: Soil bulk density (BD) and capillary water holding capacity (CMC); Chemical indicators: pH value, soil organic matter (SOM), total nitrogen (TN), available phosphorus (AP), and available potassium (AK).

3. The method according to claim 1 or 2, characterized in that, In step S2, the specific steps for constructing the minimum dataset using principal component analysis include: S2.

1. Perform principal component analysis on all indicators in the total dataset and retain the principal components with eigenvalues ​​greater than 1; S2.

2. For each principal component, select the index with the highest absolute factor loading as a candidate index; S2.

3. Perform Spearman correlation analysis on the candidate indicators. If the absolute value of the correlation coefficient between two candidate indicators is greater than or equal to 0.6, retain one of them to form the principal component analysis minimum dataset PCA-MDS.

4. The method according to claim 1 or 2, characterized in that, In step S2, the specific steps for constructing the minimum dataset using network analysis include: S2.

1. Construct a network graph based on the Spearman correlation coefficient matrix of all soil parameters, where nodes represent indicators and edges represent significant correlations (p < 0.05). S2.

2. Use the community detection algorithm to divide the network into modules, and divide the indicators into different functional modules; S2.

3. Calculate the eigenvector centrality of each node. In each module, select indicators whose eigenvector centrality value is greater than or equal to 0.75 and whose value is within 90% to 100% of the maximum value of that module. S2.

4. Perform correlation analysis on the selected indicators to remove redundancy, and finally form the minimum dataset for network analysis NA-MDS.

5. The method according to claim 1 or 2, characterized in that, In step S2, the specific steps for constructing the minimum dataset using the XGBoost machine learning method include: S2.1 Using soil quality-related variables as targets, the XGBoost algorithm is used to train the total dataset TDS; S2.2 Extract the feature importance scores of each indicator from the trained model; S2.3 Sort the indicators according to the feature importance score, and select the top few indicators whose cumulative importance contribution rate reaches the preset threshold to form XGBoost-MDS.

6. The method according to claim 1, characterized in that, In step S3, the standardized scoring of the indicators uses a linear or nonlinear function: Linear scoring function: For benefit-type indicators, an ascending function is used. ; For cost-related indicators, a descending order function is used: , Nonlinear scoring function: Using S-curve function: , where a=1, \(x_0\) is the sample mean of the indicator, b=-2.5 for benefit-type indicators, and b=2.5 for cost-type indicators.

7. The method according to claim 1, characterized in that, In step S3, the weight determination method includes: For the indicators selected by PCA, their weights Based on the factor loading of this index on its principal component Calculation of the proportion of the total factor loadings of all selected indicators: ; For the indicators selected by network analysis, their weights Based on the eigenvector centrality of this indicator Calculation of the proportion of the sum of the eigenvector centralities of all selected indicators: ; For the metrics selected by the XGBoost method, they are first sorted according to their gain values, and then the weights are calculated using the rank-sum ratio method: , where N is the total number of indicators and k is the ranking of the indicators.

8. The method according to claim 1, characterized in that, In step S4, the formula for calculating the soil quality index is: Weighted Soil Quality Index: ,in Let i be the weight of the i-th indicator. The score for the i-th indicator; Unweighted Soil Quality Index: , where n is the number of indicators.

9. The method according to claim 1, characterized in that, In step S5, the specific method for model evaluation is as follows: Calculate the sensitivity index: , Computational efficiency ratio: , where K is the number of soil parameters that are significantly correlated with the soil quality index, and N is the total number of soil parameters.

10. The method according to claim 1, characterized in that, In step S5, the specific method for determining the soil quality grade is as follows: The final selected soil quality index values ​​were divided into five levels using the equal-interval method: Grade I (Excellent): SQI > 0.8 Grade II (Good): 0.6 < SQI ≤ 0.8 Level III (Medium): 0.4 < SQI ≤ 0.6 Grade IV (Poor): 0.2 < SQI ≤ 0.4 Grade V (range): SQI ≤ 0.2.

Images