DNDC-based machine learning feature extraction method and system

Abstract

The invention relates to the technical field of nitrogen emission prediction, and discloses a DNDC-based machine learning feature extraction method and system, and the method comprises the steps: firstly extracting climate, soil and farmland management original features from agricultural ecosystem data; calculating an intermediate process variable related to the nitrogen cycle based on DNDC; training a machine learning reference model according to the original features to obtain a reference performance index value; combining each intermediate variable with the original features to construct an extended data set and train a model, and calculating an extended performance index value; by comparing the two performance indexes, screening intermediate variables which significantly improve the prediction effect to form an important variable set; and finally, combining the original features and the important variable set to train a machine learning model to obtain a high-precision nitrogen emission prediction model. According to the method, through fusion of a mechanism model and machine learning, dual improvement of prediction precision and interpretability is realized.

Description

Technical Field

[0001] This invention relates to the field of nitrogen emission prediction technology, specifically to a machine learning feature extraction method and system based on DNDC. Background Technology

[0002] With the continued growth of the global population, the conflict between food security and the ecological environment is becoming increasingly acute. Nitrogen fertilizer, as a key element for increasing crop yields, is applied in large quantities. However, the incomplete transformation of nitrogen fertilizer in agricultural ecosystems makes it the largest anthropogenic source of nitrous oxide (N2O) and ammonia (NH3) emissions, leading to problems such as global warming, stratospheric ozone depletion, soil acidification, and eutrophication of water bodies. Therefore, there is an urgent need for a regionally scalable method for predicting and optimizing N2O and NH3 emissions that can balance food security and the ecological environment.

[0003] The nitrogen cycle in agricultural ecosystems involves complex biogeochemical processes, including nitrification, denitrification, ammonia volatilization, and leaching loss. Its dynamic changes are influenced by multiple factors, such as climate conditions (temperature, precipitation), soil properties (texture, organic matter, pH), water availability, and farmland management practices (fertilizer type, application rate, and method), exhibiting high nonlinearity and uncertainty. This complexity presents two core challenges to nitrogen emission prediction: first, the complexity of the mechanistic processes, making it difficult for traditional models to fully characterize the dynamic mechanisms under the interaction of multiple factors; and second, the uncertainty of the data, as field observation data often suffers from limited sample size and strong spatiotemporal heterogeneity, thus limiting the accuracy of model predictions.

[0004] Currently, there are two main technical approaches in the field of nitrogen emission prediction, but both have significant drawbacks:

[0005] 1. Mechanistic Process Models: These models (such as DNDC (DeNitrification-DeComposition), a key biogeochemical model simulating the mechanisms of carbon and nitrogen cycling in terrestrial ecosystems (especially agricultural systems), predict greenhouse gas (N2O, CO2, etc.) emissions and nitrogen loss by quantifying microbial-driven biogeochemical processes (such as denitrification and decomposition)). Based on the understanding of the biological and chemical mechanisms of nitrogen transformation, they reflect the core processes and causal relationships of the nitrogen cycle and possess strong physical interpretability. However, they have significant limitations: numerous model parameters and strong dependence on input data; some parameters are difficult to obtain directly through observation and require empirical estimation, leading to a significant impact on prediction accuracy due to parameter errors; fixed model structure, making it difficult to adapt to the specificities of different regions and crop types, resulting in weak scalability; complex computational processes and low operating efficiency, making it difficult to meet the needs of large-scale, high spatiotemporal resolution predictions.

[0006] 2. Purely data-driven machine learning models (such as random forests, XGBoost, and neural networks) construct predictive relationships by mining statistical patterns in observed data, achieving high prediction accuracy when the sample size is sufficient. However, their drawbacks are: the model is essentially a "black box" structure, only reflecting the correlation between input and output, unable to reveal the underlying mechanisms of nitrogen emissions, lacking physical interpretability, and difficult to support mechanism-based management decisions; they rely excessively on data quality, and are prone to overfitting when the sample size is insufficient or the data distribution is uneven, resulting in poor generalization ability; feature engineering relies on human experience, and if the input features fail to effectively capture the key processes of the nitrogen cycle, the physical rationality of the model's predictions will be insufficient.

[0007] In summary, existing technologies cannot simultaneously achieve both prediction accuracy and mechanism interpretability, and therefore cannot meet the practical needs for accurate prediction and scientific management of nitrogen emissions. Summary of the Invention

[0008] In view of this, and to address the shortcomings of existing technologies in nitrogen emission prediction that struggle to simultaneously achieve both prediction accuracy and mechanistic interpretability, this invention provides a machine learning feature extraction method and system based on DNDC (Data-Driven Conversion). It creatively proposes a hybrid modeling framework that combines mechanistic-driven and data-driven models, enabling in-depth mining of the value of intermediate computational processes in mechanistic models. Key intermediate variables generated are dynamically introduced into the training process of the machine learning model as candidate explanatory features, providing a domain-mechanistic and understandable interpretive path for the model's prediction results.

[0009] In a first aspect, the present invention provides a machine learning feature extraction method based on DNDC, comprising:

[0010] Extract multiple raw features from nitrogen cycle-related data in agricultural ecosystems, including climate data, soil properties, and farmland management practices.

[0011] DNDC calculations are used to determine multiple intermediate process variables related to nitrogen emission prediction targets;

[0012] The original features are combined with each intermediate process variable to construct multiple extended datasets;

[0013] The original features are used to train a machine learning benchmark model, and the benchmark performance index value is calculated;

[0014] Machine learning models were trained using each extended dataset, and corresponding extended data performance metrics were calculated. Based on the comparison between the baseline performance metrics and the extended data performance metrics, intermediate process variables that significantly improved performance were selected and added to the set of important variables.

[0015] The final dataset is constructed using the original features and the set of important variables, and a machine learning model is trained based on the final dataset to obtain a high-precision nitrogen emission prediction model.

[0016] The DNDC-based machine learning feature extraction method provided in this invention extracts intermediate process variables related to nitrogen emissions from DNDC and combines them with a dynamic threshold screening mechanism. This not only supplements the machine learning model with mechanistic features with clear physical meaning, effectively improving the accuracy of nitrogen emission prediction, but also breaks through the "black box" limitation of pure data-driven models, clearly revealing the causal chain of the nitrogen cycle, while avoiding interference from redundant variables and enhancing the model's generalization ability. It balances versatility and flexibility, can be adapted to various machine learning models and mechanistic models, and can be efficiently integrated into the modeling process. It balances data dependence and mechanistic understanding, providing an understandable interpretation path based on domain mechanisms for the model's prediction results, and providing strong support for accurate prediction and scientific management of agricultural nitrogen emissions.

[0017] In one alternative implementation, the original characteristics include: temperature, precipitation, planting type, soil organic carbon, total nitrogen, soil clay content, soil cation exchange capacity, bulk density, fertilizer application rate, pH value, fertilizer type, and fertilizer application location.

[0018] The climate data, such as temperature and precipitation, covered by the original features in this embodiment of the invention directly reflect the key environmental driving factors affecting nitrogen transformation; soil properties, such as soil organic carbon, total nitrogen, and soil clay content, can accurately characterize the soil's ability to adsorb, retain, and transform nitrogen, and are the core carrier attributes of the nitrogen cycle process; and farmland management features, such as fertilizer application rate, type, and location, are directly related to nitrogen input sources and their distribution in the soil, and are key anthropogenic factors regulating nitrogen emissions. These twelve categories of original features comprehensively cover the three core dimensions affecting nitrogen emissions in agricultural ecosystems: climate, soil, and management. This ensures the complete capture of external driving factors of the nitrogen cycle and lays a solid data foundation for subsequent integration of intermediate variables into the mechanistic model. This allows the initial machine learning model to establish a basic correlation with the nitrogen emission prediction target based on solid fundamental features, providing a reliable starting point for improving model accuracy and interpretability through iterative screening of intermediate variables.

[0019] In one optional implementation, the intermediate process variables include at least one of the following: evapotranspiration reduction due to soil moisture, soil water flow per unit area, corrected temperature, carbon pool decomposition rate, ammonia adsorption, daily N2O emissions, daily NH4 emissions, relative growth rate of denitrifying bacteria, N2O consumption, nitrogen assimilation rate, soil N2 emissions, and soil N2O emissions.

[0020] The intermediate process variables provided in this invention are all derived from the DNDC's characterization of key biogeochemical processes in the nitrogen cycle. These variables accurately capture the core mechanisms behind nitrogen emission prediction targets. Introducing these variables into the model serves two purposes: firstly, they supplement the original features, providing richer process information to the machine learning model and effectively compensating for the limitations of the original features, which only reflect static states, thus helping the model to understand the intrinsic logic of nitrogen emissions more deeply; secondly, because these variables have clear physical, chemical, or biological significance, they significantly enhance the physical interpretability of the model, enabling the model's prediction results to not only have high accuracy but also be traceable to specific nitrogen cycle processes (such as denitrification intensity and ammonia volatilization mechanisms), thereby providing a mechanism-based scientific basis for optimizing farmland nitrogen management measures. Simultaneously, combined with an iterative screening mechanism, variables that significantly improve model performance can be retained, further enhancing prediction accuracy and avoiding redundant information interference.

[0021] In one alternative implementation, grid search and 10-fold cross-validation are used to optimize the model hyperparameters during the training of the machine learning model.

[0022] The grid search method in this invention systematically explores the model parameter space by traversing preset hyperparameter combinations, avoiding the subjectivity and limitations of manual hyperparameter selection based on experience. This helps find better parameter configurations and lays the foundation for model performance. Ten-fold cross-validation randomly divides the training data into ten equal parts, using nine parts to train the model and one part to validate it, repeating this process ten times and taking the average performance as the evaluation result. This effectively reduces the impact of data partitioning randomness on model evaluation, more stably reflecting the model's generalization ability on different data subsets and reducing the risk of overfitting. The combination of these two methods in hyperparameter optimization ensures that the selected hyperparameters not only perform well on specific training subsets but also maintain stable performance on unseen data, making the performance metrics of the initial machine learning model more reliable. This provides an accurate reference for the iterative selection of intermediate variables, avoiding performance distortion due to improper hyperparameter settings, which could affect the judgment of the importance of intermediate variables and ultimately guarantee the generation of a high-precision prediction model.

[0023] In one optional implementation, the step of selecting intermediate process variables that significantly improve performance and adding them to the important variable set based on the comparison results of the baseline performance index value and the extended data performance index value includes:

[0024] Intermediate process variables whose difference between extended data performance index and benchmark performance index exceeds a preset threshold are identified as intermediate process variables that significantly improve performance, and these intermediate process variables that significantly improve performance are added to the important variable set.

[0025] This invention, by setting a preset threshold, includes only intermediate process variables whose difference between the extended data performance index value and the baseline performance index value exceeds the threshold in the important variable set. This allows for the accurate identification of key variables that substantially improve model performance. This targeted screening mechanism, on the one hand, avoids redundant intermediate variables from entering the model, reducing data dimensionality and computational costs, lowering the risk of overfitting, and ensuring a lightweight model with stable generalization ability. On the other hand, by quantifying the performance improvement, it strictly controls the quality of the included variables, ensuring that the final dataset contains both the basic information of the original features and integrates the mechanistic process variables that truly enhance predictive ability. Thus, while improving the model's prediction accuracy, it strengthens the model's physical interpretability by retaining key intermediate variables with clear physical meaning, laying a reliable foundation for the generation of subsequent high-precision prediction models.

[0026] In one optional implementation, the prediction target includes at least one of N2O emissions and NH3 volatilization; the machine learning model includes any one of random forest, XGBoost, TabPFN, gradient boosting tree, neural network, and support vector machine.

[0027] In terms of prediction targets, this invention explicitly covers N2O emissions and NH3 volatilization, encompassing the main pathways of nitrogen loss in agricultural ecosystems. These targets are not only core concerns of farmland nitrogen management (directly related to the greenhouse effect, air pollution, and eutrophication), but also key outputs of the nitrogen cycle characterized by DNDC, ensuring the intrinsic correlation between the prediction targets and intermediate variables in the mechanistic model. This allows the method to specifically address the diverse environmental issues related to agricultural nitrogen emissions. Regarding the selection of machine learning models, it is compatible with various mainstream algorithms such as random forests, XGBoost, and TabPFN. This retains the ability of tree models to capture nonlinear relationships (such as the advantage of random forests in handling soil-climate interactions) while incorporating the fitting capabilities of neural networks for complex patterns. This flexibility allows the method to select the optimal model based on actual conditions such as data scale and feature distribution, avoiding dependence on specific algorithms and enhancing its applicability in different agricultural scenarios (such as different crop types and different climate zones).

[0028] Secondly, the present invention provides a machine learning feature extraction system based on DNDC, the system comprising:

[0029] The raw feature extraction module is used to extract multiple raw features from nitrogen cycle-related data of agricultural ecosystems, including climate data, soil properties, and farmland management practices.

[0030] The intermediate process variable calculation module is used to calculate multiple intermediate process variables related to the nitrogen emission prediction target based on DNDC.

[0031] The benchmark performance index acquisition module is used to train a machine learning benchmark model using the original features and calculate the benchmark performance index value.

[0032] The extended data performance index value acquisition module is used to combine the original features with each intermediate process variable to construct multiple extended datasets, train machine learning models using each set of extended datasets, and calculate the corresponding extended data performance index values.

[0033] The important variable screening module is used to select intermediate process variables that significantly improve performance and add them to the important variable set based on the comparison results of the baseline performance index value and the extended data performance index value.

[0034] The nitrogen emission prediction model training module is used to construct a final dataset using the original features and the set of important variables, and to train a machine learning model based on the final dataset to obtain a high-precision nitrogen emission prediction model.

[0035] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the DNDC-based machine learning feature extraction method described in the first aspect or any corresponding embodiment thereof.

[0036] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the DNDC-based machine learning feature extraction method described in the first aspect or any corresponding embodiment thereof.

[0037] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the DNDC-based machine learning feature extraction method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0038] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating a machine learning feature extraction method based on DNDC according to an embodiment of the present invention.

[0040] Figure 2This is a structural block diagram of a machine learning feature extraction system based on DNDC according to an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] This embodiment provides a machine learning feature extraction method based on DNDC. Figure 1 This is a flowchart of a machine learning feature extraction method based on DNDC according to an embodiment of the present invention. The process includes the following steps:

[0044] Step S1 involves extracting multiple raw features from nitrogen cycle-related data of agricultural ecosystems, including climate data, soil properties, and farmland management practices.

[0045] Specifically, this embodiment of the invention obtained global NH3 emission data from the literature (“Fertilizer management for global ammonia emission reduction”, Nature 626, 792-798 (2024)) and global N2O emission data from the literature (“Global mapping of crop-specific emission factors highlights hotspots of nitrous oxide mitigation”, Nature Food 2, 886-893 (2021)). Based on the overlap between the two sets of data and the data collection situation, climate data, soil properties, irrigation, fertilization type, fertilization method, and fertilization amount data were selected as examples for model application training. From the above data, twelve features were obtained: temperature (tmp), precipitation (prec), crop type, soil organic carbon (SOC), total nitrogen (TN), soil clay content (Clay), soil cation exchange capacity (CEC), bulk density (BD), fertilization amount (Nrate), pH, fertilization type, and fertilization location.

[0046] Step S2 involves calculating multiple intermediate process variables related to the nitrogen emission prediction target based on DNDC.

[0047] Specifically, the predictable targets of this invention include at least one of NO emissions and NH volatilization, comprehensively covering the main pathways of nitrogen loss in agricultural ecosystems. These targets are not only core concerns of farmland nitrogen management (directly related to the greenhouse effect, air pollution, and eutrophication), but also key outputs of the nitrogen cycle characterized by DNDC, ensuring the intrinsic correlation between the predicted targets and intermediate variables in the mechanistic model. This allows the method to specifically address the diverse environmental issues related to agricultural nitrogen emissions. By introducing intermediate process variables from DNDC, including at least one of the following: evapotranspiration reduction due to soil moisture, soil water flow per unit area, corrected temperature, carbon pool decomposition rate, ammonia adsorption, daily N2O emissions, daily NH4 emissions, relative growth rate of denitrifying bacteria, N2O consumption, nitrogen assimilation rate, soil N2 emissions, and soil N2O emissions, key information reflecting the nitrogen cycle mechanism is added to the machine learning model, significantly improving the accuracy of nitrogen emission prediction and providing a more reliable decision-making basis for farmland fertilization management.

[0048] This invention uses the twelve features from the collected data to calculate the twelve sets of DNDC intermediate process variables V (if data is missing, 1 is used directly as a substitute). This allows the machine learning model to learn the mechanism of DNDC even when data is partially missing, thereby improving the model's prediction accuracy. The calculation process for each intermediate process variable is as follows:

[0049] 1. The reduction in evapotranspiration E due to soil moisture is calculated using the following formula:

[0050]

[0051] Where E0 represents potential evapotranspiration; θ(%) represents soil moisture content, calculated using rainfall; θ fc (%) indicates the soil moisture content in the field; θ wp (%) indicates the soil moisture content at the point where the plant wilts. Where θ fc (%) and θ wp (%) can be obtained by looking up the soil clay content (Clay) in the original features and then finding the corresponding soil type in Table 1 to obtain the θ corresponding to the soil type. fc (%) and θ wp (%).

[0052] Table 1

[0053] Soil type Clay (%) <![CDATA[θ fc (%)]]> <![CDATA[θ wp (%)]]> sand grains 3 32 12 loamy sand 6 30 12 Sandy loam soil 9 45 21 Silty loam 14 46 34 loam 19 55 27 Sandy clay loam soil 28 59 35 Silty clay loam 34 66 40 clay loam soil 34 76 49 sandy clay 43 68 47 Silty clay 49 75 52 clay 63 77 55 organic matter 6 66 40

[0054] 2. The water flow rate Q per unit area of ​​soil is calculated using the following formula:

[0055] Q = -K × h / z

[0056] Where Q represents the water flow per unit area of ​​soil; h represents the average hydraulic conductivity of soil; z represents the soil depth (cm); and K represents the thermal conductivity of soil.

[0057] 3. The carbon pool decomposition rate dCdt is calculated using the following formula:

[0058] dCdt=μ CLAY ·μ CN ·μ tm ·(S·k1+(1-S)·k r )

[0059] Where S represents the unstable portion of the organic C compound in the pool; μ tm Indicates the combined temperature and humidity reduction coefficient; μ CLAY The clay content reduction factor is {=log(.14 / CLAY)+1}; μ CN The C:N ratio reduction factor is represented by {=0.2+7.2 / (CP / NP)}; k1 represents the specific decomposition rate of the unstable component, k r Indicates the specific decomposition rate of the resistant component;

[0060] 4. The corrected temperature NewTem is calculated using the following formula:

[0061] NewTem = E·Q·q

[0062] Where q represents soil heat flux; E represents the reduction in evapotranspiration due to soil moisture; and Q represents the water flow per unit area of ​​soil.

[0063] 5. The ammonia adsorption FIXNH4 is calculated using the following formula:

[0064] FIXNH4=[0.41-0.47·log(NH4)]·(CLAY / CLAY MAX )

[0065] Wherein, CLAY represents the soil clay content; CLAY max The maximum clay fraction in the model soil is represented by 0.63; NH4 represents the NH4+ concentration in the soil solution.

[0066] 6. Daily N2O emissions, N2O_pre, are calculated using the following formula:

[0067] N2O_pre=(0.0014·NH4 / 30.0)·(0.54+0.51·T) / 15.8

[0068] Where T represents time (days).

[0069] 7. The daily NH4 emission, NH4_pre, is calculated using the following formula:

[0070] NH4_pre=NewTem·dCdt

[0071] Where NewTem represents the correction temperature; dCdt represents the carbon pool decomposition rate.

[0072] 8. Relative growth rate (μ) of denitrifying bacteria DN It is calculated using the following formula:

[0073] μ DN =μ t,dn ·(μ NO3 ·μ PHNO3 +μ NO2 ·μ PHNO2 +μ N2O ·μ PHN2O )

[0074] Where, μ NxOy Indicates the relative growth rate of denitrifying bacteria of NO3-, NO2-, or N2O;

[0075] μ PHN2O =7.22·(pH-4.4) / 18.8; μ PHNO3 =7.14·(pH-3.8) / 22.8; μ PHNO2 =1.0;

[0076] Where, μ PHNxOy This represents the pH reduction factor for NO3-, NO2-, or N2O.

[0077] 9. The consumption of N2O is calculated using the following formula:

[0078] dN2O=(μ N2O / Y N2O +M N2O ·N2O / N)·B(t)·μ PHN2O ·μ t,dn

[0079] Among them, Y NxOy B(t) represents the maximum growth yield of NO3-, NO2-, and N2O; B(t) represents the total biomass of denitrifying bacteria; M NxOy This represents the maintenance coefficient of NO3-, NO2-, or N2O.

[0080] 10. Nitrogen assimilation rate dNdt arm It is calculated using the following formula:

[0081] dNdtarm = (dB / dy) g ·(1 / CNR DN )

[0082] Among them, CNR DN This indicates the C / N ratio of the denitrifying bacteria (3.45); (dB / dy) g This indicates the potential growth rate of denitrifying bacteria biomass;

[0083] 11. Soil N2 emissions PN2 are calculated using the following formula:

[0084] PN2=0.017+(0.025-0.0013·AD)·PA

[0085] Where PA represents the air fraction of total porosity; AD represents the adsorption coefficient (range = 0-2) which depends on the clay content in the soil.

[0086] 12. Soil N2O emissions (PN2O) are calculated using the following formula:

[0087] PN2O=(0.0006+0.0013·AD)+(0.013-0.005·AD)·PA

[0088] All of these intermediate process variables originate from DNDC's characterization of key biogeochemical processes in the nitrogen cycle, accurately capturing the core mechanisms behind nitrogen emission prediction targets: variables such as the reduction in evapotranspiration due to soil moisture and soil hydraulic conductivity reflect the impact of water conditions on nitrogen migration and transformation; corrected temperature is related to the rate of microbial activity (such as nitrification and denitrification); carbon pool decomposition rate and relative growth rate of denitrifying bacteria reflect the driving force of microbial nitrogen transformation under carbon-nitrogen coupling; ammonia adsorption, daily N2O emissions, N2O consumption, soil N2 emissions, and soil N2O emissions are directly related to the specific pathways and intensity of nitrogen emissions; and nitrogen assimilation rate reflects the absorption and utilization of nitrogen by plants.

[0089] Introducing these variables into the model serves two purposes. First, they supplement the original features, providing richer "process information" to the machine learning model. This effectively compensates for the limitations of the original features, which only reflect static states, and helps the model gain a deeper understanding of the intrinsic logic of nitrogen emissions. Second, because these variables have clear physical, chemical, or biological significance, they significantly enhance the physical interpretability of the model. This allows the model's predictions to not only be highly accurate but also traceable to specific nitrogen cycle processes (such as denitrification intensity and ammonia volatilization mechanisms), thus providing a mechanism-based scientific basis for optimizing farmland nitrogen management measures. Furthermore, combined with subsequent iterative screening mechanisms, variables that significantly improve model performance can be retained, further enhancing prediction accuracy and avoiding interference from redundant information.

[0090] Step S3: Train the machine learning model using the original features and calculate the benchmark performance index value.

[0091] Specifically, the twelve features of the original dataset are denoted as Base_Data, and a random seed is set to ensure the reproducibility of the experiment. A machine learning model (such as Random Forest, XGBoost, etc.) is trained using Base_Data, with 80% of the data used for training and 20% for validation. On the 80% data, a grid search method is used, employing ten-fold cross-validation to select the optimal hyperparameters. Benchmark performance metrics, such as R, are calculated on the validation set. 2 Value: R 2 _base.

[0092] This invention utilizes grid search to systematically explore the model parameter space by traversing preset hyperparameter combinations. This avoids the subjectivity and limitations of manual hyperparameter selection based on experience, helping to find better parameter configurations and laying the foundation for model performance. Ten-fold cross-validation randomly divides the training data into ten equal parts, using nine parts to train the model and one part to validate it, repeating this process ten times and taking the average performance as the evaluation result. This effectively reduces the impact of data partitioning randomness on model evaluation, more stably reflecting the model's generalization ability on different data subsets and reducing the risk of overfitting. The combination of these two methods in hyperparameter optimization ensures that the selected hyperparameters not only perform well on specific training subsets but also maintain stable performance on unseen data, making the performance metrics of the initial machine learning model more reliable. This provides an accurate reference for the iterative selection of intermediate variables, avoiding performance distortion due to improper hyperparameter settings, which could affect the judgment of the importance of intermediate variables and ultimately guarantee the generation of a high-precision prediction model.

[0093] In terms of machine learning model selection, the method is compatible with various mainstream algorithms such as Random Forest, XGBoost, and TabPFN. It retains the ability of tree models to capture nonlinear relationships (such as the advantage of Random Forest in handling soil-climate interactions), incorporates the fitting ability of neural networks to complex patterns, and includes models like TabPFN suitable for small sample scenarios. This flexibility allows the method to select the optimal model based on actual conditions such as data scale and feature distribution, avoiding dependence on specific algorithms and enhancing its applicability in different agricultural scenarios (such as different crop types and different climate zones).

[0094] Step S4: Combine the original features with each intermediate process variable to construct multiple extended datasets, train machine learning models using each extended dataset, and calculate the corresponding extended data performance index values.

[0095] Specifically, an extended dataset is constructed for each intermediate process variable:

[0096] Data_i = Base_Data ∪{v_i}

[0097] Where v_i represents the i-th intermediate process variable, and Data_i represents the i-th extended dataset, with 80% of the data used for training and 20% for validation. Calculate the extended data performance metrics for the validation set, for example: R... 2 :R 2 _i.

[0098] Step S5: Based on the comparison results of the baseline performance index value and the extended data performance index value, select intermediate process variables that significantly improve performance and add them to the important variable set.

[0099] The embodiments of the present invention are configured such that R 2 _i>R 2 Add v_i to the important variable set V_selected by adding _base+δ (δ is a preset threshold, which takes a value between 0.01 and 0.0001 depending on the size of the dataset and the number of features); otherwise, discard v_i.

[0100] This invention, by setting a preset threshold (δ, ranging from 0.01 to 0.0001 depending on the dataset size and number of features), includes only intermediate variables whose difference between the extended data performance index and the baseline performance index exceeds this threshold in the important variable set. This allows for the precise identification of key variables that substantially improve model performance. This targeted screening mechanism, on the one hand, avoids redundant intermediate variables from entering the model, reduces data dimensionality and computational costs, lowers the risk of overfitting, and ensures a lightweight model with stable generalization ability; on the other hand, it quantifies the performance improvement (e.g., R²). 2 The significant gain in the value can strictly control the quality of the included variables, ensuring that the final dataset contains both the basic information of the original features and integrates the mechanistic process variables that can truly enhance the predictive ability. Thus, while improving the model's prediction accuracy, it strengthens the physical interpretability of the model by retaining key intermediate variables with clear physical meaning, laying a reliable foundation for the generation of subsequent high-precision prediction models.

[0101] Step S6: Construct a final dataset using the original features and the set of important variables, and train a machine learning model based on the final dataset to obtain a high-precision nitrogen emission prediction model.

[0102] Specifically, the final dataset constructed in this invention is represented as: Data_final=Base_Data∪V_selected, with a preset initial machine learning model to obtain a high-precision nitrogen emission prediction model.

[0103] The method provided in this invention can accurately identify effective variables and, through iterative threshold (δ) filtering, retain only those relevant to R. 2 DNDC variables with significant gains are used to avoid interference from redundant variables and improve prediction accuracy. Through a dual-driven mechanism-data approach, the final model feature set equals the original observation data plus the selected key DNDC process variables (such as denitrifying enzyme activity), clearly revealing the causal chain of "fertilization → physical and chemical processes → N2O emissions".

[0104] Table 2 shows the model performance metrics R based on the NO2 test set, and Table 3 shows the model performance metrics R based on the NH3 test set, calculated using two machine learning algorithms: Random Forest and TabPFN. 2 The results of the coefficient of determination and RMSE (root mean square error), R 2 The value reflects the model's ability to interpret data. The closer the value is to 1, the better the model's fit and predictive ability. RMSE measures the average deviation between the predicted and actual values. The smaller the value, the higher the model's prediction accuracy.

[0105] Table 2

[0106] Random Forest Adding data to Random Forest TabPFN Add data to TabPFN <![CDATA[R 2 ]]> 0.76 0.86 0.81 0.82 RMSE 1.48 1.12 1.29 1.28

[0107] Table 3

[0108] Random Forest Adding data to Random Forest TabPFN Add data to TabPFN <![CDATA[R 2 ]]> 0.73 0.74 0.81 0.83 RMSE 10.61 10.48 9.38 9.27

[0109] It can be seen that the random forest model R based on the N2O test set 2 The TabPFN model was upgraded from version 0.81 (basic version) to 0.82 (data-added version), and the R model based on the NH3 test set was also improved. 2 The TabPFN model has been upgraded from version 0.81 (basic version) to 0.83 (data-added version), from version 0.73 (basic version) to 0.74 (data-added version). This upgrade, which adds data (combined with filtered effective variables and other operations), enhances the model's predictive and interpretative capabilities for targets such as NO and NH emissions, allowing the model to more accurately capture data patterns and achieving a technical improvement in accuracy.

[0110] The RMSE of the random forest model based on the N2O test set decreased from 1.48 (basic version) to 1.12 (data-added version), and the RMSE of the TabPFN model decreased from 1.29 (basic version) to 1.28 (data-added version). Similarly, the RMSE of the random forest model based on the NH3 test set decreased from 10.61 (basic version) to 10.48 (data-added version), and the RMSE of the TabPFN model decreased from 9.38 (basic version) to 9.27 (data-added version). This indicates that adding data reduced the error between the model's predicted values ​​and the actual values, improving prediction accuracy. This further validates the effectiveness of the technique in optimizing model performance and enhancing prediction accuracy, achieving the technical goal of avoiding redundant variable interference and improving prediction accuracy.

[0111] The method provided by this invention can simultaneously optimize the prediction accuracy and intrinsic interpretability of machine learning models. By introducing intermediate values ​​from mechanistic models derived from physical / chemical / biological processes, the machine learning model is provided with richer and more informative features, significantly improving its prediction accuracy. The ultimately adopted intermediate values ​​themselves have clear physical / chemical / biological significance, providing an understandable interpretation path for the model's prediction results based on domain mechanisms.

[0112] The method provided by this invention has good versatility and can be applied to various types of mechanistic models (such as ecological models, hydrological models, chemical process models, biodynamic models, etc.), as long as they can output intermediate variables with potential value. It can be applied to a wide range of machine learning models (such as random forests, gradient boosting trees, neural networks, support vector machines, etc.), and it is a standardized evaluation and screening process that is easy to automate and integrate into machine learning modeling, thereby improving efficiency.

[0113] This embodiment also provides a machine learning feature extraction system based on DNDC, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0114] This embodiment provides a machine learning feature extraction system based on DNDC, such as... Figure 2 As shown, it includes:

[0115] The raw feature extraction module 21 is used to extract multiple raw features from nitrogen cycle-related data of agricultural ecosystems, including climate data, soil properties, and farmland management practices.

[0116] Intermediate process variable calculation module 22 is used to calculate multiple intermediate process variables related to the nitrogen emission prediction target based on DNDC;

[0117] The benchmark performance index value acquisition module 23 is used to train a machine learning benchmark model using the original features and calculate the benchmark performance index value.

[0118] The extended data performance index value acquisition module 24 is used to combine the original features with each intermediate process variable to construct multiple extended datasets, train machine learning models using each set of extended datasets, and calculate the corresponding extended data performance index values.

[0119] The important variable screening module 25 is used to screen intermediate process variables that significantly improve performance and add them to the important variable set based on the comparison results of the baseline performance index value and the extended data performance index value.

[0120] The nitrogen emission prediction model training module 26 is used to construct a final dataset using the original features and the set of important variables, and to train a machine learning model based on the final dataset to obtain a high-precision nitrogen emission prediction model.

[0121] In some alternative implementations, the original characteristics include: temperature, precipitation, planting type, soil organic carbon, total nitrogen, soil clay content, soil cation exchange capacity, bulk density, fertilizer application rate, pH value, fertilizer type, and fertilizer application location.

[0122] In some alternative implementations, intermediate process variables include at least one of the following: evapotranspiration reduction due to soil moisture, soil water flow per unit area, corrected temperature, carbon pool decomposition rate, ammonia adsorption, daily N2O emissions, daily NH4 emissions, relative growth rate of denitrifying bacteria, N2O consumption, nitrogen assimilation rate, soil N2 emissions, and soil N2O emissions.

[0123] In some alternative implementations, grid search and 10-fold cross-validation are used to optimize model hyperparameters during the training of the machine learning model.

[0124] In some optional implementations, the step of selecting intermediate process variables that significantly improve performance and adding them to the important variable set based on the comparison results of baseline performance index values ​​and extended data performance index values ​​includes:

[0125] Intermediate process variables whose difference between extended data performance index and benchmark performance index exceeds a preset threshold are identified as intermediate process variables that significantly improve performance, and these intermediate process variables that significantly improve performance are added to the important variable set.

[0126] In one optional implementation, the prediction target includes at least one of N2O emissions and NH3 volatilization; the machine learning model includes any one of random forest, XGBoost, TabPFN, gradient boosting tree, neural network, and support vector machine.

[0127] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0128] In this embodiment, the DNDC-based machine learning feature extraction system is presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0129] This invention also provides a computer device having the above-described features. Figure 2 The machine learning feature extraction system based on DNDC is shown.

[0130] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 3 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 3 Take a processor 10 as an example.

[0131] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0132] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0133] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0134] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0135] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0136] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0137] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0138] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A machine learning feature extraction method based on DNDC, characterized in that, The method comprises the following steps: extracting a plurality of original features related to climate data, soil properties, and farmland management measures from nitrogen cycle related data of an agricultural ecosystem; calculating a plurality of intermediate process variables related to the nitrogen emission prediction target based on DNDC; training a machine learning benchmark model using the original features and calculating a benchmark performance indicator value; combining the original features with each intermediate process variable respectively to construct a plurality of extended data sets, training a machine learning model using each set of extended data sets respectively, and calculating corresponding extended data performance indicator values; based on the comparison result of the benchmark performance indicator value and the extended data performance indicator value, screening intermediate process variables that significantly improve performance to add to an important variable set; using the original features and the important variable set to construct a final data set, and training a machine learning model based on the final data set to obtain a high-precision nitrogen emission prediction model.

2. The method of claim 1, wherein, The original features include temperature, precipitation, planting type, soil organic carbon, total nitrogen, soil clay content, soil cation exchange capacity, bulk density, fertilizer amount, pH value, fertilizer type, and fertilizer location.

3. The method of claim 2, wherein, The intermediate process variables include at least one of soil water-induced evapotranspiration reduction, soil water flow per unit area, corrected temperature, carbon pool decomposition rate, ammonia adsorption amount, N2O daily emission amount, NH4 daily emission amount, relative growth rate of denitrifying bacteria, N2O consumption amount, nitrogen assimilation rate, soil N2 emission amount, and soil N2O emission amount.

4. The method of claim 1, wherein, The model hyperparameters are optimized using grid search and ten-fold cross-validation in the process of training the machine learning model.

5. The method according to claim 1 or 4, characterized in that, Based on the comparison result of the benchmark performance indicator value and the extended data performance indicator value, the intermediate process variables that significantly improve performance are screened to add to the important variable set, which comprises: intermediate process variables corresponding to an extended data performance indicator value greater than a preset threshold value by a benchmark performance indicator difference are taken as intermediate process variables that significantly improve performance, and the intermediate process variables that significantly improve performance are added to the important variable set.

6. The method of claim 3, wherein, The prediction target includes at least one of N2O emission amount and NH3 volatilization amount, and the machine learning model includes any one of random forest, XGBoost, TabPFN, gradient boosting tree, neural network, and support vector machine.

7. A DNDC based machine learning feature extraction system, characterized in that, The method comprises the following steps: an original feature extraction module is configured to extract a plurality of original features related to climate data, soil properties, and farmland management measures from nitrogen cycle related data of an agricultural ecosystem; an intermediate process variable calculation module is configured to calculate a plurality of intermediate process variables related to the nitrogen emission prediction target based on DNDC; a benchmark performance indicator value acquisition module is configured to train a machine learning benchmark model using the original features and calculate a benchmark performance indicator value; an extended data performance indicator value acquisition module is configured to combine the original features with each intermediate process variable respectively to construct a plurality of extended data sets, train a machine learning model using each set of extended data sets respectively, and calculate corresponding extended data performance indicator values; an important variable screening module is configured to screen intermediate process variables that significantly improve performance to add to an important variable set based on the comparison result of the benchmark performance indicator value and the extended data performance indicator value; and an important variable screening module is configured to screen intermediate process variables that significantly improve performance to add to an important variable set based on the comparison result of the benchmark performance indicator value and the extended data performance indicator value. The nitrogen emission prediction model training module is configured to construct a final data set using the original features and the important variable set, and train a machine learning model based on the final data set to obtain a high-precision nitrogen emission prediction model.

8. A computer device, comprising: The computer readable storage medium has computer instructions stored thereon, and the computer instructions are used to make a computer execute the DNDC-based machine learning feature extraction method in any one of claims 1 to 6. The computer readable storage medium has computer instructions stored thereon, and the computer instructions are used to make a computer execute the DNDC-based machine learning feature extraction method in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium has computer instructions stored thereon, and the computer instructions are used to make a computer execute the DNDC-based machine learning feature extraction method in any one of claims 1 to 6.

10. A computer program product, characterised in that, The computer readable storage medium has computer instructions stored thereon, and the computer instructions are used to make a computer execute the DNDC-based machine learning feature extraction method in any one of claims 1 to 6.

Images