A feature selection method for traffic emission prediction
By combining feature grouping, data cleaning, and interpretability SHAP analysis, this method solves the problems of high computational cost and redundant feature impact in existing traffic emission prediction models on high-dimensional datasets, achieving efficient and accurate feature selection and improving model stability and prediction performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TONGJI UNIV
- Filing Date
- 2024-08-14
- Publication Date
- 2026-07-21
AI Technical Summary
Existing feature selection methods struggle to balance model performance, computational efficiency, and interpretability when dealing with traffic emission prediction, especially on high-dimensional datasets where computational costs are high and redundant features negatively impact model stability.
We employ a method that combines feature grouping, data cleaning, weight initialization, in-group and out-of-group correlation calculation, Tree-enhanced model, and SHAP analysis. Key features are selected through automatic and manual grouping strategies, and the feature selection process is optimized by incorporating interpretable SHAP technology.
It improves the efficiency and accuracy of feature selection, effectively filters out key factors affecting traffic emissions, solves the interaction effects of highly correlated features, and enhances the stability and predictive performance of the model.
Smart Images
Figure CN118897974B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of traffic emission prediction, and in particular relates to a feature selection method for traffic emission prediction. Background Technology
[0002] Accurate prediction and analysis of the causes of traffic emissions are crucial for developing effective emission reduction strategies. However, the factors influencing traffic emissions are numerous and complex, including vehicle type, road conditions, weather conditions, and fuel efficiency. The presence of redundant and irrelevant features not only increases the computational complexity of model training but may also reduce the model's predictive accuracy and generalization ability. Feature selection techniques are fundamentally about filtering out the most representative and influential subset from a massive dataset, eliminating non-critical and redundant elements, and thus improving model performance.
[0003] Existing feature selection methods can be categorized into three types: filtering, wrapping, and embedding. Filtering methods use statistical measures to evaluate the correlation or dependency between features and the target variable; wrapping methods treat feature selection as an optimization problem, using the learning algorithm itself to evaluate feature subsets; embedding methods integrate feature selection into the model training process.
[0004] However, existing technologies each have their limitations: filtering methods, while computationally efficient, may overlook complex feature interactions and cannot consider the requirements of specific learning algorithms; wrapping methods, while achieving high accuracy, are computationally expensive, especially for high-dimensional datasets; embedding methods, while capable of simultaneous feature selection and model training, often rely on specific models and lack transparency in the feature selection process. Furthermore, existing technologies often struggle to strike a balance between model performance, computational efficiency, and interpretability when dealing with high-dimensional data. Summary of the Invention
[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a feature selection method for traffic emission prediction.
[0006] The objective of this invention can be achieved through the following technical solutions:
[0007] This invention provides a feature selection method for traffic emission prediction, comprising the following steps:
[0008] Step S1: Obtain the traffic emissions dataset, and group the dataset by features to obtain different feature groups;
[0009] Step S2: Perform data cleaning on the grouped feature groups;
[0010] Step S3: Initialize the weights of each feature in the cleaned feature group within each group;
[0011] Step S4: Calculate the feature correlation within and between groups based on the initial weight of each feature, and update the feature weight values based on the calculation results;
[0012] Step S5: Remove features with feature weights less than the first preset value, select a tree-based augmentation model-based machine learning method to calculate the correlation between features and the target, and combine the interpretability SHAP analysis method and the model's inherent feature importance method to calculate feature importance values, and update feature weights based on feature importance values;
[0013] Step S6: Execute step S5 until each group retains at least one feature;
[0014] Step S7: Based on the feature weights obtained in step S6, select the optimal feature combination and perform prediction verification of the optimal combination.
[0015] Furthermore, the traffic emissions dataset includes vehicle type, road conditions, weather conditions, and fuel efficiency.
[0016] Furthermore, step S1 involves grouping the dataset by features, including the following process: setting feature grouping parameters, which include automatic grouping and manual grouping; using manual grouping for datasets with fewer than a second preset value, and using automatic grouping for datasets with more than or equal to the second preset value; and dividing the dataset into different feature groups according to the two methods.
[0017] Furthermore, the automatic grouping process includes: naming each feature according to specifications, calling the API instructions of the Gemini large language model, setting prompt words, performing semantic analysis on the feature names, returning the categorized text parsed into dictionary format, mapping the features to their semantic analysis results, and automatically grouping the features into different groups to achieve automatic feature grouping.
[0018] Furthermore, the data cleaning process in step S2 includes: data missing detection, outlier identification, data completion, and normalization processing.
[0019] Furthermore, step S3 specifically includes: assigning initial weights to the key features that need to be retained. The initial weights for the remaining features are then assigned the following values:
[0020] Furthermore, step S4, the intra-group feature correlation calculation, includes: preferentially considering initial weights of... The characteristics are analyzed, and their correlation coefficients are calculated. If the coefficient exceeds a first preset threshold, the weight is adjusted accordingly. The method for calculating the correlation coefficient matrix for each group's features is as follows:
[0021]
[0022] in, Let X be a feature within group j. jk and X jl The correlation coefficient of X; jk,i It is X jk The j-th data point; X jl,i It is X jl The j-th data point; It is feature X jk The mean; It is feature X jl The mean of the data points; n is the total number of data points.
[0023] Furthermore, step S4, calculating the inter-group feature correlation, includes: calculating the initial variance inflation factor (VIF) value for all features; determining a suitable VIF threshold as the criterion for feature removal; identifying the feature with the highest current VIF value; and if the VIF value of this feature exceeds a second preset threshold, adjusting its weight accordingly. Otherwise, stop iteration; remove. Based on the characteristics, continue to recalculate the VIF values of all remaining features; the calculation formula is as follows:
[0024]
[0025] in, It is feature X jk The coefficient of determination when performing regression using all other features.
[0026] Furthermore, step S5 includes the following step: retaining the feature weights obtained in step S4 as... and The features are defined and divided into training, validation, and test sets according to a first preset ratio. Then, a tree-based augmentation model is used for training and validation. The inherent feature importance values of the model are calculated, and the feature importance values are calculated using the SHAP analysis method. The feature weight with the lowest importance ranking is updated.
[0027] Furthermore, step S7 includes the following steps: updating the feature weights based on the feature importance values calculated in step S6; sorting all weights and selecting the feature combination that yields the best prediction performance result, i.e., the optimal weight combination.
[0028] Compared with the prior art, the present invention has the following advantages:
[0029] (1) This invention takes into account the contradiction between performance, computational efficiency and interpretability faced by traditional methods when processing high-dimensional data, and provides a feature selection strategy based on interpretable machine learning, which can improve the efficiency and accuracy of feature selection at the same time.
[0030] (2) This invention is applied in the field of assessment of factors affecting traffic emissions. It can efficiently screen out key factors affecting traffic emissions and solve the problem of interactive effects caused by highly correlated factors.
[0031] (3) By combining automated feature classification and interpretable SHAP technology, this invention avoids the processing methods of two highly correlated features in existing feature engineering techniques, namely, the loss of important information that may be caused by simple screening, or the introduction of noise by retaining both features, thereby affecting the stability of the model. Attached Figure Description
[0032] Figure 1 This is a flowchart of the feature selection method of the present invention;
[0033] Figure 2 The image shows the results of the feature selection parameter tuning. Detailed Implementation
[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0035] Example 1:
[0036] This invention provides a feature selection method for traffic emission prediction, comprising the following steps:
[0037] Step S1: Obtain traffic emissions dataset and perform feature grouping: Design a feature grouping strategy that combines automatic and manual methods to systematically classify and manage various factors affecting traffic emissions;
[0038] Step S2: Data Cleaning: Perform a comprehensive traffic emissions input data cleaning process, including detecting and handling missing values, identifying and handling outliers, performing necessary data completion, and normalizing the data;
[0039] Step S3: Feature weight initialization: Assign initial weight values to each feature;
[0040] Step S4: Multicollinearity processing: Based on the feature groups divided in step S1, calculate the feature correlation within and between groups respectively, and update the feature weight values using the calculation results;
[0041] Step S5: Feature Importance Calculation: Select the machine learning method based on the Tree-Boosted Model XGBoost to calculate the correlation between features and the target, and combine the interpretability SHAP analysis method and the model's inherent feature importance method to update the feature weights;
[0042] Step S6: Iteratively update weights: Repeat step S5 until at least one feature is retained for each group;
[0043] Step S7: Based on the feature weights obtained in step S6, select the optimal feature combination and perform prediction verification of the optimal combination.
[0044] Further, step S1 includes the following sub-steps:
[0045] Step S1.1: Set the feature grouping parameter GroupType∈{"manual","automatic"};
[0046] Step S1.2: For datasets X = {x1, x2, ..., x} with a smaller number of factors influencing traffic emissions, i.e., the number of features p. p Set GroupType="manual" to use manual grouping, where x1, x2, ..., x p Factors such as speed, weather, and fuel type;
[0047] Step S1.3: For dataset X with a large number of traffic emission influencing factors p, set GroupType="automatic" and use an automatic grouping method. Utilize the natural language processing capabilities of the Gemini-api large-scale language model to analyze the meaning of the names of the emission influencing factors; the automatic feature classification process includes:
[0048] (1) Feature naming conventions: Ensure that each feature has a descriptive and meaningful name, such as "Speed" which represents the speed of a vehicle;
[0049] (2) Call the API commands of Gemini large language model: set reasonable prompt words and perform semantic analysis on feature names;
[0050] (3) Return the categorized text parsed into dictionary format, map the features to the results based on semantic analysis, and automatically classify the features into different groups, thereby achieving automatic feature classification.
[0051] Using the two methods described above, the feature is divided into m distinct combinations.
[0052] Furthermore, step S2 performs comprehensive cleaning and standardization on the input data, specifically including: data missing detection, outlier identification, data completion, and normalization. The normalization calculation method is as follows:
[0053]
[0054] Where X′ is the normalized eigenvalue; X is the original eigenvalue; X min X max denoted as minimum and maximum values; 'a' is the lower bound of the target interval; 'b' is the upper bound of the target interval.
[0055] Further, step S3 specifically includes:
[0056] (1) Assign the initial weights to the key features p that need to be retained as follows:
[0057] (2) The initial weights for the remaining features are then assigned the following values:
[0058] Furthermore, step S4 comprises two sub-steps:
[0059] Step S4.1: Within-group correlation calculation: Prioritize initial weights of... The characteristics are analyzed, and the correlation coefficient is calculated. If it exceeds a set threshold, its weight is adjusted accordingly. The method for calculating the correlation coefficient matrix for each group is as follows:
[0060]
[0061] in, Let X be a feature within group j. jk and X jl The correlation coefficient of X; jk,i It is X jk The j-th data point; X jl,i It is X jl The j-th data point; It is feature X jk The mean; It is feature X jl The mean of the data points; n is the total number of data points.
[0062] Step S4.2: Calculate inter-group correlation: Calculate the initial variance inflation factor (VIF value) for all features, determine a suitable VIF threshold as the criterion for feature removal; find the feature with the highest current VIF value, and when the VIF value of that feature exceeds the set threshold, adjust its weight accordingly. Otherwise, stop iteration; remove. Based on the characteristics, continue to recalculate the VIF values of all remaining features; the calculation formula is as follows:
[0063]
[0064] in, It is feature X jk The coefficient of determination when performing regression using all other features.
[0065] Further, step S5 includes the following sub-steps:
[0066] Step S5.1: Calculate the correlation between feature X and target Y using the Tree-Boosting Model (XGBoost): retain the feature weights obtained in step S4. The features were analyzed and divided into training, validation, and test sets in a 6:2:2 ratio, and then a machine learning model was used to complete the training and validation.
[0067] Step S5.2: Calculate the inherent feature importance value of the model: In machine learning, the Tree model calculates feature importance by accumulating the average number of samples covered by the split of each feature in each tree. The calculation method is as follows:
[0068]
[0069] Wherein, K1(X) q ) is the feature X q Importance value; N q Is feature X used in all trees? q The number of splits; splits(X) q ) is using features to enter X q The set of nodes for row splitting; cover(s) is the number of samples covered by node s;
[0070] Step S5.3: Calculate feature importance values using the SHAP analysis method: Using the TreeExplainer algorithm and a trained machine learning model, calculate the Shapley value for each feature of each sample. The calculation formula is as follows:
[0071]
[0072] Where N is the set of all features; S is a subset of N excluding feature q, f(S) is the prediction made using features in subset S, and |S| is the size of subset S; This ensures that all feature combinations are considered as weighting factors. Tree-SHAP adapts Shapley value calculation to tree-based models by decomposing the prediction process into the sum of the contributions of each tree. The Shapley value of feature q in a tree-based model is given by the following formula:
[0073]
[0074] Where H is the number of trees, L h It is the number of leaves in tree h. It is the number of samples that reach node j in tree h. It is the value of feature q at node j, and It is the value of the node preceding feature q.
[0075] Step S5.4: Set the scaling factor δ to balance the influence of eigenvalue splitting importance and gain importance. The calculation method for eigenvalue importance is as follows:
[0076]
[0077] Wherein K(X) q To obtain the normalized X in the end q Importance value, For the normalized K1(X) q ), For the normalized K2(X) q The normalization method can refer to the calculation method in step 2, where the values of a and b are set to 0 and 0.5 respectively.
[0078] Furthermore, in step S6, an improved backward elimination strategy is used to perform multiple rounds of prediction iterations. In each round of iteration, the weight of the feature with the lowest feature importance is updated to -1 until the number of iterations n is less than a preset threshold nm. The performance indicators of the model (such as accuracy, root mean square error, etc.) under different feature combinations are compared.
[0079] Furthermore, step S7 requires the feature importance value K(X) calculated based on steps S5 and S6. q Update feature X q weight All weights are sorted, and the feature combination that yields the best prediction performance is selected, which is the optimal weight combination.
[0080] Example 2:
[0081] This embodiment takes the investigation of the relationship between vehicle NOx emissions (target) and influencing factors (features) as an example, and provides a feature selection method and device based on interpretable machine learning. The implementation process is as follows: Figure 1As shown, it includes the following steps:
[0082] Step S1: Design optional automatic or manual feature grouping techniques to divide features into several groups based on correlation between features or domain knowledge. There are a total of 56 influencing factors {x1, x2, ..., x...}. p (Number of features p = 56), set the parameter to GroupType ∈ {"automatic"}, and select the automatic grouping method. The detailed process is as follows:
[0083] (1) Feature naming conventions: Ensure that each feature has a descriptive and meaningful name, such as the word "speed" which directly reveals its role and meaning in the model;
[0084] (2) Call “Gemini-api” (https: / / aistudio.google.com / app / apikey): Perform semantic analysis on feature names and descriptions, involving setting reasonable prompts and applying NLP techniques to understand the context and importance of each feature; the following is the prompt set for this patent: “prompt = Our goal is to classify pollutant emissions. The following is a list of feature names: {','.join(feature_names)}. Please divide them into no more than M categories according to their meanings and provide the category names. At the same time, determine which(features) should be considered as the target variable (y). Return the results in the following format: 'Category 1: [feature 1, feature 2, ...] Category 2: [feature 1, feature 2, ...] Target: [target 1, target 2, ...]”;
[0085] (3) The returned classification text is parsed into a dictionary format, the features are mapped to the results based on semantic analysis, and the features are automatically classified into different groups, thereby realizing the automatic classification and reasonable grouping of features. In this embodiment, the features are automatically divided into 7 groups (m=7).
[0086] Step S2: Based on the input data in Step S1, perform comprehensive cleaning and standardization processing, specifically including: missing data detection, outlier identification, data completion, and normalization. The normalization calculation method is as follows:
[0087]
[0088] Where X′ is the value of the normalized data point; X is the value of the original data point; X min X max represents the minimum and maximum values in the dataset; 'a' is the lower bound of the target interval (usually 0); 'b' is the upper bound of the target interval (usually 1). In this embodiment, 'a' is set to 0 and 'b' is set to 1.
[0089] Step S3: Initialize the weights for each feature within the i-th group. Assign initial weights to each feature based on its expected contribution to the model. For features identified as representative by domain experts, or features that need to be retained due to specific requirements, their initial weights... Features considered to have a significant impact on the model's predictive ability are given the highest weight. Other features not identified as key features have their initial weights set to... It is considered to contribute little to the model or to have no practical significance.
[0090] Step S4: Based on the grouping results of Step S1, this embodiment completes the screening of highly correlated features according to the following two steps: First, the intra-group correlation is calculated, prioritizing features with an initial weight of 1. For other highly correlated features (in this embodiment, the correlation coefficient threshold is set to 0.9), their weights are adjusted to -1, thereby reducing the influence of redundant features. The calculation method for the correlation coefficient matrix of each group is as follows:
[0091]
[0092] in, Let X be a feature within group j. jk and X jl The correlation coefficient of X; jk,i It is X jk The j-th data point; X jl,i It is X jl The j-th data point; It is feature X jk The mean; It is feature X jl The mean of the data points; n is the total number of data points.
[0093] The second step is to calculate the inter-group correlation: Calculate the initial variance inflation factor (VIF value) for all features, and determine a suitable VIF threshold (set to 10 in this embodiment) as the criterion for feature removal. Identify the feature with the highest current VIF value. If the VIF value of this feature exceeds the preset threshold, remove it from the model; otherwise, stop the iteration, as the VIF values of all features have met the requirements. After removing the feature, rebuild the model based on the updated feature set and recalculate the VIF values of all remaining features using the following formula:
[0094]
[0095] in, It is feature X jk The coefficient of determination when performing regression using all other features.
[0096] Step S5: Update the feature weights by combining the SHAP method with a feature importance evaluation technique based on the XGBoost model. Specifically, this is achieved by setting a scaling factor δ, as shown in the following figure. Figure 2 As shown. To balance the influence of eigenvalue splitting importance and gain importance, thus obtaining a more comprehensive assessment of eigenvalue importance, the final eigenvalue importance value is calculated as follows:
[0097]
[0098] Wherein K(X) q To obtain the normalized X in the end q Importance value, For the normalized K1(X) q ), For the normalized K2(X) q The normalization method can refer to the calculation method in step 2, where the values of a and b are set to 0 and 0.5 respectively. K1(X) q The feature X is calculated based on the inherent method of the model. q Importance value; N q Is feature X used in all trees? q The number of splits; splits(X) q ) is using features to enter X q The set of nodes for row splitting; cover(s) is the number of samples covered by node s. K2(X q () represents the feature X calculated based on the SHAP method. q Importance value; H is the number of trees, L h n is the number of leaves in tree h. j t It is the number of samples that reach node j in tree h. It is the value of feature q at node j, and It is the value of the node preceding feature q.
[0099] Step S6: Employ an improved backward elimination strategy for multiple rounds of prediction iterations. In each iteration, update the weight of the feature with the lowest feature importance to -1, until the iteration count n is less than a preset threshold nm. Compare the mean squared error (MSE) of the model under different feature combinations.
[0100] Step S7: The feature importance value K(X) calculated in steps S5 and S6 needs to be obtained. q Update feature X q weight Then, all weights are sorted, and the feature combination that yields the best prediction performance is selected, which is the optimal weight combination.
[0101] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A feature selection method for traffic emission prediction, characterized in that, Includes the following steps: Step S1: Obtain the traffic emissions dataset, and group the dataset by features to obtain different feature groups; Step S1 involves grouping the dataset by features. The process includes the following: Set feature grouping parameters, which include automatic grouping and manual grouping. Manual grouping is used for datasets with fewer than a second preset value, and automatic grouping is used for datasets with more than or equal to the second preset value. The dataset is divided into different feature groups according to the two methods. The automatic grouping process includes: naming each feature, calling the API instructions of the Gemini large language model, setting prompt words, performing semantic analysis on the feature name, parsing the returned classification text into dictionary format, mapping the features to their semantic analysis results, and automatically grouping the features into different groups to achieve automatic feature grouping. Step S2: Perform data cleaning on the grouped feature groups; Step S3: Initialize the weights of each feature in the cleaned feature group within each group; Step S4: Calculate the feature correlation within and between groups based on the initial weight of each feature, and update the feature weight values based on the calculation results; Step S5: Remove features with feature weights less than the first preset value, select a tree-based augmentation model-based machine learning method to calculate the correlation between features and the target, and combine the interpretability SHAP analysis method and the model's inherent feature importance method to calculate feature importance values, and update feature weights based on feature importance values; Step S6: Execute step S5 until each group retains at least one feature; Step S7: Based on the feature weights obtained in step S6, select the optimal feature combination and perform prediction verification of the optimal combination.
2. The feature selection method for traffic emission prediction according to claim 1, characterized in that, The traffic emissions dataset includes vehicle type, road conditions, weather conditions, and fuel efficiency.
3. The feature selection method for traffic emission prediction according to claim 1, characterized in that, The data cleaning process in step S2 includes: data missing detection, outlier identification, data completion, and normalization.
4. The feature selection method for traffic emission prediction according to claim 1, characterized in that, The specific steps of step S3 include: assigning initial weights to the key features that need to be retained. The initial weights for the remaining features are then assigned the following values: .
5. The feature selection method for traffic emission prediction according to claim 1, characterized in that, The intra-group feature correlation calculation in step S4 includes: prioritizing initial weights... The characteristics are analyzed, and their correlation coefficients are calculated. If the coefficient exceeds a first preset threshold, the weight is adjusted accordingly. The method for calculating the correlation coefficient matrix for features within each group is as follows: in, For is group Internal characteristics The correlation coefficient; yes The One data point; yes The One data point; It is a feature The mean; It is a feature The mean; It represents the total number of data points.
6. The feature selection method for traffic emission prediction according to claim 1, characterized in that, Step S4, calculating the inter-group feature correlation, includes: calculating the initial variance inflation factor (VIF) value for all features; determining a suitable VIF threshold as the criterion for feature removal; identifying the feature with the highest current VIF value; and adjusting its weight if the VIF value of that feature exceeds a second preset threshold. Otherwise, stop the iteration; remove Based on the characteristics, recalculate the VIF values for all remaining features; the calculation formula is as follows: in, It is a feature The coefficient of determination when performing regression using all other features.
7. The feature selection method for traffic emission prediction according to claim 1, characterized in that, Step S5 includes the following steps: retaining the feature weights obtained in step S4. and The features are defined and divided into training, validation, and test sets according to a first preset ratio. Then, a tree-based augmentation model is used for training and validation. The inherent feature importance values of the model are calculated, and the feature importance values are calculated using the SHAP analysis method. The feature weight with the lowest importance ranking is updated. .
8. The feature selection method for traffic emission prediction according to claim 1, characterized in that, Step S7 includes the following steps: updating the feature weights based on the feature importance values calculated in step S6; sorting all weights and selecting the feature combination that yields the best prediction performance index result, i.e., the optimal weight combination.