Characteristic screening method based on permutation importance technology and related system

Abstract

The invention discloses a feature selection method based on an arrangement importance technology and a related system, and belongs to the field of machine learning feature engineering. Aiming at the problem that a feature selection method is difficult to accurately evaluate contribution of features to a model prediction capability, objective evaluation of feature importance is realized through three key steps of feature random disruption, model performance evaluation and feature importance calculation, and the method comprises the following steps: selecting to-be-evaluated features and randomly disrupting values of the to-be-evaluated features; probability distribution of original features is maintained, but association with target variables is destroyed; evaluating the performance of the pre-training model by using the disordered data set; calculating feature importance scores by comparing model performance differences before and after feature disruption; repeating the process on all the features and sorting the features according to importance; and finally, selecting an optimal feature set according to a service requirement and a performance target. The method can objectively evaluate the contribution degree of each feature to the model, improves the performance and interpretability of the model, and is suitable for feature selection optimization in various machine learning scenes.

Description

Technical Field

[0001] This invention relates to the field of machine learning, specifically to a feature selection method and related system based on permutation importance technology. Background Technology

[0002] With the widespread application of machine learning technology in industrial control systems, feature selection, as a key step in data preprocessing, has a significant impact on model performance and interpretability. Feature selection refers to the process of choosing the most relevant and useful subset of features from the original feature set to improve the model's predictive performance, reduce computational complexity, minimize the risk of overfitting, and enhance model interpretability.

[0003] In existing technologies, feature selection methods are mainly divided into three categories: filtering, wrapping, and embedding. Filtering methods select features based on the statistical properties between features and the target variable; they are computationally simple but may ignore the interactions between features. Wrapping methods select features by evaluating the impact of different feature subsets on model performance; they are effective but computationally complex. Embedding methods integrate feature selection into the model training process, completing feature selection simultaneously with model training. CN117312819A discloses an efficient feature selection method suitable for models with large datasets. This method evaluates feature importance by calculating feature importance scores and comparing the score deviations before and after label shuffling.

[0004] In the field of industrial control systems, CN118353710A proposes a dual feature selection method for network traffic in industrial control systems. This method corrects the bias in calculating feature importance produced by the random forest model by comparing the feature importance scores and their relative differences before and after label vector permutation. This method can effectively improve intrusion detection accuracy while solving the computational complexity and efficiency problems caused by high-dimensional data.

[0005] To address the feature selection problem in recommender systems, CN117272130A introduces a click prediction method for recommender systems based on feature selection and bias removal. This method achieves more accurate CTR prediction by performing bias analysis and importance assessment on combined features, and building a main tower network and a bias removal tower network based on a dual-tower model architecture.

[0006] Regarding feature selection implementation techniques, CN114912628A proposes a feature selection method that learns each feature to be selected and its corresponding true label through a tree model to obtain the importance of the true features, and obtains the importance of false features by shuffling the labels, and selects features based on the comparison between the two. CN118312754A proposes a feature selection method for driving recorder quality inspection data. This method integrates the advantages of filtering, wrapping, and embedding methods, and uses a heuristic search algorithm to optimize the feature subset selection process.

[0007] However, existing feature selection methods still suffer from the following problems: First, most methods lack an intuitive mechanism for evaluating feature importance, making it difficult to accurately measure the actual contribution of features to the model's predictive ability. Second, existing methods face technical challenges in maintaining the original distribution characteristics of features while disrupting their correlation with the target variable. Third, the visualization and interpretability of feature selection results are insufficient, making it difficult to support business personnel's understanding and decision-making. Fourth, the balance between business value and model complexity is not fully considered during feature selection, easily leading to model overfitting or the selected features lacking practical business significance. Furthermore, in industrial scenarios such as power plant control systems, current adjustments to controllers are mostly performed manually, which is prone to human error, requiring modifications and redeployment, and adversely affecting the operating units. Simultaneously, when using machine learning prediction models, the lack of effective feature selection methods makes it impossible to accurately assess the importance of each feature in the model, resulting in redundant and irrelevant features in the model, affecting model performance and interpretability.

[0008] Therefore, there is an urgent need for a feature selection method that can objectively assess feature importance, maintain feature distribution characteristics, provide intuitive visualization results, and balance business value with model complexity, so as to improve model performance and interpretability and support business decisions. Summary of the Invention

[0009] The purpose of this invention is to overcome the above-mentioned shortcomings and provide a feature screening method and related system based on permutation importance technology.

[0010] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a feature selection method based on permutation importance technology, comprising the following steps: The feature values ​​in the dataset are randomly shuffled to disrupt the association between the features and the target variable, while maintaining the original probability distribution of the feature values. Pre-trained models were used to evaluate prediction performance on datasets with randomly shuffled feature values, and prediction performance metrics were recorded. Compare the model performance before and after randomizing the features to obtain the performance change relationship, and calculate the importance score of the feature value based on the performance change relationship; Repeat the above steps for each feature value in the dataset to generate importance scores for the data and all features; sort the features by importance scores to generate a ranked list of feature importance.

[0011] A further improvement of this invention lies in the following method for randomly shuffling the feature values ​​in the dataset to disrupt the association between the features and the target variable, while maintaining the original probability distribution of the feature values ​​unchanged: The feature values ​​in the dataset are obtained, and the Fisher-Yates shuffle algorithm is used to randomize the feature values ​​so that each feature value has the same probability of occurrence and maintains the original probability distribution characteristics of the feature values. For continuous feature values, the original mean and variance are maintained, and for discrete feature values, the frequency of occurrence of each category is kept consistent.

[0012] A further improvement of this invention lies in the following method for evaluating the prediction performance of a pre-trained model on a dataset with randomly shuffled feature values, and for recording the prediction performance metrics: Obtain the dataset with randomly shuffled feature values, and use the accuracy metric to measure the performance of the model in predicting the dataset with randomly shuffled feature values. A comparative model is used to compare the consistency between the prediction results and the actual target variable, and the proportion of correctly predicted samples to the total number of samples is obtained. The proportion of correctly predicted samples out of the total number of samples is used as the prediction performance metric.

[0013] A further improvement of this invention is that, after generating the feature importance ranking list, the feature importance ranking list is visualized.

[0014] Secondly, the present invention provides a feature selection system based on permutation importance technology, comprising: The feature random shuffling module is used to randomly shuffle the feature values ​​in the dataset, thereby disrupting the association between the features and the target variable while maintaining the original probability distribution of the feature values. The model performance evaluation module is used to evaluate the prediction performance of a pre-trained model on a dataset with randomly shuffled feature values, and to record the prediction performance metrics. The importance assessment module is used to compare the model performance before and after features are randomly shuffled, obtain the performance change relationship, and calculate the importance score of the feature value based on the performance change relationship; The feature ranking module is used to repeat the above operation for each feature value in the dataset to generate importance scores for the data and all features in it; then, the features are sorted according to their importance scores to generate a feature importance ranking list.

[0015] A further improvement of this invention is that the function of the feature random shuffling module is implemented through the following method: The feature values ​​in the dataset are obtained, and the Fisher-Yates shuffle algorithm is used to randomize the feature values ​​so that each feature value has the same probability of occurrence and maintains the original probability distribution characteristics of the feature values. For continuous feature values, the original mean and variance are maintained, and for discrete feature values, the frequency of occurrence of each category is kept consistent.

[0016] A further improvement of this invention is that the function of the model performance evaluation module is implemented through the following method: Obtain the dataset with randomly shuffled feature values, and use the accuracy metric to measure the performance of the model in predicting the dataset with randomly shuffled feature values. A comparative model is used to compare the consistency between the prediction results and the actual target variable, and the proportion of correctly predicted samples to the total number of samples is obtained. The proportion of correctly predicted samples out of the total number of samples is used as the prediction performance metric.

[0017] A further improvement of the present invention is that it also includes a display module, which is used to visualize the feature importance ranking list after it is generated.

[0018] Thirdly, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of a feature selection method based on permutation importance technology.

[0019] Fourthly, the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a feature selection method based on permutation importance technology.

[0020] The beneficial effects of this invention are as follows: By employing a feature selection method based on permutation importance, the inspection and control of the lower-level machines in a power plant control system can be completed from top to bottom, significantly reducing the risk of human error and improving system fault tolerance. By removing redundant or irrelevant features, the model structure is simplified, improving training efficiency and predictive stability. Quantifying the impact of features on model performance allows users to clearly understand the importance of each feature to the prediction results, enhancing model interpretability. This method exhibits model agnosticity and is applicable to any machine learning model, including deep neural networks and random forests. Furthermore, it offers high flexibility, applicable to various machine learning tasks such as classification and regression, effectively addressing the shortcomings of existing feature selection methods. Attached Figure Description

[0021] Figure 1 This is a flowchart of the present invention; Figure 2 This is a system diagram of the present invention. Detailed Implementation

[0022] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0023] See Figure 1 The feature selection method based on permutation importance technology includes the following steps: S1, randomly shuffle the feature values ​​in the dataset to disrupt the association between the features and the target variable, while keeping the original probability distribution of the feature values ​​unchanged.

[0024] S2, evaluate the prediction performance of the pre-trained model on the dataset with randomly shuffled feature values, and record the prediction performance metrics.

[0025] S3. Compare the model performance before and after the features are randomly shuffled to obtain the performance change relationship, and calculate the importance score of the feature value based on the performance change relationship.

[0026] S4. Repeat the above operation for each feature value in the dataset to generate the importance scores of the data and all features in it; sort the features according to their importance scores to generate a feature importance ranking list.

[0027] See Figure 2 Feature selection systems based on permutation importance techniques include: The feature random shuffling module is used to randomly shuffle the feature values ​​in the dataset, thereby disrupting the association between the features and the target variable while maintaining the original probability distribution of the feature values.

[0028] The model performance evaluation module is used to evaluate the prediction performance of a pre-trained model on a dataset with randomly shuffled feature values, and to record the prediction performance metrics.

[0029] The importance assessment module is used to compare the model performance before and after features are randomly shuffled, obtain the performance change relationship, and calculate the importance score of the feature value based on the performance change relationship.

[0030] The feature ranking module is used to repeat the above operation for each feature value in the dataset to generate importance scores for the data and all features in it; then, the features are sorted according to their importance scores to generate a feature importance ranking list.

[0031] Example 1: This embodiment provides a feature selection method based on permutation importance. This method systematically evaluates the contribution of each feature to the model's predictive ability, thereby identifying and selecting the most valuable feature subset to improve model performance and reduce complexity. The method includes the following steps: Step 1: Shuffling Features. Select the important features to be evaluated from the dataset and randomly shuffle the values ​​of these features to form a new feature column. The randomly shuffled feature column maintains a similar distribution to the original feature column in a statistical sense, but it disrupts its association with the target variable. Specifically: First, load the original dataset, which contains multiple feature columns and one target variable column; then, select the first feature in the dataset as the feature to be evaluated; next, use a random permutation algorithm to reorder all values ​​of this feature, generating a shuffled feature column; finally, replace the original feature column with the shuffled feature column to form a new dataset. During the feature shuffling process, the probability distribution of the original features is preserved. For example, if the original feature is normally distributed, the shuffled feature will also maintain a similar normally distributed distribution. This is achieved by changing only the order of the feature values ​​without changing the values ​​themselves, ensuring that the shuffled features have the same statistical properties as the original features, such as mean, variance, and quantiles.

[0032] Step 2: Evaluate Model Performance. Re-evaluate the predictive performance of the pre-trained model using the shuffled dataset, and record the model's performance metrics after feature shuffling. Specifically: First, load the pre-trained machine learning model, which has already been trained on the original, undisturbed dataset; then, use the new dataset containing shuffled features to predict the model; next, calculate the model's performance metrics on this dataset, such as accuracy, precision, recall, or F1 score; finally, record these performance metrics as the model's performance after feature shuffling. In this embodiment, accuracy is used as the performance metric for model performance evaluation. Accuracy is defined as the ratio of the number of correctly predicted samples to the total number of samples, calculated as: Accuracy = Number of correctly predicted samples / Total number of samples × 100%. Accuracy, as an intuitive and easy-to-understand metric, is suitable for evaluating the overall performance of classification models, especially when the class distribution is relatively balanced.

[0033] Step 3: Calculate Feature Importance. Compare the baseline performance of the model with features intact with the performance of the model after feature shuffling to calculate the feature importance score. Specifically: First, obtain the model's performance metrics on the original, intact dataset as the baseline performance; then calculate the difference between the baseline performance and the performance after feature shuffling; next, standardize this difference to obtain the feature importance score; finally, store the feature identifier and its corresponding importance score as key-value pairs. In this embodiment, features with significantly decreased importance scores are considered to make a significant contribution to the model's predictive ability, while features with little change in importance scores are considered to have a low contribution. Specifically, if shuffling a feature reduces the model accuracy from 95% to 85%, the importance score of that feature is 10%, indicating that the feature has a significant impact on the model's prediction results; conversely, if shuffling a feature only reduces the model accuracy from 95% to 94.5%, the importance score of that feature is 0.5%, indicating that the feature has a small impact on the model's prediction results.

[0034] Step 4: Repeat the above steps. For each feature in the dataset, shuffle it one by one, evaluate the model performance, and calculate the feature importance, generating importance scores for all features. Specifically, first, restore the dataset to its original state, ensuring each evaluation is based on the same initial conditions; then, select the next feature in the dataset as the feature to be evaluated; next, repeat steps one through three to shuffle, evaluate, and calculate the importance of that feature; finally, repeat the above process until all features in the dataset have been evaluated. In this embodiment, assuming the dataset contains 10 features, the feature shuffling and evaluation process needs to be performed 10 times, calculating the importance score for each feature. To ensure the stability of the evaluation results, the shuffling process for each feature can be repeated multiple times (e.g., 5 times), and the average value is taken as the final importance score.

[0035] Step 5: Sort Features. Sort all features according to their calculated importance scores from highest to lowest, generating a feature importance ranking list. Specifically: First, collect the importance scores of all features; then, use a ranking algorithm (such as quicksort) to sort all features in descending order of importance score; next, generate a ranking list containing feature names and corresponding importance scores; finally, save the ranking results as structured data for easy subsequent analysis and use. In this embodiment, the feature importance ranking results are visualized to more intuitively understand the contribution of each feature to the model. Visualization methods include, but are not limited to, bar charts, heatmaps, or radar charts, where the horizontal axis represents different features and the vertical axis represents the importance score. Through this intuitive display, data scientists and business personnel can quickly identify the most critical features, providing a basis for subsequent feature engineering and model optimization.

[0036] Step Six: Feature Selection. Based on business needs, model interpretability, and performance goals, determine the final set of features selected from the feature importance ranking list, and retrain the model using the selected features to optimize model performance. Specifically, first, set the criteria for feature selection, such as an importance threshold or the desired number of features to retain; then, filter the subset of features that meet the criteria from the ranking list; next, construct a new dataset using the selected feature subset; finally, retrain the model based on the new dataset and evaluate the performance of the optimized model. In this embodiment, the final feature selection also needs to comprehensively consider the business value of the features and the model complexity to avoid model overfitting and ensure that the selected features fully support business decisions. For example, in the field of financial risk control, even if some features have relatively low importance scores, they may still be retained in the final feature set if they have important business interpretability (such as a customer's credit history). At the same time, to control model complexity, an importance threshold (such as 1%) may be set, retaining only features with importance scores higher than this threshold.

[0037] Through the steps described above, the feature selection method based on permutation importance technology provided in this embodiment can effectively identify features that significantly contribute to the model's predictive ability and build a more efficient and accurate prediction model based on these features. The core advantage of this method lies in its model independence, meaning it can be applied to any type of pre-trained model, including but not limited to decision trees, random forests, support vector machines, and neural networks. Furthermore, this method is intuitive and easy to interpret; by comparing the changes in model performance before and after feature shuffling, the importance of each feature can be clearly quantified, providing a scientific basis for feature selection.

[0038] In practical applications, this method can significantly reduce the number of features in a model, lower computational complexity, and improve the efficiency of model training and prediction. Simultaneously, by removing redundant or irrelevant features, it can also enhance the model's generalization ability and reduce the risk of overfitting. Especially in high-dimensional data analysis scenarios, such as gene expression data analysis, text mining, and image recognition, this method can filter out the most valuable subset from thousands of features, significantly improving analytical efficiency and accuracy.

[0039] Example 2: This embodiment provides a feature selection system based on permutation importance technology. This system automates feature importance evaluation and selection through modular design. The system includes the following modules: The feature random shuffling module is used to select important features to be evaluated from the dataset and randomly shuffle the values ​​of these features to form new feature columns. These randomly shuffled feature columns maintain a similar distribution to the original feature columns in a statistical sense, but their association with the target variable is disrupted. This module first receives the original dataset as input and then performs a random shuffling operation on the specified features. The shuffling process uses the Fisher-Yates shuffling algorithm to ensure that each feature value has an equal probability of appearing in any position, while maintaining the overall distribution characteristics of the features. For example, for continuous features, the shuffled features retain their original mean, variance, and distribution shape; for discrete features, the frequency of occurrence of each category remains unchanged. This module also provides a parameter configuration interface, allowing users to specify the random seed for shuffling to ensure the reproducibility of the experiment.

[0040] The model performance evaluation module is used to re-evaluate the predictive performance of a pre-trained model using a shuffled dataset, recording the model's performance metrics after feature shuffling. This module first loads the pre-trained machine learning model, then inputs the dataset containing shuffled features into the model for prediction. Based on the comparison between the prediction results and the true labels, various performance metrics are calculated, including accuracy, precision, recall, F1 score, and AUC. These metrics are calculated and visualized using methods such as confusion matrices and ROC curves. This module also supports cross-validation, which allows the dataset to be divided into multiple folds, with performance metrics calculated separately for each fold and then averaged to reduce the impact of random fluctuations on the evaluation results.

[0041] The feature importance calculation module compares the baseline performance of the model with that of the model before feature shuffling with the performance of the model after feature shuffling, calculating the importance score of each feature. This module first obtains the model's baseline performance on the original dataset, then calculates the difference between this baseline performance and the performance after feature shuffling. A larger difference indicates a more significant contribution of the feature to the model's predictive ability. To facilitate comparison of the importance of different features, this module also provides a standardization function, converting all feature importance scores into relative percentages or Z-scores. Furthermore, this module implements statistical significance testing, evaluating the statistical reliability of feature importance by repeatedly performing the shuffling process and analyzing the distribution of performance changes.

[0042] The comprehensive importance evaluation module is used to shuffle each feature in the dataset, evaluate model performance, calculate feature importance, and generate importance scores for all features. This module coordinates the workflow of the three modules mentioned above, enabling batch evaluation of all features in the dataset. It maintains a feature queue, processes each feature sequentially, and collects the importance scores of all features. To improve processing efficiency, this module supports parallel computing, allowing the simultaneous evaluation of the importance of multiple features and fully utilizing the computing power of multi-core processors. Furthermore, this module implements progress monitoring and error handling mechanisms to ensure the stable completion of long-running evaluation tasks and provides detailed logging in case of anomalies.

[0043] The feature ranking module sorts all features according to their calculated importance scores from highest to lowest, generating a ranked list of feature importance. This module receives feature importance scores generated by the comprehensive importance assessment module and uses an efficient ranking algorithm (such as heap sort) to arrange all features in descending order of score. The ranking results are presented in tabular form, including feature name, importance score, and ranking. This module also provides various visualization functions, such as bar charts, heatmaps, and radar charts, to visually display the relative importance of each feature. Users can adjust visualization parameters, such as color scheme, chart size, and label display, through an interactive interface to generate visualization reports that meet their needs.

[0044] The final feature selection module determines the final set of features based on a feature importance ranking list, considering business needs, model interpretability, and performance objectives. It then retrains the model using these selected features to optimize performance. This module offers several feature selection strategies, including: threshold-based selection (retaining features with importance scores above a specific threshold); quantity-based selection (retaining the top N features); cumulative contribution-based selection (retaining features whose cumulative importance reaches a specific percentage of total importance); and business rule-based selection (selecting features according to predefined business rules). Users can choose the appropriate strategy or combine multiple strategies for feature selection based on their specific needs. After feature selection, the module automatically constructs a new dataset and provides model retraining functionality to train the optimized model using the selected features. Finally, the effectiveness of the feature selection is evaluated by comparing the performance differences between the original and optimized models.

[0045] The feature selection system based on permutation importance technology provided in this embodiment automates the evaluation and selection of feature importance through modular design. The various modules of the system work collaboratively to form a complete feature selection workflow: from feature shuffling to performance evaluation, then to importance calculation, ranking, and final selection. This system is highly flexible and scalable, supporting multiple machine learning models, performance metrics, and feature selection strategies, and can adapt to the feature selection needs of different fields.

[0046] In practical applications, this system serves as a crucial tool for data preprocessing and feature engineering, helping data scientists and machine learning engineers quickly identify key features, optimize model structure, and improve predictive performance. Particularly when dealing with high-dimensional data, the system significantly reduces feature dimensionality, lowers computational complexity, and improves model training and deployment efficiency. Furthermore, the feature importance reports and visualizations generated by the system provide strong support for model interpretation and business decision-making, helping users understand the model's predictive mechanism and the business value of each feature.

[0047] Example 3: The present invention also provides an electronic device 100 based on a feature selection method using permutation importance technology; the electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on the at least one processor 102, and at least one communication bus 104.

[0048] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the feature selection method based on permutation importance technology described in Embodiment 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101. The memory 101 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 100 (such as audio data), etc. In addition, the memory 101 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0049] The at least one processor 102 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 may be a microprocessor or any conventional processor. The processor 102 is the control center of the electronic device 100, connecting various parts of the electronic device 100 via various interfaces and lines.

[0050] The memory 101 in the electronic device 100 stores multiple instructions to implement a feature selection method based on permutation importance technology, and the processor 102 can execute the multiple instructions to achieve the following: The feature values ​​in the dataset are randomly shuffled to disrupt the association between the features and the target variable, while maintaining the original probability distribution of the feature values. Pre-trained models were used to evaluate prediction performance on datasets with randomly shuffled feature values, and prediction performance metrics were recorded. Compare the model performance before and after randomizing the features to obtain the performance change relationship, and calculate the importance score of the feature value based on the performance change relationship; Repeat the above steps for each feature value in the dataset to generate importance scores for the data and all features; sort the features by importance scores to generate a ranked list of feature importance.

[0051] Example 4: If the modules / units integrated in the electronic device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0052] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0053] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0054] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0055] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A feature selection method based on permutation importance technology, characterized in that, Includes the following steps: The feature values ​​in the dataset are randomly shuffled to disrupt the association between the features and the target variable, while maintaining the original probability distribution of the feature values. Pre-trained models were used to evaluate prediction performance on datasets with randomly shuffled feature values, and prediction performance metrics were recorded. Compare the model performance before and after randomizing the features to obtain the performance change relationship, and calculate the importance score of the feature value based on the performance change relationship; Repeat the above operation for each feature value in the dataset to generate the importance scores of the data and all features in the dataset; Sort the features by importance score to generate a feature importance ranking list.

2. The feature selection method based on permutation importance technology according to claim 1, characterized in that, The specific method for randomly shuffling the feature values ​​in the dataset to disrupt the association between the features and the target variable while maintaining the original probability distribution of the feature values ​​is as follows: The feature values ​​in the dataset are obtained, and the Fisher-Yates shuffle algorithm is used to randomize the feature values ​​so that each feature value has the same probability of occurrence and maintains the original probability distribution characteristics of the feature values. For continuous feature values, the original mean and variance are maintained, and for discrete feature values, the frequency of occurrence of each category is kept consistent.

3. The feature selection method based on permutation importance technology according to claim 1, characterized in that, The specific method for evaluating the prediction performance of a pre-trained model on a dataset with randomly shuffled feature values ​​and recording the prediction performance metrics is as follows: Obtain the dataset with randomly shuffled feature values, and use the accuracy metric to measure the performance of the model in predicting the dataset with randomly shuffled feature values. A comparative model is used to compare the consistency between the prediction results and the actual target variable, and the proportion of correctly predicted samples to the total number of samples is obtained. The proportion of correctly predicted samples out of the total number of samples is used as the prediction performance metric.

4. The feature selection method based on permutation importance technology according to claim 1, characterized in that, After generating the feature importance ranking list, the feature importance ranking list is visualized.

5. A feature selection system based on permutation importance technology, characterized in that, include: The feature random shuffling module is used to randomly shuffle the feature values ​​in the dataset, thereby disrupting the association between the features and the target variable while maintaining the original probability distribution of the feature values. The model performance evaluation module is used to evaluate the prediction performance of a pre-trained model on a dataset with randomly shuffled feature values, and to record the prediction performance metrics. The importance assessment module is used to compare the model performance before and after features are randomly shuffled, obtain the performance change relationship, and calculate the importance score of the feature value based on the performance change relationship; The feature ranking module is used to repeat the above operation for each feature value in the dataset to generate importance scores for the data and all features in the dataset. Sort the features by importance score to generate a feature importance ranking list.

6. The feature selection system based on permutation importance technology according to claim 5, characterized in that, The feature random shuffling module is implemented using the following method: The feature values ​​in the dataset are obtained, and the Fisher-Yates shuffle algorithm is used to randomize the feature values ​​so that each feature value has the same probability of occurrence and maintains the original probability distribution characteristics of the feature values. For continuous feature values, the original mean and variance are maintained, and for discrete feature values, the frequency of occurrence of each category is kept consistent.

7. The feature selection system based on permutation importance technology according to claim 5, characterized in that, The functionality of the model performance evaluation module is implemented through the following methods: Obtain the dataset with randomly shuffled feature values, and use the accuracy metric to measure the performance of the model in predicting the dataset with randomly shuffled feature values. A comparative model is used to compare the consistency between the prediction results and the actual target variable, and the proportion of correctly predicted samples to the total number of samples is obtained. The proportion of correctly predicted samples out of the total number of samples is used as the prediction performance metric.

8. The feature selection system based on permutation importance technology according to claim 5, characterized in that, It also includes a display module, which is used to visualize the feature importance ranking list after it is generated.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the feature selection method based on permutation importance technology as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the feature selection method based on the permutation importance technique as described in any one of claims 1 to 7.

Images