Feature selection method and device and electronic equipment

Abstract

The invention discloses a feature selection method and device and electronic equipment. The method comprises the following steps: acquiring an initial feature set to be subjected to feature selection; determining the correlation between the feature dimensions in the initial feature set, and screening out a candidate feature set to be deleted based on the correlation between the feature dimensions; sorting the plurality of feature dimensions in the to-be-deleted candidate feature set based on at least one preset algorithm, and dividing the to-be-deleted candidate feature set into at least one group of feature subsets based on at least one sorting result and a preset division parameter; and deleting the feature subsets in the initial feature set one by one, determining model prediction scores obtained by training the target algorithm model by using the obtained feature sets to be selected, and selecting the target feature set from the feature sets to be selected as a feature selection result based on the model prediction scores. Through the method, the accuracy and the stability of feature selection can be improved, the calculation overhead can be reduced, and the applicability is high.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a feature selection method, apparatus, and electronic device. Background Technology

[0002] The detection performance of traditional machine learning is highly dependent on feature quality, but most scenarios currently face the problem of "feature dimension explosion", which is manifested in the following ways: (1) conventional solutions contain hundreds of features, most of which are strongly correlated, resulting in feature redundancy; (2) the computational complexity of some time-series features (such as DNS query frequency fluctuations) can reach O(n 2 (3) Some features contribute more to the detection accuracy, resulting in uneven feature representation capabilities.

[0003] Currently, common feature selection methods include: filter methods, wrapper methods, and embedding methods. Filter methods score features based on divergence or relevance, setting selection thresholds to filter features. This process is independent of model training, resulting in fast computation, but lower fit to the target problem and poor feature selection performance. Embedding methods use machine learning algorithms to train the model, obtaining weight coefficients for each feature, and selecting features based on these coefficients from largest to smallest. This method automatically selects features during learner training, achieving high fit to the target problem, but may carry the risk of overfitting. Furthermore, the model trainer must support obtaining feature weight coefficients, such as feature importance in tree models. Wrapper methods score features based on their prediction performance, continuously adding or removing features to test model accuracy and find the optimal feature subset. Because of model involvement, wrapper methods fit the target problem well, resulting in good feature extraction performance. As a non-integrated feature extraction method, it reduces the risk of overfitting and does not restrict the model trainer, making it a general feature extraction method. However, because the model needs to be retrained for each feature change, the computational cost is high.

[0004] There is currently no effective solution to the problem of how to improve feature selection performance while reducing the computational cost of feature selection. Summary of the Invention

[0005] The purpose of this application is to provide a feature selection method, apparatus, and electronic device that solves the problem that the prior art cannot improve the feature selection effect while reducing the computational overhead of feature selection.

[0006] To solve the above-mentioned technical problems, the first aspect of this specification provides a feature selection method, including:

[0007] Obtain the initial feature set to be selected. The initial feature set includes multiple feature dimensions, and each feature dimension includes at least one feature data.

[0008] Determine the correlation between each feature dimension in the initial feature set, and filter out a set of candidate features to be deleted from the initial feature set based on the correlation between each feature dimension.

[0009] The candidate feature set to be deleted is sorted according to at least one preset algorithm, and the candidate feature set to be deleted is divided into at least one set of feature subsets according to at least one sorting result and preset partitioning parameters. The multiple feature subsets in each set of feature subsets are obtained by partitioning a sorting result using a set of partitioning parameters.

[0010] For the initial feature set, each feature subset in the at least one set of feature subsets is deleted one by one to obtain multiple feature sets to be selected. The model prediction score obtained by training the target algorithm model using each feature set to be selected is determined, and the target feature set is selected as the feature selection result based on the multiple model prediction scores in the multiple feature sets to be selected.

[0011] In some embodiments of this specification, before sorting multiple feature dimensions in the candidate feature set to be deleted based on at least one preset algorithm, the method further includes:

[0012] The permutation importance of each feature dimension in the initial feature set is determined. The permutation importance is used to characterize the impact of the feature data of the corresponding feature dimension on the model prediction score of the target algorithm model when the data is out of order.

[0013] In some embodiments of this specification, the candidate feature set to be deleted is determined in the following manner:

[0014] The feature dimension whose permutation importance is less than a first preset threshold is determined as the target feature dimension; or, the feature dimension whose relevance importance is greater than a second preset threshold is determined as the target feature dimension.

[0015] The feature data of the target feature dimension are combined as the candidate feature set to be deleted.

[0016] In some embodiments of this specification, determining the permutation importance of each feature dimension in the initial feature set includes:

[0017] The target algorithm model is trained based on the initial feature set, and the model prediction score of the first target model obtained by training is determined as the baseline prediction score.

[0018] For any feature dimension in the initial feature set, randomly adjust the order of the feature data for that feature dimension.

[0019] The target algorithm model is trained based on the adjusted initial feature set, and the model prediction score of the trained second target model is determined as the candidate prediction score for the corresponding feature dimension.

[0020] Based on the baseline prediction score and the candidate prediction scores for each feature dimension, the permutation importance of each feature dimension is determined.

[0021] In some embodiments of this specification, the preset division parameters include multiple division dimensions;

[0022] Accordingly, based on at least one sorting result and preset partitioning parameters, the candidate feature set to be deleted is divided into at least one set of features, including:

[0023] For any sorting result, the candidate feature set to be deleted is divided according to each partitioning size to obtain multiple feature subsets.

[0024] In some embodiments of this specification, the preset algorithm includes at least one of the following: a first sorting algorithm based on permutation importance, and a natural sorting algorithm.

[0025] In some embodiments of this specification, sorting multiple feature dimensions in the candidate feature set to be deleted is performed based on at least one preset algorithm, including:

[0026] Based on the first sorting algorithm, multiple feature dimensions in the candidate feature set to be deleted are sorted to obtain a first sorting result; and / or,

[0027] The natural sorting algorithm is used to sort multiple feature dimensions in the candidate feature set to be deleted, and a second sorting result is obtained.

[0028] Accordingly, based on at least one sorting result and preset partitioning parameters, the candidate feature set to be deleted is divided into at least one set of feature subsets, including:

[0029] For the first sorting result and / or the second sorting result, the candidate feature set to be deleted is divided into multiple feature subsets by using multiple division dimensions in the preset division parameters.

[0030] The second aspect of this specification provides a feature selection method, including:

[0031] Obtain the initial feature set to be selected as the current feature set. The initial feature set includes multiple feature dimensions, and each feature dimension includes at least one feature data.

[0032] Determine the correlation between each feature dimension in the initial feature set, and based on the correlation between each feature dimension, filter out a set of candidate features to be deleted in the current feature set, and use the set of candidate features to be deleted as the current candidate feature set;

[0033] For the current feature set, iterative feature selection is performed, which includes:

[0034] The current candidate feature set is sorted based on at least one preset algorithm, and the current candidate feature set is divided into at least one set of feature subsets based on at least one sorting result and preset partitioning parameters. The multiple feature subsets in each set of feature subsets are obtained by partitioning a sorting result using a set of partitioning parameters.

[0035] For the current feature set and the current candidate feature set, each feature subset in the at least one set of feature subsets is deleted one by one to obtain multiple feature sets to be selected and multiple candidate feature sets. The model prediction score obtained by training the target algorithm model using each feature set to be selected is determined. Based on the multiple model prediction scores, the target feature set is selected from the multiple feature sets to be selected as the feature selection result of the current feature set. Each feature set to be selected corresponds to a candidate feature set.

[0036] The candidate feature set corresponding to the target feature set is used as the current candidate feature set, and the target feature set is used as the current feature set for the next feature selection.

[0037] If the candidate feature set to be deleted corresponding to the current feature set is empty, or if the model prediction score obtained by training the target algorithm model no longer changes, then the current feature set is taken as the result of iterative feature selection.

[0038] A third aspect of this specification provides a feature selection device, comprising:

[0039] The acquisition module is used to acquire the initial feature set to be selected. The initial feature set includes multiple feature dimensions, and each feature dimension includes at least one feature data.

[0040] The filtering module is used to determine the correlation between each feature dimension in the initial feature set, and to filter out a set of candidate features to be deleted from the initial feature set based on the correlation between each feature dimension.

[0041] The partitioning module is used to sort multiple feature dimensions in the candidate feature set to be deleted based on at least one preset algorithm, and to divide the candidate feature set to be deleted into at least one set of feature subsets based on at least one sorting result and preset partitioning parameters. The multiple feature subsets in each set of feature subsets are obtained by partitioning a sorting result using a set of partitioning parameters.

[0042] The selection module is used to delete each feature subset from the at least one set of feature subsets one by one for the initial feature set to obtain multiple feature sets to be selected, determine the model prediction score obtained by training the target algorithm model using each feature set to be selected, and select the target feature set as the feature selection result based on the multiple model prediction scores from the multiple feature sets to be selected.

[0043] A fourth aspect of this specification provides a feature selection device, comprising:

[0044] The acquisition module is used to acquire the initial feature set to be selected as the current feature set. The initial feature set includes multiple feature dimensions, and each feature dimension includes at least one feature data.

[0045] The filtering module is used to determine the correlation between each feature dimension in the initial feature set, and to filter out a set of candidate features to be deleted in the current feature set based on the correlation between each feature dimension, and to use the set of candidate features to be deleted as the current candidate feature set.

[0046] The iterative selection module is used to perform an iterative feature selection process on the current feature set, including:

[0047] A partitioning unit is used to sort multiple feature dimensions in the current candidate feature set based on at least one preset algorithm, and to divide the current candidate feature set into at least one set of feature subsets based on at least one sorting result and preset partitioning parameters. The multiple feature subsets in each set of feature subsets are obtained by partitioning a sorting result using a set of partitioning parameters.

[0048] The selection unit is configured to, for the current feature set and the current candidate feature set, delete each feature subset in the at least one set of feature subsets one by one to obtain multiple feature sets to be selected and multiple candidate feature sets, determine the model prediction score obtained by training the target algorithm model using each feature set to be selected, and select the target feature set as the feature selection result of the current feature set based on the multiple model prediction scores from the multiple feature sets to be selected, with each feature set to be selected corresponding to a candidate feature set;

[0049] An iterative unit is used to take the candidate feature set corresponding to the target feature set as the current candidate feature set, and take the target feature set as the current feature set for the next feature selection;

[0050] The termination unit is used to take the current feature set as the result of iterative feature selection if the candidate feature set to be deleted corresponding to the current feature set is empty, or if the model prediction score obtained by training the target algorithm model no longer changes.

[0051] A fifth aspect of this specification provides an electronic device, comprising: a memory and a processor, the processor and the memory being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to implement the steps of the aforementioned method.

[0052] A sixth aspect of this specification provides a computer storage medium storing computer program instructions that, when executed, implement the steps of the aforementioned method.

[0053] The seventh aspect of this specification provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the aforementioned method.

[0054] The feature selection method, apparatus, and electronic device in the embodiments of this specification involve: obtaining an initial feature set to be selected, the initial feature set including multiple feature dimensions, each feature dimension including at least one feature data; determining the correlation between each feature dimension in the initial feature set, and filtering out candidate feature sets to be deleted from the initial feature set based on the correlation between each feature dimension; sorting the multiple feature dimensions in the candidate feature set to be deleted based on at least one preset algorithm, and dividing the candidate feature set to be deleted into at least one set of feature subsets based on at least one sorting result and preset partitioning parameters, wherein the multiple feature subsets in each set of feature subsets are obtained by partitioning a sorting result using a set of partitioning parameters; deleting each feature subset in the at least one set of feature subsets one by one for the initial feature set, obtaining multiple feature sets to be selected; determining the model prediction score obtained by training the target algorithm model using each feature set to be selected, and selecting the target feature set as the feature selection result based on the multiple model prediction scores from the multiple feature sets to be selected. By employing the aforementioned method to screen the initial feature set using correlation calculations, the masking effect caused by strong correlations can be effectively mitigated, improving the accuracy and stability of feature selection. Furthermore, by dividing the candidate feature set to be deleted into subsets, and determining the impact of discarding corresponding feature subsets on the model prediction score of the target algorithm, feature selection efficiency can be improved, achieving both reduced computational overhead and enhanced performance. Moreover, this feature selection process does not rely on a specific target algorithm model structure, making it highly applicable, and it retains a decision log of the entire process, enhancing interpretability and reproducibility. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0056] Figure 1 The diagram shown is a schematic representation of a feature selection method provided in an embodiment of this specification.

[0057] Figure 2 The diagram shown is a schematic representation of a method for determining the importance of permutations provided in an embodiment of this specification.

[0058] Figure 3 The diagram shown is a schematic of a feature selection device provided in an embodiment of this specification;

[0059] Figure 4 The diagram shown is a schematic representation of a feature selection method provided in an embodiment of this specification.

[0060] Figure 5 The diagram shown is a schematic representation of the iterative feature selection process provided in an embodiment of this specification.

[0061] Figure 6 The diagram shown is a schematic of a feature selection device provided in an embodiment of this specification;

[0062] Figure 7 The diagram shown is a schematic representation of a feature selection process based on permutation importance and binning provided in an embodiment of this specification.

[0063] Figure 8 The diagram shown is a schematic of an electronic device provided in an embodiment of this specification. Detailed Implementation

[0064] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0065] It should be noted that the information and data related to users involved in the embodiments of this specification are all information and data authorized by the user or fully authorized by the relevant parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with relevant laws, regulations, and standards, do not violate public order and good morals, and provide corresponding operation entry points for users or relevant parties to choose to authorize or refuse.

[0066] It should also be noted that in the embodiments of this specification, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that this application has used or necessarily used such a solution.

[0067] Figure 1 The diagram illustrates a feature selection method provided in an embodiment of this specification. While this specification provides method operation steps or apparatus structures as shown in the following embodiments or figures, the method or apparatus may include more or fewer operation steps or module units, either combined or without inventive effort, based on conventional methods or without inventive effort. In steps or structures where there is no logically necessary causal relationship, the execution order of these steps or the module structure of the apparatus is not limited to the execution order or module structure shown in the embodiments or figures of this specification. When the method or module structure is applied in actual devices, servers, or terminal products, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiments or figures (e.g., in a parallel processor or multi-threaded processing environment, or even in a distributed processing or server cluster implementation environment). Figure 1 As shown, the method may include:

[0068] S101: Obtain the initial feature set to be selected. The initial feature set includes multiple feature dimensions, and each feature dimension includes at least one feature data.

[0069] Specifically, the initial feature set may include multiple feature sequences, each feature sequence may correspond to a feature dimension, and each feature sequence may include at least one data point, which is the feature data corresponding to the feature dimension.

[0070] S102: Determine the correlation between each feature dimension in the initial feature set, and filter out a set of candidate features to be deleted from the initial feature set based on the correlation between each feature dimension.

[0071] It's understandable that the correlation between feature dimensions can be used to measure their similarity. Highly correlated feature dimensions can lead to problems such as high redundancy, multicollinearity, obscured importance assessment, overfitting risk, and low efficiency due to duplicate features. High redundancy can be understood as highly correlated feature dimensions being almost linearly repetitive. Retaining two feature dimensions contributes only a limited increase to the available information in the target algorithm model, but increases the number of feature dimensions and training cost. Multicollinearity can be understood as, during the training of linear regression, logistic regression, and support vector machine (SVM) models, highly correlated feature dimensions can lead to unstable coefficients and increased variance, affecting interpretability and generalization. Obscured importance assessment can be understood as, in strongly correlated scenarios, a distribution / masking effect often occurs. When one feature dimension is replaced, another highly correlated feature dimension can fill the gap, causing the importance of the feature dimension to be underestimated, thus misleading the selection. Including highly correlated feature dimensions in the candidate feature set helps to break the masking effect. Overfitting risk can be understood as, highly correlated feature dimensions make the target algorithm model more prone to memorizing noisy patterns, reducing the robustness of the model in validation / testing. Furthermore, highly correlated feature dimensions can be regarded as duplicate feature dimensions. Duplicate feature dimensions can be filtered out through correlation, and then duplicate features can be filtered out in subsequent feature selection, which can improve the speed of training or prediction and improve the interpretability of the model.

[0072] It is understandable that the candidate feature set to be deleted can serve as a set of features to be confirmed for deletion during the feature selection process. The feature dimensions to be deleted can be determined from this candidate feature set. Specifically, when determining the candidate feature set to be deleted, feature data corresponding to feature dimensions with a relevance higher than a set threshold (e.g., 0.96) can be combined to form the candidate feature set to be deleted. This set includes at least two feature sequences, each consisting of at least one feature data, and each feature sequence corresponding to a feature dimension.

[0073] In some embodiments of this specification, the correlation between feature dimensions can be characterized by the Pearson correlation coefficient. That is, the correlation coefficient between each pair of feature dimensions is calculated using the Pearson correlation coefficient algorithm, and the calculated correlation coefficient is used as the correlation between the corresponding feature dimensions.

[0074] S103: Sort multiple feature dimensions in the candidate feature set to be deleted based on at least one preset algorithm, and divide the candidate feature set to be deleted into at least one set of feature subsets based on at least one sorting result and preset partitioning parameters. The multiple feature subsets in each set of feature subsets are obtained by partitioning a sorting result using a set of partitioning parameters.

[0075] It is understandable that the candidate feature set to be deleted can include multiple feature sequences, each of which can correspond to a feature dimension. The sorting of feature dimensions can be a sorting of multiple feature sequences. That is, by determining the sorting principle of the preset algorithm, such as based on natural ordering or importance ordering, the position of each feature sequence in the candidate feature set to be deleted can be adjusted based on this sorting principle. Furthermore, the sorting of multiple feature dimensions can be done using a single preset algorithm, resulting in one sorting result; or it can be done using multiple preset algorithms, resulting in a sorting result for each preset algorithm. Additionally, the sorting of multiple feature dimensions can also be done by partially or completely fusing multiple preset algorithms, resulting in at least one sorting result. For example, under different preset algorithms, each feature dimension will correspond to a value (e.g., importance, relevance, natural order, etc.). The values ​​corresponding to each feature dimension under different algorithms can be integrated to obtain a comprehensive value for each feature dimension, and a sorting result can be obtained based on this comprehensive value.

[0076] It is understood that the preset partitioning parameters can include at least one partitioning parameter, and each partitioning parameter can correspond to a partitioning method of the sorting result. For example, the partitioning parameter can be the number of feature dimensions (i.e., partition size) contained in each feature subset. Furthermore, the preset partitioning parameters can include at least one partition size, and each partition size can correspond to the number of feature dimensions of a feature subset. When partitioning feature subsets, for any sorting result, the sorted candidate feature set to be deleted can be partitioned using each set of partitioning parameters in the preset partitioning parameters to obtain multiple sets of feature subsets. Multiple feature subsets in a set of feature subsets correspond to the partitioning results of a set of partitioning parameters under the sorting result.

[0077] In some embodiments of this specification, the partitioning of feature subsets can be based on size, and can be of equal size, meaning that each feature subset contains the same number of feature dimensions. In this case, each set of partitioning parameters can include one partitioning size. Alternatively, it can be of non-uniform size, meaning that the number of feature dimensions in each feature subset is not necessarily the same. In this case, each set of partitioning parameters can include multiple partitioning sizes, and the number of partitioning sizes can correspond to the number of feature subsets in each set of feature subsets, and the number of feature dimensions in each feature subset. It is understood that in other embodiments, the partitioning of feature subsets can also be based on other factors, and this specification does not limit this.

[0078] S104: For the initial feature set, delete each feature subset in the at least one set of feature subsets one by one to obtain multiple feature sets to be selected, determine the model prediction score obtained by training the target algorithm model using each feature set to be selected, and select the target feature set as the feature selection result based on the multiple model prediction scores in the multiple feature sets to be selected.

[0079] It can be understood that after obtaining at least one set of feature subsets, for each set of feature subsets, each feature subset within that set is deleted from the initial feature set. One feature subset can be deleted at a time, resulting in multiple filtered initial feature sets as candidate feature sets. That is, based on each set of feature subsets, multiple deletion operations are performed on the initial feature set. Each deletion operation removes feature data from one feature dimension of that set from the initial feature set. The initial feature set after each deletion operation can serve as a candidate feature set. Multiple candidate feature sets corresponding to that set of feature subsets can then be obtained through these multiple deletion operations. If multiple sets of feature subsets exist, a set of multiple candidate feature sets can be obtained. Further, for each of the multiple candidate feature sets, the target algorithm model can be trained using each candidate feature set, resulting in a target model corresponding to each candidate feature set. The model prediction score of each target model is then determined, and a candidate feature set can be selected from the multiple candidate feature sets based on the multiple model prediction scores. For example, the feature set with the highest prediction score from the corresponding model can be selected as the feature selection result.

[0080] In some embodiments of this specification, correlation calculation is used to screen the initial feature set, which can effectively alleviate the masking effect caused by strong correlation and improve the accuracy and stability of feature selection. Furthermore, by dividing the candidate feature set to be deleted into subsets, and determining the impact of discarding corresponding feature subsets on the model prediction score of the target algorithm model, feature selection efficiency can be improved, achieving both reduced computational overhead and improved performance. In addition, the above feature selection process does not depend on a specific target algorithm model structure, making it highly applicable, and it can retain a decision log of the entire process, enhancing interpretability and reproducibility.

[0081] It is understood that the methods described in the embodiments of this specification can be applied to electronic devices, which can refer to electronic devices with data computing, processing, and storage capabilities. These electronic devices can be terminals such as PCs (Personal Computers), tablets, smartphones, wearable devices, and intelligent robots; they can also be servers. A server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0082] In some embodiments of this specification, a set of candidate features to be deleted can be obtained by using correlation, combined with permutation importance (PI), which measures the correlation between each feature dimension and the model's final prediction target. Permutation importance can be used to measure the feature representation ability of each feature dimension. Specifically, before sorting multiple feature dimensions in the set of candidate features to be deleted based on at least one preset algorithm, the following may also be included:

[0083] The permutation importance of each feature dimension in the initial feature set is determined. The permutation importance is used to characterize the impact of the feature data of the corresponding feature dimension on the model prediction score of the target algorithm model when the data is out of order.

[0084] It is understandable that the importance of permutation can be used to determine the correlation between the corresponding feature dimension and the model prediction or training. Feature dimensions with low permutation importance may be noisy feature dimensions. By using permutation importance, noisy feature dimensions in the initial feature set can be filtered out, thereby improving the model prediction accuracy.

[0085] Furthermore, the candidate feature set to be deleted can be determined in the following ways: determining the feature dimension whose permutation importance is less than a first preset threshold as the target feature dimension; or, determining the feature dimension whose relevance is greater than a second preset threshold as the target feature dimension; and combining the feature data of the target feature dimension as the candidate feature set to be deleted.

[0086] It is understandable that feature dimensions with a permutation importance less than the first preset threshold may be noisy feature dimensions, while feature dimensions with a relevance importance greater than the second preset threshold are feature dimensions with high similarity. These features are prone to problems such as high redundancy, multicollinearity, obscured importance assessment, overfitting risk, and low efficiency caused by duplicate features. Using the above two conditions as criteria for screening candidate feature sets to be deleted, as long as one of the conditions is met, it can be used as a feature dimension in the candidate feature set to be deleted. This can provide more accurate candidate feature dimensions to be deleted for feature selection, and then determine the feature dimensions to be deleted based on this, thereby improving the accuracy of feature selection.

[0087] refer to Figure 2 As shown, in some embodiments of this specification, determining the permutation importance of each feature dimension in the initial feature set may include:

[0088] S201: Train the target algorithm model based on the initial feature set, and determine the model prediction score of the first target model obtained by training as the baseline prediction score.

[0089] S202: For any feature dimension in the initial feature set, randomly adjust the order of the feature data corresponding to that feature dimension.

[0090] S203: Train the target algorithm model based on the adjusted initial feature set, and determine the model prediction score of the trained second target model as the candidate prediction score for the corresponding feature dimension.

[0091] S204: Based on the baseline prediction score and the candidate prediction scores of each feature dimension, determine the permutation importance of each feature dimension.

[0092] It can be understood that each feature dimension corresponds to a feature sequence, which includes multiple feature data arranged in a certain order. When determining the importance of permutations, the order of feature data in the feature sequence can be randomly adjusted. Then, the corresponding feature sequences in the initial feature set can be replaced with the adjusted feature sequences to obtain an adjusted initial feature set. Based on this, the target algorithm model can be trained, and candidate prediction scores corresponding to the adjusted initial feature set can be determined. Furthermore, the permutation importance of the corresponding feature dimension can be determined based on the candidate prediction scores and the baseline prediction scores.

[0093] In some embodiments of this specification, when determining the permutation importance of each feature dimension, the difference between the candidate predicted score and the baseline predicted score for the corresponding feature dimension can be determined, and this difference can be used as the permutation importance of the corresponding feature dimension. Of course, in other embodiments, other calculations can be performed on the candidate predicted score and the baseline predicted score, such as taking a ratio, and the corresponding calculation results can be used as the permutation importance. This specification does not limit this.

[0094] In some embodiments of this specification, the preset partitioning parameters may include multiple partitioning sizes. Furthermore, dividing the candidate feature set to be deleted into at least one set of features based on at least one ranking result and the preset partitioning parameters may include: for any ranking result, using each partitioning size to partition the candidate feature set to be deleted corresponding to that ranking result, thereby obtaining multiple sets of feature subsets.

[0095] In some embodiments of this specification, the preset algorithm may include at least one of the following: a first sorting algorithm based on permutation importance, and a natural sorting algorithm. In other embodiments, the preset algorithm may also include other sorting methods; the above-described first sorting algorithm based on permutation importance and natural sorting algorithm are merely examples of this specification.

[0096] In some embodiments of this specification, sorting multiple feature dimensions in the candidate feature set to be deleted based on at least one preset algorithm may include:

[0097] The first sorting algorithm is used to sort multiple feature dimensions in the candidate feature set to be deleted, resulting in a first sorting result; and / or, the natural sorting algorithm is used to sort multiple feature dimensions in the candidate feature set to be deleted, resulting in a second sorting result.

[0098] It is understandable that sorting the candidate feature set to be deleted based on a preset algorithm can be a sorting of multiple feature dimensions. Specifically, each feature dimension can correspond to a feature sequence, and thus sorting multiple feature dimensions can be a sorting of multiple feature sequences, with the order of multiple feature data in each feature sequence remaining unchanged.

[0099] Furthermore, dividing the candidate feature set to be deleted into at least one set of feature subsets based on at least one sorting result and preset partitioning parameters may include: for the first sorting result and / or the second sorting result, dividing the candidate feature set to be deleted into multiple sets of feature subsets by using multiple partitioning dimensions in the preset partitioning parameters respectively.

[0100] For example, the preset partitioning parameters include 8 partitioning sizes. The candidate feature set to be deleted is sorted using the first sorting algorithm and the natural sorting algorithm respectively, resulting in two sorting results. Based on the two sorting results and the 8 partitioning sizes, the re-sorted candidate feature set to be deleted is partitioned to obtain 16 feature subsets.

[0101] In the embodiments of this specification, an improved wrap-around backward search and filtering method is used to achieve feature selection, which can solve two major problems of conventional wrap-around backward search in large-scale feature scenarios: first, the high computational cost and limited search space (approximately O(T·N)) caused by feature-by-feature greedy deletion. 2First, it is difficult to efficiently find a good subset; second, the static order and strong correlation masking effect lead to selection bias and performance instability. The model-independent wrapper feature selection method in the embodiments of this specification uses a permutation importance threshold (i.e., the first threshold) and a relevance threshold (i.e., the second threshold) to accurately screen the candidate feature set to be deleted, and then uses a grouped backward search with "multiple bin sizes (corresponding partition sizes) × NAT (natural ordering) / PI (permutation importance) double sorting", which significantly reduces the computational cost (approximately O(T·(N / L)). 2 (where N represents the number of feature dimensions, T represents the time required for each training / validation of the model, and L represents the partition size), while improving the accuracy and robustness of feature selection.

[0102] Based on the feature selection method described above, one or more embodiments of this specification also provide a feature selection device. The device may include an apparatus (including a distributed system), software (application), module, plug-in, server, client, etc., using the method described in the embodiments of this specification, combined with necessary implementation hardware. Based on the same innovative concept, the devices in one or more embodiments provided in this specification are as described in the following embodiments. Since the implementation schemes and methods for solving the problem are similar, the implementation of specific devices in the embodiments of this specification can refer to the implementation of the foregoing method, and repeated details will not be repeated. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated. Figure 3 The diagram shown is a schematic representation of a feature selection device provided in an embodiment of this specification. Figure 3 As shown, the feature selection device 300 may include:

[0103] The acquisition module 301 is used to acquire the initial feature set to be selected. The initial feature set includes multiple feature dimensions, and each feature dimension includes at least one feature data.

[0104] The filtering module 302 is used to determine the correlation between each feature dimension in the initial feature set, and to filter out a set of candidate features to be deleted from the initial feature set based on the correlation between each feature dimension.

[0105] The partitioning module 303 is used to sort multiple feature dimensions in the candidate feature set to be deleted based on at least one preset algorithm, and to partition the candidate feature set to be deleted into at least one set of feature subsets based on at least one sorting result and preset partitioning parameters. The multiple feature subsets in each set of feature subsets are obtained by partitioning a sorting result using a set of partitioning parameters.

[0106] The selection module 304 is used to delete each feature subset in the at least one set of feature subsets one by one for the initial feature set to obtain multiple feature sets to be selected, determine the model prediction score obtained by training the target algorithm model using each feature set to be selected, and select the target feature set as the feature selection result based on the multiple model prediction scores in the multiple feature sets to be selected.

[0107] In some embodiments of this specification, the screening module 302 may also be used to: determine the permutation importance of each feature dimension in the initial feature set, wherein the permutation importance is used to characterize the influence of the feature data of the corresponding feature dimension on the model prediction score of the target algorithm model in the case of disordered order.

[0108] In some embodiments of this specification, the screening module 302 determines the candidate feature set to be deleted in the following ways: determining the feature dimension whose permutation importance is less than a first preset threshold as the target feature dimension; or, determining the feature dimension whose relevance is greater than a second preset threshold as the target feature dimension; and combining the feature data of the target feature dimension as the candidate feature set to be deleted.

[0109] In some embodiments of this specification, when determining the permutation importance of each feature dimension in the initial feature set, the screening module 302 is specifically used for: training the target algorithm model based on the initial feature set, and determining the model prediction score of the trained first target model as the baseline prediction score; randomly adjusting the order of feature data for any feature dimension in the initial feature set; training the target algorithm model based on the adjusted initial feature set, and determining the model prediction score of the trained second target model as the candidate prediction score for the corresponding feature dimension; and determining the permutation importance of each feature dimension based on the baseline prediction score and the candidate prediction scores of each feature dimension.

[0110] In some embodiments of this specification, the preset partitioning parameters include multiple partitioning sizes; correspondingly, when the partitioning module 303 divides the candidate feature set to be deleted into at least one set of features based on at least one sorting result and the preset partitioning parameters, it is specifically used to: for any sorting result, divide the candidate feature set to be deleted of the corresponding sorting result into multiple sets of feature subsets using each partitioning size.

[0111] In some embodiments of this specification, the preset algorithm includes at least one of the following: a first sorting algorithm based on permutation importance, and a natural sorting algorithm.

[0112] In some embodiments of this specification, when the partitioning module 303 sorts multiple feature dimensions in the candidate feature set to be deleted based on at least one preset algorithm, it is specifically used to: sort the multiple feature dimensions in the candidate feature set to be deleted based on the first sorting algorithm to obtain a first sorting result; and / or, sort the multiple feature dimensions in the candidate feature set to be deleted based on the natural sorting algorithm to obtain a second sorting result. Further, when the partitioning module 303 divides the candidate feature set to be deleted into at least one set of feature subsets based on at least one sorting result and preset partitioning parameters, it is specifically used to: for the first sorting result and / or the second sorting result, divide the candidate feature set to be deleted into multiple sets of feature subsets using multiple partitioning sizes in the preset partitioning parameters.

[0113] The descriptions and functions of the above modules can be found in the section on feature selection methods, and will not be repeated here.

[0114] Based on the aforementioned feature selection method, this specification also provides a feature selection method, as described in the embodiments. Figure 4 and Figure 5 As shown, the method may include:

[0115] S401: Obtain the initial feature set to be selected as the current feature set. The initial feature set includes multiple feature dimensions, and each feature dimension includes at least one feature data.

[0116] S402: Determine the correlation between each feature dimension in the initial feature set, and based on the correlation between each feature dimension, filter out a candidate feature set to be deleted in the current feature set, and use the candidate feature set to be deleted as the current candidate feature set;

[0117] S403: Perform iterative feature selection on the current feature set.

[0118] The iterative feature selection includes:

[0119] S501: Sort multiple feature dimensions in the current candidate feature set based on at least one preset algorithm, and divide the current candidate feature set into at least one set of feature subsets based on at least one sorting result and preset partitioning parameters. The multiple feature subsets in each set of feature subsets are obtained by partitioning a sorting result using a set of partitioning parameters.

[0120] S502: For the current feature set and the current candidate feature set, delete each feature subset in the at least one set of feature subsets one by one to obtain multiple feature sets to be selected and multiple candidate feature sets. Determine the model prediction score obtained by training the target algorithm model using each feature set to be selected. Based on the multiple model prediction scores, select the target feature set from the multiple feature sets to be selected as the feature selection result of the current feature set. Each feature set to be selected corresponds to a candidate feature set.

[0121] S503: Use the candidate feature set corresponding to the target feature set as the current candidate feature set, and use the target feature set as the current feature set for the next feature selection;

[0122] S504: If the candidate feature set to be deleted corresponding to the current feature set is empty, or the model prediction score obtained by training the target algorithm model no longer changes, then the current feature set is taken as the result of iterative feature selection.

[0123] It is understood that in step S502, feature subsets can be deleted from both the current feature set and the current candidate feature set. Each time, one feature subset can be deleted, resulting in a selectable feature set and a candidate feature set. The selectable and candidate feature sets obtained by deleting the same feature subset correspond. That is, the deleted feature subsets can be used as feature subsets to be filtered out after the feature selection process. After filtering out the corresponding feature subset in the current feature set, the corresponding current candidate feature set should also not contain any feature subsets that need to be deleted. Of course, in other embodiments, only the corresponding feature subsets of the current feature set can be deleted one by one. After determining the target feature set, the feature subsets to be deleted in the current iteration cycle can be determined based on the target feature set, and the corresponding feature subsets deleted from the current candidate feature set can be used as objects for sorting and partitioning in the next iteration.

[0124] It is understandable that steps S401, S402, S501, and S502 are related to the previous steps. Figure 1 The specific implementation processes of steps S101 to S104 are basically the same or similar, and can be referred to the previous text. Figure 1 The descriptions of each step are omitted here. The feature selection method in the embodiments of this specification can be achieved by... Figure 1 The feature selection method in [the document] is implemented through multiple iterations, that is... Figure 1The feature selection process in this embodiment involves using the result of each feature selection as input for the next. After multiple iterations, the iterative feature selection ends when the candidate feature set corresponding to the current feature set is empty, or when the model prediction score obtained from training the target algorithm no longer changes. The current feature set obtained is then the result of the iterative feature selection. Furthermore, the feature selection method in this embodiment, by retraining and recalculating PI after each round of bin deletion, can achieve dynamic updates of importance, further improving the accuracy of feature selection.

[0125] Based on the above feature selection method, embodiments of this specification also provide a feature selection device, referencing... Figure 6 As shown, the feature selection device 600 may include:

[0126] The acquisition module 601 is used to acquire the initial feature set to be selected as the current feature set. The initial feature set includes multiple feature dimensions, and each feature dimension includes at least one feature data.

[0127] The filtering module 602 is used to determine the correlation between each feature dimension in the initial feature set, and to filter out a candidate feature set to be deleted in the current feature set based on the correlation between each feature dimension, and to use the candidate feature set to be deleted as the current candidate feature set.

[0128] Iterative selection module 603 is used to perform an iterative feature selection process on the current feature set, including:

[0129] The partitioning unit 6031 is used to sort multiple feature dimensions in the current candidate feature set based on at least one preset algorithm, and to divide the current candidate feature set into at least one set of feature subsets based on at least one sorting result and preset partitioning parameters. The multiple feature subsets in each set of feature subsets are obtained by partitioning a sorting result using a set of partitioning parameters.

[0130] Selection unit 6032 is used to delete each feature subset in the at least one set of feature subsets one by one for the current feature set and the current candidate feature set, to obtain multiple feature sets to be selected and multiple candidate feature sets, to determine the model prediction score obtained by training the target algorithm model using each feature set to be selected, and to select the target feature set as the feature selection result of the current feature set based on the multiple model prediction scores from the multiple feature sets to be selected, wherein each feature set to be selected corresponds to a candidate feature set;

[0131] The iteration unit 6033 is used to take the candidate feature set corresponding to the target feature set as the current candidate feature set, and take the target feature set as the current feature set for the next feature selection;

[0132] Termination unit 6034 is used to take the current feature set as the result of iterative feature selection if the candidate feature set to be deleted corresponding to the current feature set is empty, or if the model prediction score obtained by training the target algorithm model no longer changes.

[0133] The descriptions and functions of the above modules can be found in the section on feature selection methods, and will not be repeated here.

[0134] The embodiments in this specification also provide an automated feature selection strategy based on permutation importance and binning, including two key steps: subset evaluation based on permutation importance and backward search based on binning.

[0135] (1) Subset evaluation based on permutation importance

[0136] Subset evaluation primarily uses evaluation metrics to determine the next candidate subset of features. In this embodiment, the importance of each feature (i.e., feature dimension) is measured using permutation importance, while the relevance of features is measured based on the Pearson correlation coefficient. Features with low permutation importance values ​​and high correlation coefficients are selected to form the candidate subset of features to be deleted.

[0137] In this algorithm, permutation importance assesses feature importance by calculating the impact of shuffling the feature sequence on the model's prediction score. The "permutation operation" disrupts the relationship between the feature and the model's prediction objective. Therefore, a decrease in the model's prediction score can indicate the model's dependence on that feature, thus describing the feature's importance. The pseudocode for the permutation importance algorithm is shown in Algorithm 1.

[0138] Algorithm 1 CALC_PI_SCORE

[0139] Input: The entire feature set (feature_set).

[0140] Output: List of permutation importance PI score .

[0141]

[0142] The specific steps of Algorithm 1, i.e., the CALC_PI_SCORE function, may include: training a model on the entire feature set (i.e., the initial feature set), and recording the model's prediction score at this point as the baseline score (corresponding to line 1 of Algorithm 1). Then, iterating through the features in the entire feature set, for any feature f... i Randomly shuffle its feature sequence, calculate the model prediction score corresponding to the new feature set (corresponding to line 3 of Algorithm 1), and use the difference between this score and the baseline score as the permutation importance of the feature (corresponding to line 4 of Algorithm 1).

[0143] Permutation importance calculation is fast, and the objectives of feature importance assessment and model evaluation are consistent, making it applicable to any algorithm model and highly interpretable. However, when multicollinear or correlated features exist in the entire feature set, permuting a single feature column does not significantly affect the model's prediction score, making it difficult to accurately assess feature importance. Therefore, the embodiments in this specification also introduce consideration of feature correlation, specifically using the Pearson correlation coefficient to measure the correlation between two features, as shown in Formula 1 below. Where f i f j Let f represent the i-th and j-th feature vectors (corresponding feature sequences) respectively. ik Represents the eigenvector f i The k-th dimension, with a total of n dimensions. Represents the eigenvector f i The mean, Pearson correlation coefficient corr(f) i ,f j The larger the value, the more correlated the two features are.

[0144]

[0145] (2) Binning-based backward search

[0146] The backward search strategy is a typical subset search strategy in wrapper-style feature selection algorithms. Given a set of candidate features to be deleted, it successively removes one feature at a time, ensuring the predicted score shows a monotonically increasing trend until it reaches its maximum. This is a greedy strategy with high computational complexity. This specification's embodiments propose a binning-based backward search strategy: for the set of candidate features to be deleted, it is divided into multiple fixed-size feature bins (i.e., the feature subsets mentioned above). Each backward search removes any feature bin, which contains multiple features. By simultaneously removing multiple features, the problem of the greedy strategy getting trapped in local optima can be avoided to some extent, and computational overhead can be effectively reduced. This specification's embodiments use an equal-frequency binning strategy, dividing the set of candidate features to be deleted into n bins (intervals), each interval containing approximately the same number of features. The pseudocode is shown in Algorithm 2.

[0147] Algorithm 2BIN_SBS

[0148] Input: bin size (bin_size), candidate feature set (candidate_set), feature set (feature_set).

[0149] Output: Best score (best_score), best feature set (best_fl).

[0150]

[0151] The specific steps of the BIN_SBS function in Algorithm 2 can include: performing equal-frequency binning with a bin size of bin_size on the candidate feature set to be deleted, resulting in multiple lists of feature boxes to be deleted, del_list (corresponding to line 1 of Algorithm 2). Then, iterating through any one of the feature boxes to be deleted, del_fs, calculates the model prediction score on the new feature set obtained by discarding any del_fs from the entire feature set, and saves it to the prediction score list, score_list (corresponding to lines 2-4 of Algorithm 2). Next, the highest prediction score in score_list is taken as the best_score output, and the feature box to be deleted corresponding to best_score is taken as the optimal feature box to be deleted, del_fl. The optimal feature box to be deleted, del_fl, is then discarded from the entire feature set, feature_set, to obtain the optimal feature set, best_fl (corresponding to lines 5-7 of Algorithm 2).

[0152] In the embodiments of this specification, different bin size sequences (bin_size) can be set to capture feature subsets (feature bins) of different lengths to be deleted. Simultaneously, the candidate feature sets to be deleted are sorted according to two rules: one is natural sorting, which attempts to remove features with consistent or similar detection approaches in batches, resulting in strong interpretability; the other is PI value sorting, which attempts to remove features with low model dependence in batches, leading to faster performance improvement.

[0153] The feature selection schemes described in this specification can be applied to different models and different fields. For the same feature set, different algorithm models can extract different feature subsets. The entire feature selection process can be an iterative process, as shown in the pseudocode of Algorithm 3.

[0154] The FS_PI_BIN function corresponding to Algorithm 3 describes an automated feature selection strategy based on permutation importance and binning. This algorithm takes the original feature set `feature_set`, the permutation importance threshold `PI_thr`, the Pearson correlation coefficient threshold `Corr_thr`, and the bin size list `bin_size_list` as input; and outputs the final feature set `feature_selection`.

[0155] First, initialize the model to predict the highest score (best_score) and the optimal feature set (best_fl) (lines 24-25). Then, iteratively execute the feature subset evaluation function PI_FS_EVAL based on permutation importance and the feature selection function BIN_FS based on binning until the candidate subset is empty or the predicted score no longer improves (lines 26-31). Finally, output the current feature set as the feature selection result (feature_selection) (line 32). The two functions are described below:

[0156] Algorithm 3FS_PI_BIN

[0157] Input: feature set (feature_set), permutation importance threshold (PI) thr Correlation coefficient threshold Corr thr Bin size list.

[0158] Output: The feature set `feature_selection` after feature selection.

[0159]

[0160]

[0161] (1) The feature subset evaluation function PI_FS_EVAL based on permutation importance (corresponding to lines 1 to 10 in Algorithm 3): calculates the permutation importance score and Pearson correlation coefficient of all features on the feature set (corresponding to lines 2 to 3 in Algorithm 3), and further selects features that meet the conditions according to the threshold to form a candidate feature set to be deleted (corresponding to lines 4 to 8 in Algorithm 3), and uses it as the function return value (corresponding to line 9 in Algorithm 3).

[0162] (2) Bin-based feature selection function BIN_FS (corresponding to lines 11-23 in Algorithm 3): First, perform natural sorting and permutation importance sorting on the candidate feature set candidate_set (corresponding to lines 12-13 in Algorithm 3). Then, traverse different bin sizes bin_size and perform natural sorting fl respectively. nat Importance ranking of permutations fl pi The binning-based backward search function BIN_SBS is used (corresponding to lines 14-29 in Algorithm 3). Finally, the highest model prediction score and the corresponding feature list are taken and assigned the values ​​best_score and best_fl, and returned as the function output values ​​(corresponding to lines 20-23 in Algorithm 3).

[0163] The automated feature selection method based on permutation importance and binning described in this specification can eliminate invalid, redundant, and noisy features, reducing the computational cost of feature selection and improving detection performance. Taking the feature selection process in malicious domain name detection as an example, the effectiveness of the above-mentioned automated feature selection method based on permutation importance and binning is verified through experiments. Two feature sets are set up for the experiment, as shown in Table 1.

[0164] Table 1

[0165]

[0166] The Feature_Append group represents a set of general features that have been used in multiple literatures in the field of malicious domain name detection. These features mainly include domain name character features, DNS traffic features, and Whois features.

[0167] The Feature Selection group performs a feature selection method based on permutation importance and binning on 299 features in the full feature set. The specific parameters of the feature selection method are set as follows: 1) The threshold for initializing the subset of candidate features to be deleted is set to PI. thr =7E-6,Corr thr =0.95; 2) Bin size sequence bin_size_list = [5, 10, 15, 25, 50]. The entire feature selection process is iteratively performed under different label ratios (c ∈ [0.1~0.9]) and different evaluation metrics (F1 and MCC) to obtain a 92-dimensional feature set. For example, the feature selection process under a label ratio c = 0.5 and the F1 metric can be found in [link to documentation]. Figure 7 As shown, the horizontal axis represents the iteration time (i.e., Figure 7 The iterate time (in the figure) is represented by the vertical axis, which is the model prediction score (i.e., the iteration time). Figure 7 (F1_score in the text).

[0168] refer to Figure 7 As shown, Figure 7 The overall feature selection process involved 8 iterations, with the first 7 being effective. The 8th iteration failed to improve detection accuracy, and the feature selection process terminated. In the early stages of iteration, natural ranking was superior to permutation importance ranking. This was primarily observed from the perspective of feature extraction (natural ranking, i.e....). Figure 7 Feature removal is performed using the curve corresponding to NAT in the middle, and then the feature is eliminated based on the importance of PI (i.e., Figure 7 The curve corresponding to pi in the model is used to remove features with low model dependency. Furthermore, the bin size is generally larger in the early stages of iteration, and gradually decreases in subsequent iterations, achieving an effective feature selection scheme of first broad-based screening and then fine-grained screening. Figure 7It can be seen that, in the field of malicious domain detection, feature selection methods can effectively eliminate invalid, redundant, and noisy features, enhance the representational ability of feature sets, reduce feature computation costs, and improve detection performance.

[0169] In the embodiments of this specification, feature selection strongly aligned with task metrics is achieved through "dynamic closed-loop driven by permutation importance + binning and backward search": On the one hand, the candidate feature set to be deleted is accurately locked using PI threshold and high correlation threshold, and PI is retrained and recalculated after each round of binning, effectively mitigating the masking effect caused by strong correlation and improving the accuracy and robustness of selection; on the other hand, parallel evaluation using multiple bin sizes and NAT / PI dual sorting, and adoption of the globally optimal binning in each round, transforms the traditional wrapper-style O(T·N) binning into a more robust and efficient feature selection mechanism. 2 The training overhead is approximately reduced to O(T·(N / L)). 2 This significantly improves efficiency in large-scale scenarios; at the same time, it is method- and model-agnostic and retains the entire process decision log, enhancing interpretability and reproducibility.

[0170] This application also provides an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 801 and a memory 802, wherein the processor 801 and the memory 802 may be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0171] Processor 801 can be a Central Processing Unit (CPU). Processor 801 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0172] The memory 802, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the feature selection method in the embodiments of the present invention. The processor 801 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions, and modules stored in the memory 802, thereby implementing the feature selection method in the above method embodiments.

[0173] The memory 802 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor 801, etc. Furthermore, the memory 802 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 802 may optionally include memory remotely located relative to the processor 801, and these remote memories may be connected to the processor 801 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0174] The one or more modules are stored in the memory 802, and when executed by the processor 801, they perform the aforementioned functions. Figure 1 or Figure 4 Feature selection methods in [the context of the text].

[0175] The specific details of the aforementioned electronic device can be understood by referring to the relevant descriptions and effects in the above method embodiments, and will not be repeated here.

[0176] This specification also provides a computer storage medium storing computer program instructions that, when executed, implement the steps of the above-described feature selection method.

[0177] This specification also provides a computer program product, which includes a computer program that, when executed, implements the steps of the method for determining the fracturing construction parameters described above.

[0178] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0179] The various embodiments in this specification are described in a progressive manner. For the same or similar parts between the various embodiments, please refer to each other. The focus of each embodiment is to describe the differences from other embodiments.

[0180] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions.

[0181] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0182] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute certain parts of the methods of various embodiments of this application.

[0183] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0184] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0185] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to the embodiments described herein by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A feature selection method, characterized in that, include: Obtain the initial feature set to be selected. The initial feature set includes multiple feature dimensions, and each feature dimension includes at least one feature data. Determine the correlation between each feature dimension in the initial feature set, and filter out a set of candidate features to be deleted from the initial feature set based on the correlation between each feature dimension. The candidate feature set to be deleted is sorted according to at least one preset algorithm, and the candidate feature set to be deleted is divided into at least one set of feature subsets according to at least one sorting result and preset partitioning parameters. The multiple feature subsets in each set of feature subsets are obtained by partitioning a sorting result using a set of partitioning parameters. For the initial feature set, each feature subset in the at least one set of feature subsets is deleted one by one to obtain multiple feature sets to be selected. The model prediction score obtained by training the target algorithm model using each feature set to be selected is determined, and the target feature set is selected as the feature selection result based on the multiple model prediction scores in the multiple feature sets to be selected.

2. The feature selection method according to claim 1, characterized in that, Before sorting multiple feature dimensions in the candidate feature set to be deleted based on at least one preset algorithm, the method further includes: The permutation importance of each feature dimension in the initial feature set is determined. The permutation importance is used to characterize the impact of the feature data of the corresponding feature dimension on the model prediction score of the target algorithm model when the data is out of order.

3. The feature selection method according to claim 2, characterized in that, The candidate feature set to be deleted is determined in the following way: The feature dimension whose permutation importance is less than a first preset threshold is determined as the target feature dimension; or, the feature dimension whose relevance importance is greater than a second preset threshold is determined as the target feature dimension. The feature data of the target feature dimension are combined as the candidate feature set to be deleted.

4. The feature selection method according to claim 2, characterized in that, Determining the permutation importance of each feature dimension in the initial feature set includes: The target algorithm model is trained based on the initial feature set, and the model prediction score of the first target model obtained by training is determined as the baseline prediction score. For any feature dimension in the initial feature set, randomly adjust the order of the feature data for that feature dimension. The target algorithm model is trained based on the adjusted initial feature set, and the model prediction score of the trained second target model is determined as the candidate prediction score for the corresponding feature dimension. Based on the baseline prediction score and the candidate prediction scores for each feature dimension, the permutation importance of each feature dimension is determined.

5. The feature selection method according to claim 2, characterized in that, The preset division parameters include multiple division dimensions; Accordingly, based on at least one sorting result and preset partitioning parameters, the candidate feature set to be deleted is divided into at least one set of features, including: For any sorting result, the candidate feature set to be deleted is divided according to each partitioning size to obtain multiple feature subsets.

6. The feature selection method according to claim 2 or 5, characterized in that, The preset algorithm includes at least one of the following: a first sorting algorithm based on permutation importance, and a natural sorting algorithm.

7. The feature selection method according to claim 6, characterized in that, The candidate feature set to be deleted is sorted according to at least one preset algorithm, including: Based on the first sorting algorithm, multiple feature dimensions in the candidate feature set to be deleted are sorted to obtain a first sorting result; and / or, The natural sorting algorithm is used to sort multiple feature dimensions in the candidate feature set to be deleted, and a second sorting result is obtained. Accordingly, based on at least one sorting result and preset partitioning parameters, the candidate feature set to be deleted is divided into at least one set of feature subsets, including: For the first sorting result and / or the second sorting result, the candidate feature set to be deleted is divided into multiple feature subsets by using multiple division dimensions in the preset division parameters.

8. A feature selection method, characterized in that, include: Obtain the initial feature set to be selected as the current feature set. The initial feature set includes multiple feature dimensions, and each feature dimension includes at least one feature data. Determine the correlation between each feature dimension in the initial feature set, and based on the correlation between each feature dimension, filter out a set of candidate features to be deleted in the current feature set, and use the set of candidate features to be deleted as the current candidate feature set; For the current feature set, iterative feature selection is performed, which includes: The current candidate feature set is sorted based on at least one preset algorithm, and the current candidate feature set is divided into at least one set of feature subsets based on at least one sorting result and preset partitioning parameters. The multiple feature subsets in each set of feature subsets are obtained by partitioning a sorting result using a set of partitioning parameters. For the current feature set and the current candidate feature set, each feature subset in the at least one set of feature subsets is deleted one by one to obtain multiple feature sets to be selected and multiple candidate feature sets. The model prediction score obtained by training the target algorithm model using each feature set to be selected is determined. Based on the multiple model prediction scores, the target feature set is selected from the multiple feature sets to be selected as the feature selection result of the current feature set. Each feature set to be selected corresponds to a candidate feature set. The candidate feature set corresponding to the target feature set is used as the current candidate feature set, and the target feature set is used as the current feature set for the next feature selection. If the candidate feature set to be deleted corresponding to the current feature set is empty, or if the model prediction score obtained by training the target algorithm model no longer changes, then the current feature set is taken as the result of iterative feature selection.

9. A feature selection device, characterized in that, include: The acquisition module is used to acquire the initial feature set to be selected. The initial feature set includes multiple feature dimensions, and each feature dimension includes at least one feature data. The filtering module is used to determine the correlation between each feature dimension in the initial feature set, and to filter out a set of candidate features to be deleted from the initial feature set based on the correlation between each feature dimension. The partitioning module is used to sort multiple feature dimensions in the candidate feature set to be deleted based on at least one preset algorithm, and to divide the candidate feature set to be deleted into at least one set of feature subsets based on at least one sorting result and preset partitioning parameters. The multiple feature subsets in each set of feature subsets are obtained by partitioning a sorting result using a set of partitioning parameters. The selection module is used to delete each feature subset from the at least one set of feature subsets one by one for the initial feature set to obtain multiple feature sets to be selected, determine the model prediction score obtained by training the target algorithm model using each feature set to be selected, and select the target feature set as the feature selection result based on the multiple model prediction scores from the multiple feature sets to be selected.

10. A feature selection device, characterized in that, include: The acquisition module is used to acquire the initial feature set to be selected as the current feature set. The initial feature set includes multiple feature dimensions, and each feature dimension includes at least one feature data. The filtering module is used to determine the correlation between each feature dimension in the initial feature set, and to filter out a set of candidate features to be deleted in the current feature set based on the correlation between each feature dimension, and to use the set of candidate features to be deleted as the current candidate feature set. The iterative selection module is used to perform an iterative feature selection process on the current feature set, including: A partitioning unit is used to sort multiple feature dimensions in the current candidate feature set based on at least one preset algorithm, and to divide the current candidate feature set into at least one set of feature subsets based on at least one sorting result and preset partitioning parameters. The multiple feature subsets in each set of feature subsets are obtained by partitioning a sorting result using a set of partitioning parameters. The selection unit is configured to, for the current feature set and the current candidate feature set, delete each feature subset in the at least one set of feature subsets one by one to obtain multiple feature sets to be selected and multiple candidate feature sets, determine the model prediction score obtained by training the target algorithm model using each feature set to be selected, and select the target feature set as the feature selection result of the current feature set based on the multiple model prediction scores from the multiple feature sets to be selected, with each feature set to be selected corresponding to a candidate feature set; An iterative unit is used to take the candidate feature set corresponding to the target feature set as the current candidate feature set, and take the target feature set as the current feature set for the next feature selection; The termination unit is used to take the current feature set as the result of iterative feature selection if the candidate feature set to be deleted corresponding to the current feature set is empty, or if the model prediction score obtained by training the target algorithm model no longer changes.

11. An electronic device, characterized in that, include: A memory and a processor, the processor and the memory being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to implement the steps of the method according to any one of claims 1 to 8.

12. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed, implement the steps of the method according to any one of claims 1 to 8.

13. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images