Feature selection method and device, equipment and storage medium
By introducing horizontal crossing, vertical crossing and adaptive differential evolution mechanisms into the exponential distribution optimizer, EDO is easily trapped in local optimality in high-dimensional feature selection, achieving more efficient search space exploration and better feature selection effects.
Patent Information
- Application Number
- CN202510167907.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-17
- Publication Date
- 2025-03-14
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing exponential distribution optimizers (EDOs) are prone to falling into local optimal solutions in high-dimensional feature selection problems, limiting their effectiveness in exploring the entire search space and responding to complex challenges.
Horizontal crossing, vertical crossing and adaptive differential evolution mechanisms are introduced into the exponential distribution optimizer, through which individual positions in the population are updated to form an improved exponential distribution optimizer.
By adding these update mechanisms, the algorithm can more effectively explore the search space and jump out of local optimal solutions, improving the classification accuracy of feature selection and the ability to cope with complex challenges.
Smart Images
Figure CN119622299A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a feature selection method, device, equipment and storage medium. Background Art
[0002] In today's world of rapid technological development, the size and complexity of data are growing at an unprecedented rate. Classification tasks are a basic and crucial form of data analysis, and accurate and efficient classification algorithms have become a core research topic in the field of data science. However, with the increase in data dimensions and the complexity of classification problems, traditional classification algorithms face problems such as low efficiency, easy to fall into local optimality, and sensitivity to noisy data.
[0003] Feature selection is an important step in data preprocessing and is crucial to improving the performance of classification algorithms. High-dimensional datasets usually contain a large number of features, some of which may be irrelevant or redundant to the prediction task. These unnecessary features not only increase the computational burden during model training, but may also reduce classification accuracy, so feature selection is required for high-dimensional datasets.
[0004] Metaheuristic algorithms are a type of optimization technology that draws inspiration from natural processes. They provide a powerful global search capability by simulating natural phenomena such as evolutionary processes. They are highly praised by researchers for their ability to effectively find optimal solutions in a vast search space and are widely used in engineering optimization, data mining, and machine learning. Recent research will focus on developing new strategies and mechanisms to improve the search capabilities and efficiency of heuristic algorithms, including introducing various search strategies to balance global and local searches, using chaos theory and complex network theory to prevent premature convergence, and using adaptive and learning mechanisms to dynamically adjust algorithm parameters. Through these improvements, researchers hope to enable the algorithm to converge to the global optimal solution faster while maintaining effective exploration of complex problem spaces. The improved algorithm has shown high competitiveness in dealing with specific problems.
[0005] Exponential Distribution Optimizer (EDO), as a meta-heuristic algorithm, has shown great potential in multiple application fields. EDO is an optimization algorithm proposed based on the mathematical model of exponential probability distribution. It initializes a group of random solutions representing multiple exponential distribution models, and the position of each solution represents an exponential random variable. EDO updates the current solution based on the relationship between memorylessness, guided solutions, and exponential random variables. However, when applying EDO to high-dimensional feature selection problems, algorithms often fall into local optimal solutions, which hinders their ability to fully explore the entire search space, limits their effectiveness in dealing with complex challenges, and reduces the classification accuracy of feature selection. Summary of the invention
[0006] The present invention provides a feature selection method, apparatus, device and storage medium, which solve the problem that the existing EDO algorithms often fall into local optimal solutions in high-dimensional feature selection problems, which hinders their ability to fully explore the entire search space and limits their effectiveness in dealing with complex challenges.
[0007] In a first aspect, the present invention provides a feature selection method, comprising the following steps: Take the feature subset in the high-dimensional data set to be processed as individuals, and construct the initial population through multiple individuals; Get the fitness value of each individual in the initial population; The positions of multiple individuals in the initial population are updated by an improved exponential distribution optimizer to obtain a new population; wherein, horizontal crossover, vertical crossover and adaptive differential evolution mechanism are added to the exponential distribution optimizer to obtain an improved exponential distribution optimizer; the horizontal crossover and vertical crossover are for updating the individual positions through the mutual crossover results of the individual characteristic values in different dimensions, and the adaptive differential evolution mechanism is for dynamically adjusting the scaling factor through the individual fitness value, and updating the individual position through the scaling factor; Obtain the fitness values of multiple individuals in the new population and determine whether they meet the termination conditions. If not, iterate the new population again through the improved exponential distribution optimizer and determine the fitness value. If it meets the conditions, select the individual with the highest fitness value from the current population as the optimal solution, that is, the optimal feature subset.
[0008] Preferably, the fitness value of each individual in the initial population is obtained as follows: ; In the formula, is the fitness value, is the classification accuracy weight, is the classification error rate, R is the length of the selected feature subset, N is the total number of features, is the weight of the number of selected features.
[0009] Preferably, the updating of the positions of multiple individuals in the initial population by using an improved exponential distribution optimizer to obtain a new population comprises the following steps: Each individual in the initial population is updated in turn through the exponential distribution optimizer, horizontal crossover, vertical crossover and adaptive differential evolution mechanism to obtain the first new individual, the second new individual, the third new individual and the fourth new individual corresponding to each individual; The best individual among the first new individual, the second new individual, the third new individual and the fourth new individual corresponding to each individual is obtained based on the fitness value; Replace each individual in the initial population with the corresponding optimal individual to obtain a new population.
[0010] Preferably, the updating of the positions of any two individuals in the initial population based on horizontal crossover and vertical crossover to obtain multiple new individuals specifically includes: ; ; In the formula, and are two new individuals obtained by horizontal crossover, and is a random variable, and For the parent individual and In the n The eigenvalues of the dimensions, and is a random variable; ; In the formula, is the new individual obtained by vertical crossover, and is a random number, is the crossover rate of vertical crossover.
[0011] Preferably, the adaptive differential evolution mechanism dynamically adjusts the scaling factor through the fitness value of the individual, and updates the individual position through the scaling factor, specifically including: ; In the formula, and is the fitness value of the two individuals, k is the scaling factor; ; In the formula, To obtain new individuals through adaptive differential evolution mechanism, is the parent individual to be mutated, and are two random parent individuals.
[0012] Preferably, before obtaining the fitness value of each individual in the initial population and obtaining the fitness values of multiple individuals in the new population, it is necessary to perform binary coding on the multiple individuals in the initial population and the new population, which specifically includes: ; ; In the formula, Indicates t The first i The individual p eigenvalues, is a random number, is the threshold function, x is the value of an individual on a specific dimension.
[0013] Preferably, the termination condition is that the maximum number of iterations or the current fitness value reaches a set threshold.
[0014] In a second aspect, the present invention provides a feature selection device, comprising: A construction module is used to take a feature subset in the high-dimensional data set to be processed as an individual and construct an initial population through multiple individuals; An acquisition module is used to obtain the fitness value of each individual in the initial population; An updating module is used to update the positions of multiple individuals in the initial population through an improved exponential distribution optimizer to obtain a new population; wherein, horizontal crossover, vertical crossover and adaptive differential evolution mechanism are added to the exponential distribution optimizer to obtain an improved exponential distribution optimizer; the horizontal crossover and vertical crossover are to update the individual positions through the mutual crossover results of the individual characteristic values in different dimensions, and the adaptive differential evolution mechanism is to dynamically adjust the scaling factor through the individual fitness value, and update the individual position through the scaling factor; The iteration module is used to obtain the fitness values of multiple individuals in the new population and determine whether they meet the termination conditions. If not, the new population is iterated again through the improved exponential distribution optimizer and the fitness value is determined; if it is satisfied, the individual with the highest fitness value is selected from the current population as the optimal solution, that is, the optimal feature subset.
[0015] In a third aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned feature selection method when executing the program.
[0016] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program implements the above-mentioned feature selection method when executed by a processor.
[0017] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a feature selection method, which obtains an improved exponential distribution optimizer by adding horizontal crossover, vertical crossover and adaptive differential evolution mechanisms to the exponential distribution optimizer. The present invention updates the positions of multiple individuals in the population through multiple update mechanisms, and can explore more potential optimal solutions. Furthermore, horizontal crossover and vertical crossover are to update the individual positions through the mutual crossover results of the individual's characteristic values in different dimensions, and the adaptive differential evolution mechanism is to dynamically adjust the scaling factor through the individual's fitness value, and update the individual position through the scaling factor. The individual positions updated by different update mechanisms are all different. The different update mechanisms used in the present invention cooperate with each other, which can help the algorithm jump out of the local optimal solution, increase the ability to fully explore the entire search space, increase the ability to cope with complex challenges, achieve better feature selection effects, and improve the classification accuracy of feature selection. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0019] Figure 1 The present invention is a flowchart of a feature selection method. DETAILED DESCRIPTION
[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0021] In order to solve the shortcomings of the existing technology, refer to Figure 1 The present invention provides a feature selection method, comprising the following steps: Step 1: Get the data from the dataset and normalize it to interval.
[0022] Step 2: Divide the dataset into multiple training sets and test sets.
[0023] Step 3: Randomly initialize the population, each individual represents a feature subset, and each feature subset contains at least one feature. Update the feature subset through the conversion function to obtain a binary population.
[0024] Step 4: Use the KNN (K-Nearest Neighbor) classifier to calculate the fitness value of each feature subset, as shown in Formula 1: (1); In the formula, is the fitness value, Represents the classification accuracy weight, ranging from [0,1]; represents the classification error rate; The weight representing the number of selected features; represents the length of the selected feature subset; and N Represents the total number of features.
[0025] Step 5: Determine whether the termination condition of the algorithm is met (for example, the maximum number of iterations or the fitness reaches a certain set threshold). If the termination condition is met, jump to step 8, otherwise continue to the next step.
[0026] Step 6: Update the position information of each individual through the enhanced exponential distribution optimizer method that combines horizontal and vertical crossover mechanisms with adaptive differential evolution mechanisms.
[0027] EDO focuses on guiding individuals to achieve global optimality, which is divided into two stages: development stage and exploration stage. The two stages are executed alternately according to a certain probability. is defined as the average of the top three best solutions , calculated as follows: (2); In the formula, is the first best solution, is the second best solution, is the third best solution.
[0028] The formula for updating individuals in the development phase is as follows: (3); here, Representing new individuals, It is i The memory matrix of each individual, is a random number in the range [0, 1], is a memoryless vector, It is i In addition, and is a parameter, and the formula is as follows: (4); (5); (6); here, is a random number between [-1, 1], is a random number in the range [0, 1].
[0029] In the exploration phase, the goal is to identify regions in the search space that are likely to contain global optimal solutions. This phase is constructed by selecting two optimal solutions from the original population following an exponential distribution, as shown below: (7); In the formula, It is a population individual, , and is an adaptive parameter.
[0030] Here, M is the population mean, calculated as: (8); Here, N represents the scale. The formulas for other variables are as follows: (9); (10); (11); (12); (13); (14); here, Indicates the current iteration number, represents the maximum number of iterations, and represents two random individuals, , and is a process variable.
[0031] In horizontal crossover, it is assumed that the parent individuals and In the n The formula for generating offspring by horizontally crossing dimensions is as follows: (15); (16); In the formula, and are two new individuals obtained by horizontal crossover, and is a random variable, and For the parent individual in n The eigenvalues of the dimensions, and is a random variable. Here, the random variable and is uniformly distributed in the interval [0,1]. In addition, the random variable and It is also uniformly distributed in the interval [-1,1].
[0032] In vertical crossover, assuming that i No. m Dimensions and n Perform a vertical crossover operation on the dimension, and the formula for generating offspring is as follows: (17); here, and represents a random number in the interval [0,1], Indicates through individual i No. m Dimensions and n The new individuals of the offspring generated by the vertical crossover operation of the dimension, p is the crossing rate of vertical crossings.
[0033] Through the adaptive differential evolution mechanism, the fitness value dynamically adjusts the scaling factor based on differential evolution. k , optimize the feature subset selection process, the formula is as follows: (18); here, and Represents the fitness values of two individuals.
[0034] (19); In the formula, To obtain new individuals through adaptive differential evolution mechanism, is the parent individual to be mutated, and are two random parent individuals.
[0035] Each individual in the initial population is updated in turn through the exponential distribution optimizer, horizontal crossover, vertical crossover and adaptive differential evolution mechanism to obtain the first new individual, the second new individual, the third new individual and the fourth new individual corresponding to each individual.
[0036] Calculate the fitness values of the first new individual, the second new individual, the third new individual and the fourth new individual, and select the best individual according to the fitness values.
[0037] Replace each individual in the initial population with the corresponding optimal individual to obtain a new population.
[0038] Step 7: Update the feature subset through the transformation function to further optimize the selection of the subset. Use the KNN classifier to calculate the fitness value of the updated feature subset. The transformation function formula is as follows: (20); (twenty one).
[0039] Since feature selection can be regarded as a discrete combinatorial optimization problem, the selection of each feature can be represented by "0" or "1" respectively, and each feature value of each individual in the population is converted into a binary coding value. In the formula, Indicates t The first i The individual p eigenvalues, is a random number in [0,1], Is to input x A threshold function that maps to between 0 and 1, x is the value of an individual on a specific dimension.
[0040] Step 8: When the termination condition is reached, the individual with the highest fitness value is selected from the population as the optimal solution, that is, the optimal feature subset.
[0041] K-nearest neighbor classifier (KNN) is a widely used classification method, and many studies have highlighted its advantages over other techniques. The basic concept of KNN is to classify a given individual according to the categories of its K nearest neighbors through majority voting. To determine the distance between individuals, the commonly used Euclidean distance is used, which is calculated as follows: (twenty two); here, represents the distance between individuals and, and are the coordinates of the individual in the kth dimension, It is the number of dimensions of the individual.
[0042] Ten-fold cross validation is an effective model evaluation method, especially when using the K-nearest neighbor (KNN) classifier. It randomly divides the data set into ten parts, uses nine parts for training each time, and uses the remaining part for validation, repeats ten times and takes the average, thereby reducing the risk of overfitting and improving model stability. This method can evaluate the consistency of the model on different data segmentations, optimize the selection of K values, and provide a more comprehensive performance evaluation to avoid deviations caused by improper data segmentation. Combined with ten-fold cross validation, the performance evaluation of the KNN classifier will be more reliable, thereby enhancing its generalization ability in practical applications.
[0043] Based on the same concept, the present invention also provides a feature selection device, including a construction module, an acquisition module, an update module and an iteration module.
[0044] The construction module is used to take a feature subset in the high-dimensional data set to be processed as an individual and construct an initial population through multiple individuals.
[0045] The acquisition module is used to obtain the fitness value of each individual in the initial population.
[0046] The update module is used to update the positions of multiple individuals in the initial population through an improved exponential distribution optimizer to obtain a new population; wherein, horizontal crossover, vertical crossover and adaptive differential evolution mechanisms are added to the exponential distribution optimizer to obtain an improved exponential distribution optimizer; horizontal crossover and vertical crossover are for updating the individual positions through the mutual crossover results of the individual characteristic values in different dimensions, and the adaptive differential evolution mechanism is for dynamically adjusting the scaling factor through the individual fitness value, and updating the individual position through the scaling factor.
[0047] The iteration module is used to obtain the fitness values of multiple individuals in the new population and determine whether they meet the termination conditions. If not, the new population is iterated again through the improved exponential distribution optimizer and the fitness value is determined; if it is satisfied, the individual with the highest fitness value is selected from the current population as the optimal solution, that is, the optimal feature subset.
[0048] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the above-mentioned feature selection method is implemented when the processor executes the program.
[0049] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned feature selection method is implemented.
[0050] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0051] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A feature selection method, characterized in that: The following steps are involved: Take the feature subset in the high-dimensional data set to be processed as individuals, and construct the initial population through multiple individuals; Get the fitness value of each individual in the initial population; The positions of multiple individuals in the initial population are updated by an improved exponential distribution optimizer to obtain a new population; wherein, horizontal crossover, vertical crossover and adaptive differential evolution mechanism are added to the exponential distribution optimizer to obtain an improved exponential distribution optimizer; the horizontal crossover and vertical crossover are for updating the individual positions through the mutual crossover results of the individual characteristic values in different dimensions, and the adaptive differential evolution mechanism is for dynamically adjusting the scaling factor through the individual fitness value, and updating the individual position through the scaling factor; Obtain the fitness values of multiple individuals in the new population and determine whether they meet the termination conditions. If not, iterate the new population again through the improved exponential distribution optimizer and determine the fitness value. If it meets the conditions, select the individual with the highest fitness value from the current population as the optimal solution, that is, the optimal feature subset.
2. A feature selection method according to claim 1, characterized in that: The fitness value of each individual in the initial population is obtained as follows: ； In the formula, is the fitness value, is the classification accuracy weight, is the classification error rate, R is the length of the selected feature subset, N is the total number of features, is the weight of the number of selected features.
3. A feature selection method according to claim 1, characterized in that: The method of updating the positions of multiple individuals in the initial population by using an improved exponential distribution optimizer to obtain a new population includes the following steps: Each individual in the initial population is updated in turn through the exponential distribution optimizer, horizontal crossover, vertical crossover and adaptive differential evolution mechanism to obtain the first new individual, the second new individual, the third new individual and the fourth new individual corresponding to each individual; The best individual among the first new individual, the second new individual, the third new individual and the fourth new individual corresponding to each individual is obtained based on the fitness value; Replace each individual in the initial population with the corresponding optimal individual to obtain a new population.
4. A feature selection method according to claim 1, characterized in that: The horizontal crossover and vertical crossover are to update the individual position through the mutual crossover results of the individual characteristic values in different dimensions, specifically including: ； ； In the formula, and are two new individuals obtained by horizontal crossover, and is a random variable, and For the parent individual and In the n The eigenvalues of the dimensions, and is a random variable; ； In the formula, is the new individual obtained by vertical crossover, and is a random number, is the crossover rate of vertical crossover.
5. A feature selection method according to claim 1, characterized in that: The adaptive differential evolution mechanism dynamically adjusts the scaling factor through the fitness value of the individual and updates the individual position through the scaling factor, specifically including: ； In the formula, and is the fitness value of the two individuals, k is the scaling factor; ； In the formula, To obtain new individuals through adaptive differential evolution mechanism, is the parent individual to be mutated, and are two random parent individuals.
6. A feature selection method according to claim 1, characterized in that: Before obtaining the fitness value of each individual in the initial population and obtaining the fitness values of multiple individuals in the new population, it is necessary to perform binary coding on the multiple individuals in the initial population and the new population, which specifically includes: ； ； In the formula, Indicates t The first i The individual p eigenvalues, is a random number, is the threshold function, x is the value of an individual on a specific dimension.
7. A feature selection method according to claim 1, characterized in that: The termination condition is that the maximum number of iterations or the current fitness value reaches a set threshold.
8. A feature selection device, characterized in that: include: A construction module is used to take a feature subset in the high-dimensional data set to be processed as an individual and construct an initial population through multiple individuals; An acquisition module is used to obtain the fitness value of each individual in the initial population; An updating module is used to update the positions of multiple individuals in the initial population through an improved exponential distribution optimizer to obtain a new population; wherein, horizontal crossover, vertical crossover and adaptive differential evolution mechanism are added to the exponential distribution optimizer to obtain an improved exponential distribution optimizer; the horizontal crossover and vertical crossover are to update the individual positions through the mutual crossover results of the individual characteristic values in different dimensions, and the adaptive differential evolution mechanism is to dynamically adjust the scaling factor through the individual fitness value, and update the individual position through the scaling factor; The iteration module is used to obtain the fitness values of multiple individuals in the new population and determine whether they meet the termination conditions. If not, the new population is iterated again through the improved exponential distribution optimizer and the fitness value is determined; if it is satisfied, the individual with the highest fitness value is selected from the current population as the optimal solution, that is, the optimal feature subset.
9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the feature selection method described in any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the feature selection method according to any one of claims 1 to 7 is implemented.