Self-adaptive multi-objective evolution method for solving high-dimensional feature selection optimization problem

Abstract

The invention provides a self-adaptive multi-objective evolution method for solving a high-dimensional feature selection optimization problem, relates to the technical field of feature selection, and aims to improve the efficiency and accuracy of feature selection. The method comprises two key technologies of population adaptive initialization AI and offspring adaptive breeding AR, the population adaptive initialization generates an initial population with better diversity and convergence through an interpolation method, and the offspring adaptive breeding dynamically adjusts the variation rate according to the occurrence frequency of features so as to increase the diversity of offsprings and the quality of solutions. The combination of the two technologies not only improves the search capability of the adaptive multi-objective evolution method in the aspect of searching the globally optimal solution, but also contributes to balancing population diversity and algorithm convergence performance in the evolution process.

Description

Technical Field

[0001] The present invention relates to the technical field of feature selection, and in particular to an adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems. Background Art

[0002] In the era of big data, data preprocessing has become an important part of machine learning and pattern recognition. As one of the key steps in data preprocessing, feature selection aims to select the most helpful subset for classification tasks from the original feature set to improve the accuracy and efficiency of the classification model. However, as the data dimension increases, the feature selection problem becomes more and more complicated, especially in high-dimensional data sets, where the difficulty of feature selection increases significantly.

[0003] Currently, traditional feature selection methods are mainly divided into two categories: wrapper-based methods and filter-based methods. Among them, wrapper-based methods rely on classification models to evaluate the quality of feature subsets, which are usually more accurate, but have higher computational costs. Filter-based methods do not rely on classifiers, but evaluate the importance of features by analyzing the data itself, which have lower computational costs but relatively poor accuracy.

[0004] In order to overcome the limitations of these traditional methods, multi-objective evolutionary algorithms (MOEAs) have been introduced into the field of feature selection. These algorithms optimize two objectives simultaneously by simulating the process of natural selection: minimizing the number of selected features and minimizing the classification error rate. However, existing MOEAs face the curse of dimensionality when dealing with high-dimensional feature selection problems, that is, as the number of features increases, the algorithm's search ability decreases significantly, and it is difficult to find the global optimal solution. Specifically, among the existing multi-objective evolutionary algorithms, such as non-dominated sorting-based algorithms (such as NSGA-II) and decomposition-based algorithms (such as MOEA / D), although they perform well on low-dimensional data sets, in high-dimensional feature selection tasks, due to the exponential growth of the search space, these algorithms are difficult to efficiently search for the global optimal solution, and are prone to fall into local optimal solutions, making it difficult to maintain the diversity of the population and balanced convergence.

[0005] In recent years, some scholars have proposed some improved MOEA frameworks, trying to improve the algorithm's search ability by improving the initialization and reproduction process; for example: a segmented initialization method and offspring modification method SIOM, an evolutionary algorithm based on repeated analysis DAEA, a problem reconstruction mechanism PRDH, an evolutionary algorithm based on a dual search mechanism BSEA, etc. Although these studies have improved the performance of MOEAs to a certain extent, they still have problems such as high algorithm complexity, difficulty in adapting to the characteristics of different data sets, and fixed parameter settings.

[0006] In view of this, this application is filed. Summary of the invention

[0007] The present invention provides an adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems, which can at least partially improve the above problems.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] An adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems, comprising:

[0010] Obtaining a preset population size N, a total number of features D, a total number of evolutionary iterations T, and a current number of iterations C, and assigning the current number of iterations C to 0;

[0011] The NewSubPop algorithm is used to perform adaptive initialization AI processing on the population size N, the total number of features D and the interpolation position Ip to obtain the initial population Pop, wherein the adaptive initialization processing includes NewSubPop preprocessing, generating the first initialization sub-population Pop1 and the second initialization sub-population Pop2, and merging and screening the first initialization sub-population Pop1 and the second initialization sub-population Pop2;

[0012] When it is determined that the current number of iterations C is less than the total number of evolutionary iterations T, the initial population Pop is subjected to adaptive reproduction AR preprocessing to generate a set of offspring solutions Pop*, wherein the adaptive reproduction AR preprocessing includes frequency feature calculation processing, crossover processing and adaptive mutation processing;

[0013] The initial population Pop and the offspring solution set Pop* are merged to obtain a joint population Uop, and the joint population Uop is deduplicated to remove duplicate solutions in the joint population Uop;

[0014] A non-dominated sorting and crowding distance algorithm is used to randomly select N solutions from the joint population Uop to replace the solutions in the initial population Pop, to obtain an updated initial population Pop, and the current number of iterations C is cumulatively added by 1, where N is a preset natural number;

[0015] Repeat the above adaptive reproduction AR preprocessing, merging processing, and replacement steps until it is determined that the current number of iterations C is greater than or equal to the total number of evolutionary iterations T, and obtain the final evolved population Pop.

[0016] In summary, the adaptive multi-objective evolutionary method for solving the high-dimensional feature selection optimization problem is specifically used for the evolutionary optimization problem of two-objective high-dimensional feature selection. It designs two key technologies, population adaptive initialization AI and offspring adaptive reproduction AR, to improve the algorithm's search ability in high-dimensional feature space, and further enhance and balance the diversity and convergence of the population.

[0017] Among them, the population adaptive initialization AI generates an initial population with better diversity and convergence through interpolation methods; analyzes the distribution of the current population in the target space, and adaptively adjusts the initialization parameters to cover a wide range of search spaces. The offspring adaptive reproduction AR dynamically adjusts the mutation rate during the offspring reproduction process according to the frequency of occurrence of features to increase the diversity of offspring and the quality of offspring solutions, so as to avoid premature convergence and easy to fall into local optimality. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of an adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems provided by the first aspect of the present invention;

[0019] Figure 2 It is a flow chart of an adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems provided by the second aspect of the present invention;

[0020] Figure 3 It is a schematic flow chart of an adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems provided by the third aspect of the present invention;

[0021] Figure 4 It is a schematic diagram of the process of population adaptive initialization AI provided by an embodiment of the present invention;

[0022] Figure 5 is a schematic diagram of the process of offspring adaptive reproduction AR provided by an embodiment of the present invention;

[0023] Figure 6 It is a brief example diagram of a population adaptive initialization method AI in an adaptive multi-objective evolutionary method for solving a high-dimensional feature selection optimization problem when the population size is set to 8, provided in an embodiment of the present invention;

[0024] Figure 7 This is a brief example diagram of the offspring adaptive breeding method AR in the adaptive multi-objective evolutionary method for solving the high-dimensional feature selection optimization problem when the total number of features is set to 9, provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0026] refer to Figure 1 , Figure 2 As shown, the first embodiment of the present invention discloses an adaptive multi-objective evolution method for solving a high-dimensional feature selection optimization problem, which can be executed by an adaptive multi-objective evolution device for solving a high-dimensional feature selection optimization problem (hereinafter referred to as an evolution device), in particular, by one or more processors in the evolution device to implement the following method:

[0027] S1, obtaining a preset population size N, a total number of features D, a total number of evolutionary iterations T, and a current number of iterations C, and assigning the current number of iterations C to 0;

[0028] In this embodiment, the population size N and the total number of features D are input as initial parameters.

[0029] S2, using the NewSubPop algorithm to perform adaptive initialization AI processing on the population size N, the total number of features D and the interpolation position Ip to obtain the initial population Pop, wherein the adaptive initialization processing includes NewSubPop preprocessing, generating the first initialization sub-population Pop1 and the second initialization sub-population Pop2, and merging and screening the first initialization sub-population Pop1 and the second initialization sub-population Pop2;

[0030] Specifically, step S2 includes: obtaining the interpolation position Ip, using the NewSubPop algorithm to preprocess the population size N, the total feature number D and the interpolation position Ip, and generating the first initialized subpopulation Pop1, Pop1 = NewSubPop (K, D, Ip), where K is the subpopulation size, K = N, and Ip = 0.5;

[0031] Obtain the number t1 of target values ​​f2 in the first initialized subpopulation Pop1, and calculate the repetition rate R according to the number t1 and the population size N, where R=1-t1 / N;

[0032] Calculate the size k of the second subpopulation according to the repetition rate R, k = [N*R], and select the minimum value t2 between the population size N and the total number of features D, t2 = min(N, D) as the interpolation position of the second subpopulation;

[0033] The size k, the total number of features D and the minimum value t2 of the second subpopulation are preprocessed using the NewSubPop algorithm to generate a second initialized subpopulation Pop2, Pop1 = NewSubPop (k, D, Ip*t2 / D);

[0034] The first initialized subpopulation Pop1 and the second initialized subpopulation Pop2 are merged, and N subpopulations are randomly selected from the merged subpopulations using a non-dominated sorting and crowding distance algorithm as the final initial population Pop.

[0035] Preferably, the NewSubPop algorithm is used to preprocess the population size N, the total number of features D and the interpolation position Ip to generate the first initialized subpopulation Pop1, specifically:

[0036] A zero matrix SubPop of size K×D is created according to the subpopulation size K and the total number of features D, where SubPop=Zeros(N,D), and a random probability ρ1 is obtained according to the zero matrix SubPop, that is, the ρ1 value generated each time is a random decimal between 0 and 1;

[0037] The random probability ρ1 and the interpolation position Ip are compared and processed. When it is determined that the random probability ρ1 is less than the interpolation position Ip, the feature corresponding to the random probability ρ1 is selected to obtain the first initialized subpopulation Pop1, and the zero matrix SubPop(i,j) corresponding to the feature is set to 1, where i is the individual and j is the feature.

[0038] In this embodiment, the process of adaptively initializing AI is as follows: Figure 2As shown in the left part of . First, by calling the NewSubPop method, the first initialized subpopulation Pop1 is initialized to a sample subpopulation located in the middle of the target space. The input parameters of the NewSubPop method include a key parameter value Ip=0.5, which serves as the interpolation position for randomly generating new solutions in the target space. Specifically, in the NewSubPop method, the input interpolation position is compared with a random probability to determine whether to select the current feature. Therefore, the 0.5 interpolation position input value means randomly creating a new solution near f1=0.5, which is roughly located in the middle of the target space. The repetition rate is then calculated, which adaptively determines the number of new solutions that need to be initialized and located in front of the target space. Basically, the more repeated f2 values ​​there are, the larger the second initialized subpopulation Pop2 initialized by the NewSubPop method. Therefore, the input interpolation position of the second initialized subpopulation Pop2 is set to a value much smaller than 0.5, and is closely related to the comparison between the population size N and the total number of features D, aiming to randomly create solutions with a smaller selected feature subset size. Finally, N best solutions are selected from the joint population of the previously initialized first subpopulation Pop1 and the second initialized subpopulation Pop2 by non-dominated sorting and crowding distance methods.

[0039] See also Figure 4 Specifically, in this embodiment, it is necessary to generate the first initialized subpopulation Pop1; call the NewSubPop method to generate the first initialized subpopulation Pop1 located in the middle of the target space, Pop1 = NewSubPop (K, D, 0.5), K = N. When generating subpopulations, the NewSubPop method controls whether each feature is selected according to the given interpolation position Ip. The diversity of the population is guaranteed by randomizing the feature selection of each individual. Flexible interpolation position control enables the algorithm to generate subpopulations with specific feature ratios, thereby improving the diversity and exploration ability of solutions. It should be noted that in the process of generating the first initialized subpopulation Pop1, a loop is used to traverse each individual i and each feature j. And compare it with Ip according to the random probability ρ, Ip = 0.5, if ρ < 0.5, select the feature and set the corresponding SubPop (i, j) to 1.

[0040] Then, the number t1, repetition rate R, the size k of the second subpopulation, and the minimum value t2 are calculated in turn to obtain the second initialized subpopulation Pop2. Among them, the number t1 is used to measure the diversity of the first initialized subpopulation Pop1. By counting the number of unique target values, the diversity of the first initialized subpopulation Pop1 can be further evaluated, and the basis for the calculation of the repetition rate in the subsequent steps is provided. The repetition rate R is used to measure the degree of repetition of the solution in the current population; the calculation of the repetition rate can help identify the proportion of repeated solutions in the population, and adaptively adjust the population size and interpolation parameters to avoid premature convergence of the population. If the repetition rate is high, it means that there are many repeated solutions in the initial population. At this time, a larger second initialized subpopulation Pop2 needs to be generated to increase diversity. On the contrary, if the diversity of the initial population is good, a smaller second initialized subpopulation Pop2 can be generated.

[0041] In this embodiment, the minimum value t2 before the population size N and the total number of features D is screened out and used as one of the control parameters in the sub-population generation process. Among them, the minimum value t2 controls the interpolation parameters when the second sub-population is generated to further adjust the feature selection probability of the population, so that the generated solution has richer diversity. The second initialized sub-population Pop2 is compared with the first initialized sub-population Pop1. The interpolation parameter of the second initialized sub-population Pop2 is much less than 0.5, so the probability of the feature being selected is lower. The generated second initialized sub-population Pop2 will focus on fewer feature selections, avoid the population solution being too concentrated in a specific area, and help explore the edge solutions of the target space.

[0042] Finally, N optimal solutions are selected as the final initial population Pop through non-dominated sorting and crowding distance. The non-dominated sorting method ensures the Pareto frontier solution, and the crowding distance ensures the diversity of solutions. The final generated population can evenly cover the target space with high quality.

[0043] See also Figure 6 This embodiment provides a simple example to briefly explain the working principle of population adaptive initialization AI. In this example, the population size is set to N = 8. First, a subpopulation with a size of 8 solutions is initialized near the interpolation position f1 = 0.5 (i.e., the middle position of the target space). Here, the first initialized subpopulation Pop1 acts as a normal subpopulation for sampling, and its solution will be analyzed in the f2 direction to calculate the repetition rate R. Figure 2As shown, in this case R = 0.5. Then, a new subpopulation with a size of 4 solutions (8*0.5) is initialized, namely the second initialized subpopulation Pop2, but located near the interpolation position f1 = N / D*0.5. Assuming that N is much smaller than D, this situation is quite common when processing high-dimensional data; so that the interpolation position of the second initialized subpopulation Pop2 is distributed in front of the target space. Here, the second initialized subpopulation Pop2 acts as a forward subpopulation to explore more potential frontier areas in the target space, where the size of the selected feature subsets is relatively small. In this way, the mixed initial population will be composed of the above-mentioned normal subpopulation, namely the first initialized subpopulation Pop1, and the forward subpopulation, namely the second initialized subpopulation Pop2, and the proportion of the second initialized subpopulation Pop2 is adaptively set based on the analysis of the repeatability of the solution in the f2 target direction. The above adaptive processing method for the size of the vanguard subpopulation is very reasonable, because sparser and higher-dimensional feature spaces usually lead to more repeated f2 values, especially when the number of classification samples is seriously insufficient compared to the classification of high-dimensional data itself. This makes it more necessary for us to use a certain size or larger feature subset to explore the frontier area in the target space, that is, to expand the scale of use of the vanguard subpopulation.

[0044] S3, when it is determined that the current number of iterations C is less than the total number of evolutionary iterations T, the initial population Pop is subjected to adaptive reproduction AR preprocessing to generate a set of offspring solutions Pop*, wherein the adaptive reproduction AR preprocessing includes frequency feature calculation processing, crossover processing and adaptive mutation processing;

[0045] Specifically, step S3 includes: performing frequency feature calculation processing on the initial population Pop to obtain a feature mutation rate vector MR;

[0046] Based on the characteristic mutation rate vector MR, crossover processing and adaptive mutation processing are performed to generate a set of offspring solutions Pop*.

[0047] Preferably, the frequency characteristic calculation process is performed on the initial population Pop to obtain the characteristic mutation rate vector MR, which is specifically:

[0048] Calculate the frequency vector FR composed of the occurrence frequency of each feature in the initial population Pop;

[0049] Initializing the frequency vector FR, setting the element value of the characteristic frequency value of 0 in FR to 1;

[0050] According to the initialized frequency vector FR, the feature mutation rate vector MR is calculated, MR = FR / D.

[0051] Preferably, based on the characteristic mutation rate vector MR, crossover processing and adaptive mutation processing are performed to generate a set of offspring solutions Pop*, specifically:

[0052] Randomly select N pairs of parent individuals from the initial population Pop as the basis for crossover processing and mutation processing;

[0053] For each of the N pairs of parents, such as P1 and P2, a crossover process is performed to exchange the features of the two parents to obtain a new offspring solution, and a one-dimensional random probability vector ρ2 with a length of D is dynamically and randomly generated for P1 and P2;

[0054] The offspring obtained by the crossover process are then subjected to adaptive mutation processing. The one-dimensional random probability vector ρ2 is compared with the element value of the corresponding position of the feature mutation rate vector MR to determine whether to mutate the feature at that position, and the feature value of the selected position is subjected to inverse mutation processing;

[0055] The N offspring individuals corresponding to each pair of the N pairs of parents generated after the crossover and mutation operations are stored in Pop* to obtain an offspring solution set Pop*.

[0056] In this embodiment, the adaptive breeding AR process is as follows Figure 2 As shown in the right part of , this process works well with the adaptive initialization introduced earlier and can better balance population diversity and algorithm convergence. First, the feature mutation rate vector MR is calculated based on the frequency of occurrence of each feature in all solutions of the initial population Pop. The mutation rate of each feature is then used in the adaptive mutation operation to decide whether to mutate the feature. Secondly, the parent solution is randomly selected from the current population, and each solution has an equal chance of mating. But it is worth noting that the first parent Parent1 in each pair of parents will continue to be used as the basis for the offspring. An effective crossover operation is performed by selecting different feature values ​​to be exchanged between the two parents. An adaptive mutation operation is performed by providing more mutation opportunities for features that appear more frequently in the current population. Finally, the first parent solution that undergoes crossover and mutation is copied as the expected new offspring.

[0057] See also Figure 5Specifically, in this embodiment, the frequency of occurrence of each feature in the population is calculated to analyze the prevalence of the feature in all individuals; frequently occurring features are more likely to belong to local optimal solutions, so these features will have a higher chance of mutation to increase the diversity of the population and explore new solution spaces. In addition, the value of the feature frequency of zero is set to 1 to avoid the error of the denominator being zero in subsequent division operations and to ensure that each feature has a certain chance of mutation, even if it does not appear in the current population. Finally, the frequency of occurrence of each feature is divided by the total number of features to calculate the mutation rate of the feature. The feature mutation rate is used to control the probability of mutation of the feature, and the higher the frequency, the greater the probability of mutation of the feature.

[0058] In this embodiment, N pairs of parent individuals are randomly selected as the basis for crossover processing and mutation processing to increase the diversity of the population and prevent the algorithm from converging to the local optimal solution too early; then a random probability vector ρ2 of length D is generated to control the mutation of the feature. By introducing randomness, it is controlled whether each feature will mutate, so that the mutation operation has a certain degree of randomness.

[0059] Among them, the crossover operation generates offspring from the parent generation; specifically, the crossover operation generates new solutions by exchanging the features of the two parent generations, maintaining the diversity of the population, and may introduce new excellent feature combinations. The mutation operation helps the population escape from the local optimal solution and explore new solution spaces. The mutation rate is proportional to the frequency of occurrence of the feature. The more frequently a feature appears, the higher the probability of mutation, thereby adaptively increasing the frequency and diversity of useful features in the population.

[0060] Finally, the parent generation after crossover and mutation is added as a new offspring individual to the newly generated sub-population, that is, the offspring solution set Pop*. Generating new offspring through crossover and mutation operations helps the population maintain diversity and improve the quality of existing solutions in the population.

[0061] See also Figure 7 This embodiment provides a simple example to briefly explain the working principle of offspring adaptive reproduction AR. In this example, the total number of features is set to D = 9. Figure 7 As shown in , each solution is stored in a decision vector with 9 features, where the value of the element, i.e., the gene position, is 1, indicating that the feature with the sequence number is selected, and 0, indicating that the feature with the sequence number is not selected. The core reproduction process consists of crossover and mutation operations, which are respectively Figure 7 As shown on the left and right sides of Figure 7 As shown in , the first parent solution will be used as the basis for the offspring, and the gene positions with the same index position in the two parents will be compared with each other, and the valid positions with different element values ​​will be randomly selected and exchanged, that is, randomly changed from 1 to 0 or from 0 to 1, thereby generating a new offspring solution vector. Figure 7In the figure, the gene loci randomly selected for crossover or mutation are marked in gray to facilitate identification. Figure 7 The crossover process in shows that only different eigenvalues ​​are exchanged in the crossover operation, so it is called effective crossover. Figure 7 The mutation process in follows the principle of adaptive mutation, that is, the features that appear more frequently in the current population will have a greater probability of mutating during the reproduction process, thereby increasing the diversity of the population and the feature quality of the solution.

[0062] S4, merging the initial population Pop and the offspring solution set Pop* to obtain a joint population Uop, and performing deduplication processing on the joint population Uop to remove duplicate solutions in the joint population Uop;

[0063] Preferably, the joint population Uop=Pop∪Pop*.

[0064] Preferably, the condition for the end of the deduplication process is: there are no duplicate solutions in the joint population Uop or there are only N solutions left in the joint population Uop.

[0065] S5, randomly selecting N solutions from the joint population Uop by using a non-dominated sorting and crowding distance algorithm to replace the solutions in the initial population Pop, obtaining an updated initial population Pop, and cumulatively adding 1 to the current number of iterations C, where N is a preset natural number (i.e., the population size or the number of solutions contained in the population);

[0066] S6, repeat the above-mentioned adaptive reproduction AR preprocessing, merging processing, and replacement steps until it is determined that the current number of iterations C is greater than or equal to the total number of evolutionary iterations T, and the final evolved population Pop is obtained.

[0067] Specifically, in this embodiment, from Figure 3 It can be seen that the overall framework of the adaptive multi-objective evolutionary method for solving the high-dimensional feature selection optimization problem is similar to other traditional multi-objective evolutionary algorithms based on non-dominated sorting, but it adopts specially designed initialization and breeding methods, namely adaptive initialization AI and adaptive breeding AR. In addition, the environmental selection process has also been slightly modified, adding a step to remove duplicate solutions in the current joint population. Despite this, the method still adopts the widely used truncation method based on non-dominated sorting and crowding distance, which first appeared in the classic dominance-based algorithm NSGA-II and was used to select and retain the best N solutions in the current joint population. The termination criterion of the adaptive multi-objective evolutionary method for solving the high-dimensional feature selection optimization problem is based on the total number of calculated evaluation objective functions, which is usually equal to the total number of solutions generated during the evolution process.

[0068] Specifically, in this embodiment, five of the most advanced and classic multi-objective evolutionary algorithms are used as comparison algorithms with this method for experimental analysis; these five algorithms are BSEA (a dual search evolutionary algorithm), PRDH (a multi-objective evolutionary algorithm based on problem reconstruction and repeated processing), PMEA (an evolutionary algorithm based on polar coordinates), MOEA / D (a multi-objective evolutionary algorithm based on decomposition) and NSGA-II (a genetic algorithm based on non-dominated sorting).

[0069] Among them, BSEA and PRDH are the latest multi-objective evolutionary algorithms, which are specially used to solve the two-objective feature selection in classification. BSEA is based on a dual-search multi-task mechanism, and PRDH is mainly based on adaptive repeated processing of solutions. PMEA is a multi-objective evolutionary algorithm based on polar coordinates published in recent years. It can well balance the population diversity and convergence in the evolutionary process. NSGA-II and MOEA / D are among the most classic and most common multi-objective evolutionary algorithms. NSGA-II is based on the dominance principle and guides population evolution through non-dominated sorting and crowding distance, while MOEA / D is based on the decomposition method, decomposing the multi-objective optimization problem into multiple single-objective problems and using neighborhood information for collaborative optimization.

[0070] In this embodiment, a total of 20 public real classification data sets are used as test problems to test the optimization and classification performance of each two-objective feature selection algorithm. The attribute values ​​of each test data set are shown in Table 1, and the total number of features is arranged in ascending order. As can be seen from Table 1, the total number of features of each data set varies from 60 to 10509, covering a variety of features, including both relatively low-dimensional features and relatively high-dimensional features. In addition, the number of classification samples ranges from 50 to 2600, and the number of classification categories ranges from 2 to 26, which fully takes into account the comprehensiveness of the test cases.

[0071] Table 1 Attributes of each classification dataset used as test instances

[0072] Serial number Dataset Number of features Sample size Number of categories 1 Sonar 60 208 2 2 Libras 90 360 15 3 HillValley 100 606 2 4 Synthetic 100 200 5 5 MUSK1 166 476 2 6 Arrhythmia 278 452 13 7 LSVT 310 126 2 8 Madelon 500 2600 2 9 ISOLET1 617 1559 26 10 ISOLET2 617 1560 26 11 Yale 1024 165 15 12 PIE10P 2420 210 10 13 Leukemia1 5327 72 3 14 Tumor9 5726 60 9 15 TOX171 5748 171 4 16 ALLAML 7129 72 2 17 Carcinom 9182 174 11 18 Nci9 9712 60 9 19 Brain2 10367 50 5 20 Prostate 10509 102 2

[0073] Each algorithm uses the hypervolume (HV)43 index as the main indicator for evaluating optimization performance, and its global reference point is set to (1,1). Generally speaking, the larger the HV value, the better the optimization performance, taking into account both convergence and diversity. Table 2 gives the statistical data of the average performance of each algorithm on the test data set. For easy distinction, the algorithm with the best performance on each data set is marked with a gray background. It should be noted that the statistical data uses the Wilcoxon test with a significance level of 5% to distinguish performance differences, and the algorithms with insignificant performance differences are prefixed with the symbol ★.

[0074] First, as can be seen from Table 2, the performance of this method is better than that of the comparison algorithm on all test datasets, and the performance advantage of this method is obvious in most test instances. Since the HV metric considers both population diversity and algorithm convergence, the leading position of this method in this metric shows its advantages in diversity and convergence. In addition, the excellent performance of this method on various classification datasets also shows the effectiveness of its versatility and search ability in solving two-objective feature selection, not only for relatively low-dimensional datasets, but also for relatively high-dimensional datasets that are more challenging for traditional MOEA. In fact, compared with those MOEAs that are not specifically designed to solve the two-objective feature selection problem (i.e., PMEA, MOEA / D, and NSGA-II), the advantage of this method on high-dimensional datasets is more obvious, because feasible solutions will become more sparse and difficult to find in high-dimensional feature space. Overall, compared with all other algorithms, this method has the best optimization search performance overall.

[0075] Table 2 Average HV performance of each algorithm on each dataset (the best algorithm is marked in gray (the larger the better), and insignificant differences are prefixed with ★)

[0076]

[0077]

[0078] The minimum classification error (MCE) and the number of selected features (SFN) are used as complementary indicators to the HV indicator proposed above to provide additional analysis of the classification performance obtained by each algorithm. More specifically, MCE is the minimum classification error among all solutions in the final population, indicating the best classification accuracy obtained, while NSF is the number of selected features corresponding to the MCE value, usually rounded up. In general, a smaller MCE value represents a better classification performance, while a smaller NSF value usually leads to a more efficient classification process and relatively less computational cost. Here, Table 3 provides the MCE values ​​and corresponding NSF values ​​obtained by each algorithm on the test dataset, with the average performance statistics shown below each row of MCE, and the best MCE performance on each dataset is marked with a gray shading to distinguish. It should be noted that the statistics use the Wilcoxon test with a significance level of 5% to identify the MCE performance differences, and those with insignificant differences are prefixed with the symbol ★.

[0079] As can be seen from Table 3, on most of the tested datasets (18 out of 20 datasets), the adaptive multi-objective evolutionary method for solving the high-dimensional feature selection optimization problem performs best in terms of the average MCE metric. The only two losses occur on the Libras and MUSK1 datasets, and the performance difference between AIR and the corresponding best algorithms, namely NSGA-II and PMEA, is actually not significant. In addition, the proposed method has a better NSF average value than NSGA-II and PMEA on these two datasets, which means that it has better classification efficiency when the size of the feature subset selected for classification is smaller. In relatively high-dimensional datasets, the advantages of the proposed method in terms of MCE and NSF indicators are more obvious. For example, on the Prostate dataset with the highest feature dimension, compared with the other three traditional MOEAs, namely PMEA, MOEA / D and NSGA-II, the proposed method, BSEA and PRDH all show a huge advantage in the mean of NSF, which is significantly reduced. The proposed method, BSEA and PRDH are all specially designed to solve two-objective feature selection, but the performance of the proposed method is generally better than BSEA and PRDH because of its smaller MCE value and comparable NSF value. Generally, the MCE value indicates the best classification accuracy obtained, while the NSF value usually affects the classification efficiency in computational cost. Therefore, this method usually has the best classification performance compared to all other algorithms.

[0080] Table 3 Average MCE and NSF performance of each algorithm on each dataset (the best performance is marked in gray (the smaller the better), and insignificant MCE differences are prefixed with ★)

[0081]

[0082] In summary, the adaptive multi-objective evolutionary method for solving the high-dimensional feature selection optimization problem uses the population adaptive initialization AI and offspring adaptive reproduction AR designed specifically for the two-objective feature selection optimization environment, so that the evolutionary feature selection algorithm of this method can achieve faster convergence speed while improving population diversity, thereby achieving a more efficient global optimal solution search capability. In addition, the Pareto optimal solution set searched can give decision makers more options, in which each solution vector selects fewer features, which usually makes the corresponding classification process more efficient and reduces the computing resource loss of the entire classification. And the global optimal solution set searched and the feature subsets represented by it can significantly reduce the classification error rate in the corresponding classification process and greatly improve the accuracy of the classification model. Finally, due to the use of an adaptive parameter setting method, this method can adaptively set the optimal initialization and reproduction parameters according to the actual optimization environment of the specific classification data set, thereby obtaining a more general and universal evolutionary feature selection method suitable for processing low-dimensional and high-dimensional classification data sets.

[0083] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. An adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems, characterized in that: include: Obtaining a preset population size N, a total number of features D, a total number of evolutionary iterations T, and a current number of iterations C, and assigning the current number of iterations C to 0; The NewSubPop algorithm is used to perform adaptive initialization AI processing on the population size N, the total number of features D and the interpolation position Ip to obtain the initial population Pop, wherein the adaptive initialization processing includes NewSubPop preprocessing, generating the first initialization sub-population Pop1 and the second initialization sub-population Pop2, and merging and screening the first initialization sub-population Pop1 and the second initialization sub-population Pop2; When it is determined that the current number of iterations C is less than the total number of evolutionary iterations T, the initial population Pop is subjected to adaptive reproduction AR preprocessing to generate a set of offspring solutions Pop*, wherein the adaptive reproduction AR preprocessing includes frequency feature calculation processing, crossover processing and adaptive mutation processing; The initial population Pop and the offspring solution set Pop* are merged to obtain a joint population Uop, and the joint population Uop is deduplicated to remove duplicate solutions in the joint population Uop; A non-dominated sorting and crowding distance algorithm is used to randomly select N solutions from the joint population Uop to replace the solutions in the initial population Pop, to obtain an updated initial population Pop, and the current number of iterations C is cumulatively added by 1, where N is a preset natural number; Repeat the above adaptive reproduction AR preprocessing, merging processing, and replacement steps until it is determined that the current number of iterations C is greater than or equal to the total number of evolutionary iterations T, and obtain the final evolved population Pop.

2. The adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems according to claim 1, characterized in that: The NewSubPop algorithm is used to perform adaptive initialization AI processing on the population size N, the total number of features D, and the interpolation position Ip to obtain the initial population Pop, specifically: Obtain the interpolation position Ip, use the NewSubPop algorithm to preprocess the population size N, the total number of features D and the interpolation position Ip, and generate the first initialized subpopulation Pop1, Pop1 = NewSubPop (K, D, Ip), where K is the subpopulation size, K = N, and Ip = 0.5; Obtain the number t1 of target values ​​f2 in the first initialized subpopulation Pop1, and calculate the repetition rate R according to the number t1 and the population size N, where R=1-t1 / N; Calculate the size k of the second subpopulation according to the repetition rate R, k = [N*R], and select the minimum value t2 between the population size N and the total number of features D, t2 = min(N, D) as the interpolation position of the second subpopulation; The size k, the total number of features D and the minimum value t2 of the second subpopulation are preprocessed using the NewSubPop algorithm to generate a second initialized subpopulation Pop2, Pop1 = NewSubPop (k, D, Ip*t2 / D); The first initialized subpopulation Pop1 and the second initialized subpopulation Pop2 are merged, and N subpopulations are randomly selected from the merged subpopulations using a non-dominated sorting and crowding distance algorithm as the final initial population Pop.

3. The adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems according to claim 2, characterized in that: The NewSubPop algorithm is used to preprocess the population size N, the total number of features D and the interpolation position Ip to generate the first initialized subpopulation Pop1, specifically: A zero matrix SubPop of size K×D is created according to the subpopulation size K and the total number of features D, SubPop=Zeros(N,D), and a random probability ρ1 is obtained according to the zero matrix SubPop; The random probability ρ1 and the interpolation position Ip are compared and processed. When it is determined that the random probability ρ1 is less than the interpolation position Ip, the feature corresponding to the random probability ρ1 is selected to obtain the first initialized subpopulation Pop1, and the zero matrix SubPop(i,j) corresponding to the feature is set to 1, where i is the individual and j is the feature.

4. The adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems according to claim 1, characterized in that: When it is determined that the current number of iterations C is less than the total number of evolutionary iterations T, the initial population Pop is subjected to adaptive reproduction AR preprocessing to generate a set of offspring solutions Pop*, specifically: Performing characteristic frequency calculation processing on the initial population Pop to obtain a characteristic mutation rate vector MR; Based on the characteristic mutation rate vector MR, crossover processing and adaptive mutation processing are performed to generate a set of offspring solutions Pop*.

5. The adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems according to claim 4, characterized in that: The characteristic frequency calculation process is performed on the initial population Pop to obtain the characteristic mutation rate vector MR, which is specifically: Calculate the frequency vector FR composed of the occurrence frequency of each feature in the initial population Pop; Initializing the frequency vector FR, setting the element value of the characteristic frequency value of 0 in FR to 1; According to the initialized frequency vector FR, the feature mutation rate vector MR is calculated, MR = FR / D.

6. The adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems according to claim 4, characterized in that: Based on the characteristic mutation rate vector MR, crossover processing and adaptive mutation processing are performed to generate a set of offspring solutions Pop*, specifically: Randomly select N pairs of parent individuals from the initial population Pop as the basis for crossover processing and mutation processing; For each of the N pairs of parents, such as P1 and P2, a crossover process is performed to exchange the features of the two parents to obtain a new offspring solution, and a one-dimensional random probability vector ρ2 with a length of D is dynamically and randomly generated for P1 and P2; The offspring obtained by the crossover process are then subjected to adaptive mutation processing. The one-dimensional random probability vector ρ2 is compared with the element value of the corresponding position of the feature mutation rate vector MR to determine whether to mutate the feature at that position, and the feature value of the selected position is subjected to inverse mutation processing; The N offspring individuals corresponding to each pair of the N pairs of parents generated after the crossover and mutation operations are stored in Pop* to obtain an offspring solution set Pop*.

7. The adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems according to claim 1, characterized in that: The joint population Uop=Pop∪Pop*.

8. The adaptive multi-objective evolutionary method for solving high-dimensional feature selection optimization problems according to claim 1, characterized in that: The condition for the deduplication process to end is: there are no duplicate solutions in the joint population Uop or there are only N solutions left in the joint population Uop.

Images