Mixed feature selection method for improving particle swarm optimization

Abstract

The invention discloses a hybrid feature selection method for improving a particle swarm algorithm, and the method comprises the following steps: completing the dimension reduction of high-dimensional data through employing a dynamic clustering method based on an unbalanced class, and dividing the high-dimensional features into feature cluster sets of different classes; initializing two PSO (Particle Swarm Optimization) populations with the same particle number and different objective functions through a sampling feature initialization strategy; and finishing screening of a final feature subset by using a double-target double-population coevolution PSO algorithm. The invention provides a novel dynamic clustering method based on unbalanced classes, provides a PSO-based double-population bifocus multivariate selection coevolution algorithm, and sets an archive-based PSO multi-direction search strategy to update particle positions. The file is used for storing the optimal solutions with the same objective function value and different feature combinations, and when the speed of the particles is updated, the different optimal solutions in the file are considered, so that multi-direction guidance is provided for evolution of the particles, more areas with potential values are explored, and the probability of falling into local optimum is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to a hybrid feature selection method of an improved particle swarm algorithm. Background Art

[0002] Machine learning is constantly penetrating into all aspects of human life, providing a convenient and easy-to-operate way for the originally tedious and complex work in human life. With the deepening of the aging of the population in recent years, medical services are facing increasing pressure. In machine learning, classification is an important task, which is to complete the classification of instances through the characteristics of instances. According to this feature of machine learning, the classification task of machine learning is very suitable for the diagnosis of some medical diseases, so as to reduce the diagnostic pressure of medical service personnel and reduce the burden of the aging population on the medical service industry. Although this method is feasible in theory, in practical applications, since most medical data is high-dimensional data and contains a high proportion of redundant features and noise, this phenomenon will not only bring huge computational costs, but also affect the classification performance of the final model. Therefore, how to reasonably and effectively perform feature selection processing on medical data is a huge challenge for the actual application of machine learning in the medical field. Because medical data has the characteristics of a large number of features, it will easily lead to "dimensionality disaster", which greatly increases the computational difficulty and time of analyzing and processing medical data. In addition, there are a lot of noise and redundant features in these data. The existence of these features and noise not only increases the computational cost, but also affects the classification accuracy of the final model. From this, we can conclude that feature selection is a means of eliminating redundant features and noise and ultimately selecting relevant and complementary feature subsets to achieve the purpose of reducing data dimensions while ensuring classification accuracy.

[0003] Feature selection methods can be roughly divided into filter methods, embedded methods, wrapper methods and hybrid methods. The filter method is to filter out redundant features and noise in the data set by calculating the dependency between features and labels or the correlation between features, so as to achieve the effect of dimensionality reduction. However, this method is irrelevant to the subsequent learning algorithm, is too simple and lacks accuracy. The embedding method automatically executes feature selection in the process of training the subsequent learning algorithm. However, the embedding method depends on the model of the subsequent learning algorithm, and some models may not be suitable. Wrapper method, using classification performance to evaluate feature subsets, traditional evolutionary algorithms have been proven to have strong search capabilities. In the past period of time, many researchers have applied evolutionary algorithms with global optimization capabilities to the field of feature selection. Among them, the particle swarm optimization algorithm has become a popular and representative group intelligence algorithm applied to various feature selection problems due to its simple concept, easy implementation, strong global search capability and fast convergence speed, and has quickly derived many improved versions of the PSO algorithm. However, it is worth noting that although these existing PSO variant methods have made great progress in the field of feature selection, when they are applied to high-dimensional data, the shortcomings and weaknesses of these methods have been exposed, for example: the global optimum may have multiple different feature combinations, but how to deal with these combinations and select the particle evolution direction during the evolution process; how to more reasonably balance the relationship between the classification performance and the number of features of the selected feature subset; the PSO algorithm is more likely to fall into local optimality, but how to effectively monitor and escape is still a huge challenge.

[0004] Hybrid methods combine the advantages of different types of feature selection methods and use different methods in different stages of dimensionality reduction tasks to further improve the classification accuracy of feature selection. Existing hybrid methods can be divided into two categories: filter-wrapper hybrid methods and embedded-wrapper hybrid methods. This type of hybrid method is a two-stage hybrid method. The first stage is to reduce the search space by removing irrelevant or redundant features, and the second stage is to find the best feature subset from a small range of feature search space. Although these hybrid methods have achieved remarkable results in the field of feature selection, they still face the following challenges. For example, in the filtering stage, only some features are deleted by correlation, but the ability to reduce the search space is extremely limited when processing high-dimensional data, and it is often easy to mistakenly delete features that are effective in combination with other features but have weak correlation. Although the feature search space can be further reduced to a certain extent by using clustering methods in the filtering stage, there are still certain limitations. Summary of the invention

[0005] The purpose of the present invention is to provide a hybrid feature selection method of an improved particle swarm algorithm in response to the above-mentioned problems currently existing.

[0006] The technical solution of the present invention is as follows:

[0007] A hybrid feature selection method for improving particle swarm optimization algorithm comprises the following steps:

[0008] A dynamic clustering method based on unbalanced classes is used to reduce the dimensionality of high-dimensional data and divide high-dimensional features into feature clusters of different classes.

[0009] Initialize two PSO populations with the same number of particles but different objective functions through sampling feature initialization strategy;

[0010] The dual-objective dual-population co-evolution PS0 algorithm is used to complete the screening of the final feature subset.

[0011] Furthermore, the dynamic clustering method based on unbalanced classes comprises the following steps:

[0012] MI is used to measure the degree of mutual dependence between features and target labels, capturing the linear or nonlinear relationship between features and target labels. The MI formula is:

[0013]

[0014] Where X and Y are two variables to be measured, p(x, y) is the joint probability distribution of X and Y, and p(x) and p(y) are their respective marginal probability distributions. If X and Y are independent, MI(X, Y) = 0. Otherwise, if X and Y are completely correlated, MI(X, Y) = 1.

[0015] The SU value is used to compare the interdependence between different data sets or different variable pairs. The SU formula is:

[0016]

[0017] H(X)=-∑x ∈X p(x)log2 p(x),

[0018] H(Y)=-∑y ∈Y p(y)log2 p(y),

[0019] Among them, H(X) and H(Y) are the entropies of variables X and Y, respectively, which are used to measure the uncertainty of random variables, and p(x) and p(y) are the probabilities of variables X and Y taking x and y, respectively;

[0020] Aggregate highly redundant features into the same feature cluster: Use an adaptive unbalanced class feature screening strategy to separate key features for the class, and each key feature is independently clustered; analyze and screen candidate feature differences; analyze feature correlations and form feature clusters.

[0021] Furthermore, the screening of the adaptive imbalanced features includes the following steps:

[0022] In the initial stage of clustering each class, the FL-Relrvance between each candidate feature and the current class is calculated, and the candidate features are arranged in descending order according to the FL-Relrvance. FL-Relrvance is the feature f i With a certain label label c The correlation between the two is determined by the uncertainty value express,

[0023]

[0024] The number of key features that need to be separated is calculated based on the number of categories. The features with the highest relevance ranking of key_num are separated. The separated key features will form a cluster independently, and the remaining features will continue to enter the second stage as candidate features to complete clustering.

[0025]

[0026] Among them, x represents the number of categories, and feature_num represents the number of all candidate features.

[0027] Furthermore, the candidate feature difference analysis and screening includes the following steps:

[0028] Select the feature f1 with the highest FL-Relrvance from the candidate features and use f1 as the feature cluster to be formed in this round i The first feature of , and f1 is called the selected feature;

[0029] Calculate the G-Differences between the selected feature f1 and all other candidate features, and store the calculated G-Differences in the list GAP_Array. G-Differences are the same class label label c Next, two features f i and f j The difference between them is determined by the feature f i With class label label c FL-Relevance value and feature f j With class label label c FL-Relevance value The absolute value of the difference express,

[0030]

[0031] The larger the value of , the greater the difference between the two features, and vice versa, the greater the correlation between the two features;

[0032] After completing the difference analysis between the selected feature f1 and the candidate features, sort the GAP_Array from small to large and obtain the GAP with the largest G-Differences value max ;

[0033] Set the difference threshold Flag based on the currently selected feature f1 to remove features that are too different from the selected feature f1 to avoid putting them into the same feature cluster.

[0034]

[0035] Where n is the number of current candidate features;

[0036] When each class performs clustering for the first time, it is also necessary to complete the initialization of the dynamic threshold th. The specific calculation process of th initialization is shown in the following formula. After that, each time a complete feature cluster is formed, th is updated using other methods:

[0037]

[0038] Where n is the number of all features except the separated key features.

[0039] Furthermore, the feature correlation analysis and feature cluster formation include the following steps:

[0040] Select the features that are strongly correlated with the selected feature f1 from the candidate features screened by differences, and cluster the features and the selected feature f1 into a feature cluster cluster i In the above example, the FF-Relevance between the selected feature and the candidate feature is calculated. FF-Relevance is the sum of the two features f i and f j The correlation between them is determined by the uncertainty value SU ij express,

[0041]

[0042] If the FF-Relevance between the selected feature f1 and the candidate feature is greater than th, it is considered that there is a strong correlation between the two features, and the candidate feature is stored in the feature cluster formed in this round. i On the contrary, it is considered that there is a weak correlation between the two, and the candidate feature is abandoned and stored in the feature cluster formed in this round, and is left for the subsequent clustering process;

[0043] After the complete execution, a new feature cluster based on the current class will be obtained. i , a dynamic mechanism is used to update the dynamic threshold th, and finally the candidate feature difference analysis and screening are returned to continue clustering the candidate features that have not yet been clustered;

[0044] After completing feature clustering, each class will form an ordered sequence of feature clusters, and the feature distribution within each feature cluster is also ordered, where the feature distribution within each feature cluster satisfies:

[0045] For class label c The i-th feature cluster in the ordered feature cluster sequence formed i ={f3, f6, ..., f i}, where the distribution of features within the cluster is based on each feature and class label c The FL-Relevance between them is sorted from large to small.

[0046] Furthermore, the sampling feature initialization strategy includes the following steps:

[0047] Arrange multiple ordered feature cluster sequences formed by all classes in rows. The first row is the feature cluster set formed by the first class, and so on, eventually forming n rows of feature cluster sequences, where n is the number of classes.

[0048] Cluster them vertically and ensure the order, that is, feature clusters with the same index in each row form a group and are sorted by row number;

[0049] The groups formed by vertical aggregation are horizontally spliced ​​and the order is guaranteed, that is, starting from the last column, the groups are gradually spliced ​​to the end of the groups in the previous column;

[0050] The feature cluster sequence formed by horizontal splicing is divided according to the number of categories n. The first n feature clusters are indexed as one group. By analogy, N groups will be obtained. The number of feature clusters in the first N-1 groups is n, and the number of feature clusters in the last group is in the interval [1, n].

[0051] After the feature clusters are grouped, the dynamic threshold p of each feature cluster is calculated in each group. p acts on the probability of determining whether the feature cluster participates in the initialization of the current particle. Before determining whether a feature cluster participates in the initialization of the current particle, a random number r is generated. If the random number r is less than p of the corresponding cluster, the feature is selected in the cluster. Otherwise, the feature is not selected in the cluster. The dynamic threshold p is the FL-Relevance value of the first feature of each cluster calculated in the same feature cluster group. And the sum of the FL-Relevance values ​​of the first feature in all feature clusters in this group (SU all ), and use p as the dynamic threshold for selecting features in this cluster.

[0052]

[0053] Integer encoding is used to describe the selected feature subset, X = (x1, x2, ..., x M ), x i ∈{0,1,...,|cluster i |},

[0054] Among them, x i =a means that the feature with index a in feature cluster i is selected into the feature subset; but if x i = -1 means not from the feature cluster cluster i Select any feature in .

[0055] Furthermore, the dual-objective dual-population co-evolution PSO algorithm includes the following steps:

[0056] Divide the population into PureACC_pop and HybridFeature_pop populations;

[0057] The PureACC_pop population is optimized with the classification performance of the selected feature subset as the goal, and the objective function is:

[0058] Fitness1 = 1-BACC,

[0059] Among them, BACC is the balanced accuracy of the base classifier used,

[0060] Among them, c is the number of label categories, TP i is the number of samples correctly classified in the i-th class label, N i is the number of samples of the i-th label;

[0061] The HybridFeature_pop population takes the comprehensive score of the classification performance of the selected feature subset and the number of features as the optimization goal, and the objective function is:

[0062]

[0063] Among them, α is a weight ranging from 0 to 1, and BACC is the balanced accuracy of the base classifier used.

[0064] Furthermore, the dual-objective dual-population co-evolution PSO algorithm also includes an optimal solution update strategy based on archives and a dual elite fusion strategy based on a population stagnation detector. The optimal solution update strategy based on archives includes:

[0065] Each population sets the global optimal archive gb ={gb1, gb2, ..., gb n}, used to store optimal solutions with the same objective function score but different feature combinations;

[0066] If the objective function score of a particle exceeds the global optimum, the global optimum file will be cleared and the particle will be stored in it; if the objective function score of a particle is equal to the global optimum and is not in the global optimum file, it will be added to the global optimum file; the speed update formula is:

[0067] V ij =w*V ij +c1*r1*(gb rj -dp ij )+c2*r2*(pb ij -dp ij ),

[0068] Among them, w is the inertia weight, c1 and c2 are constants, r1 and r2 are random numbers in the interval [0, 1], and r is a random number in the interval [0, n-1], representing the number of samples from the archive. gb Randomly select particles to complete the velocity update.

[0069] Furthermore, the dual elite fusion strategy based on population stagnation detector includes:

[0070] Set up a population stagnation detector monitor stagntion , through the monitor of HybridFeature_pop and PureACC_pop stagntion The value of is used to determine whether a population has fallen into a local optimum. Once a population is found to have fallen into a local optimum, the population immediately requests to execute a dual elite fusion strategy with another population, and uses the communication between the two populations with different objective functions to update some particles in the population that has fallen into the local optimum.

[0071] monitor stagntionThe evaluation criteria for whether the population evolution occurs are whether the objective function score of the global optimal particle in the PSO population is improved or the objective function remains the same and the number of features decreases. If, in a round of population iteration, the objective function score of a new particle in the HybridFeature_pop or PureACC_pop population is improved compared with the global optimal particle or the objective function remains the same as the global optimal particle and the number of features decreases, it is considered that the HybridFeature_pop or PureACC_pop population is optimized in this round, and the monitor of the population is stagntion Perform the clear operation, otherwise, the monitor of the population stagntion Perform accumulation operations to record the optimization stagnation of this round.

[0072] monitor stagntion =0,

[0073] monitor stagntion +=1,

[0074] Among them, monitor stagntion Represents the population stagnation detector of HybridFeature_pop or PureACC_pop. When the monitor of HybridFeature_pop or PureACC_pop stagntion If the value of is greater than the predetermined threshold qz, it is considered that HybridFeature_pop or PureACC_pop is trapped in the local optimum, and the population immediately initiates an information exchange request to PureACC_pop or HybridFeature_pop, executes the dual elite fusion strategy, and updates some particles of the population to escape the local optimum.

[0075] Furthermore, the dual elite fusion strategy based on population stagnation detector specifically includes the following steps:

[0076] According to the objective function of the population HybridFeature_pop or PureACC_pop that is trapped in the local optimum, obtain the particle dp with the best objective function score among the existing particles in HybridFeature_pop or PureACC_pop best , suboptimal dp sbest and worst dp worse , and the best particle gb recorded in the population best and the worst particle gW worse ; Using the same objective function, obtain the particle dp with the best objective function score among the existing particles from PureACC_pop or HybridFeature_pop best ′, suboptimal dp sbest′ and worst dp worse ′, and the best particle gb recorded in the population best ′ and the worst particle gW worse ′;

[0077] The three high-quality particles dp obtained from the population HybridFeature_pop or PureACC_pop that are trapped in the local optimum best , dp sbest and gb best With 3 high-quality particles dp obtained from PureACC_pop or HybridFeature_pop best ′, dp sbest ′ and gb best 'Forming a combination of 6 high-quality particles; secondly, 3 high-quality particles dp best , dp sbest and gb best Each particle will be hybridized with the other five particles in the high-quality particle combination except for itself. After hybridization, each particle will generate five new particles. Then, the particle with the best objective function score is selected from the five new particles, so three new particles are finally obtained. and

[0078] Two high quality particles dp i and dp j During hybridization, in order to further retain the genes of high-quality particles, the new particles formed by hybridization are dp new Each gene of is determined according to the ratio of the inverse objective function score,

[0079]

[0080]

[0081] Among them, Finess i and Finess j dp respectively i and dp j The score calculated based on the objective function of the population trapped in the local optimum, r is a random number in the interval [0, 1];

[0082] First, the two inferior particles dp obtained from the population HybridFeature_pop or PureACC_pop that are trapped in the local optimum are worse and gw worse With 2 inferior particles dp obtained from PureACC_pop or HybridFeature_pop worse ′ and gW worse'Forming a combination of 4 inferior particles;

[0083] Secondly, 2 inferior particles dp worse and gw worse Hybridize with the three particles in the inferior particle combination except for the particle itself. After hybridization, each particle will generate three new particles. Then, the particle with the worst objective function score is selected from the three new particles, so two new particles are finally obtained. and

[0084] Two inferior particles dp i and dp j During hybridization, in order to further retain the genes of inferior particles, the new particles formed by hybridization are dp new Each gene of is determined according to the ratio of the objective function score.

[0085]

[0086] Among them, Finess i and Finess j dp respectively i and dp j The score calculated based on the objective function of the population trapped in the local optimum;

[0087] Three high-quality particles and two low-quality particles will be formed. In the population HybridFeature_pop or PureACC_pop that falls into the local optimum, five particles are randomly selected except the particles used in the hybridization process and replaced with the five new particles formed by hybridization. stagntion The value is cleared to zero.

[0088] Compared with the prior art, the present invention has the following beneficial effects:

[0089] 1. A hybrid feature selection method for improving the particle swarm algorithm. A novel dynamic clustering method based on unbalanced classes is proposed. Different from the existing methods, different feature cluster sets are generated for different classes based on the feature distribution of class imbalance. The clustering method forms a feature cluster with class characteristics for each class, effectively avoiding ignoring the class features with a small proportion. After that, a new population initialization method is proposed to initialize the population, group the feature clusters generated by different classes, and the feature clusters in each group are used as the probability of selecting features in this feature cluster based on the proportion of the correlation with its original class in the entire group. This can not only avoid the problem of a surge in the number of initial particle features due to changes in the clustering method, but also consider the importance of feature clusters of different classes to the classification of the class to more reasonably select the feature initialization population and improve the quality of the initial solution.

[0090] 2. A hybrid feature selection method for improving particle swarm algorithm. A dual-population dual-focus multi-selection co-evolution algorithm based on PSO is proposed. Two objective functions with different focuses are designed. One population uses the objective function of balanced classification accuracy, and the other population uses the objective function of fusion balanced classification accuracy and feature number. The two populations evolve in different directions without interfering with each other, thereby improving the exploration ability and adaptability of the entire algorithm. At the same time, the concept of population continuous stagnation monitor is proposed to monitor the stagnation of the global optimal particles in the population, so as to determine whether the population is trapped in the local optimum. Once a population is found to be trapped in the local optimum, the dual elite fusion strategy is immediately executed to exchange information with populations with different objective functions, help the population jump out of the local optimum and increase population diversity, better explore the solution space, and find the truly optimal solution.

[0091] 3. A hybrid feature selection method for improving the particle swarm algorithm. In order to consider more areas with potential search value, a PSO multi-directional search strategy based on archives is proposed to update the particle position. The archives are used to store the optimal solutions with the same objective function value but different feature combinations. When updating the particle speed, different optimal solutions in the archives are considered, which provides multi-directional guidance for the evolution of particles, explores more areas with potential value, and reduces the probability of falling into local optimality. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 The framework diagram of a hybrid feature selection method for an improved particle swarm algorithm. DETAILED DESCRIPTION

[0093] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0094] The features and performance of the present invention are further described in detail below in conjunction with the embodiments.

[0095] See also Figure 1 , a hybrid feature selection method for improving particle swarm algorithm, comprising the following steps:

[0096] A dynamic clustering method based on unbalanced classes is used to reduce the dimensionality of high-dimensional data and divide high-dimensional features into feature clusters of different classes.

[0097] Initialize two PSO populations with the same number of particles but different objective functions through sampling feature initialization strategy;

[0098] The dual-objective dual-population co-evolution PSO algorithm is used to complete the screening of the final feature subset.

[0099] The dynamic clustering method based on unbalanced classes generates feature clusters that match the characteristics of each class, avoiding ignoring important features of classes with a small proportion of instances, thereby providing a more comprehensive search space for subsequent wrapper algorithms. The following steps are included:

[0100] MI is used to measure the degree of mutual dependence between features and target labels, capturing the linear or nonlinear relationship between features and target labels. The MI formula is:

[0101]

[0102] Where X and Y are two variables to be measured, p(x, y) is the joint probability distribution of X and Y, and p(x) and p(y) are their respective marginal probability distributions. If X and Y are independent, MI(X, Y) = 0. Otherwise, if X and Y are completely correlated, MI(X, Y) = 1.

[0103] In order to standardize the mutual information so that it is not affected by the entropy of each variable, so as to more fairly compare the interdependence of different variable pairs, the SU value is used to compare the interdependence between different data sets or different variable pairs. The SU formula is:

[0104]

[0105] H(X)=-∑ x∈X p(x)log2 p(x),

[0106] H(T)=-∑ y∈Y p(y)log2 p(y),

[0107] Among them, H(X) and H(Y) are the entropies of variables X and Y, respectively, which are used to measure the uncertainty of random variables, and p(x) and p(y) are the probabilities of variables X and Y taking x and y, respectively;

[0108] When performing correlation-guided clustering on each class, the highly redundant features are clustered into the same feature cluster by evaluating the correlation between the features of the current class instance and the class, the difference between features, and the correlation between features.

[0109] Aggregate highly redundant features into the same feature cluster: Use an adaptive unbalanced class feature screening strategy to separate key features for the class, and each key feature is independently clustered; analyze and screen candidate feature differences; analyze feature correlations and form feature clusters.

[0110] The adaptive imbalance feature screening includes the following steps:

[0111] In the initial stage of clustering each class, the FL-Relrvance between each candidate feature and the current class is calculated, and the candidate features are arranged in descending order according to the FL-Relrvance. FL-Relrvance is the feature f i With a certain label label c The correlation between the two is determined by the uncertainty value express,

[0112]

[0113] The number of key features that need to be separated is calculated based on the number of categories. The features with the highest relevance ranking of key_num are separated. The separated key features will form a cluster independently, and the remaining features will continue to enter the second stage as candidate features to complete clustering.

[0114]

[0115] Among them, x represents the number of categories, and feature_num represents the number of all candidate features.

[0116] This method can not only highlight the key features of the current class and avoid storing the key features in the same feature cluster so that they cannot form feature combinations, but also control the number of feature clusters of the key features of each class based on the number of categories, thereby controlling the number of key feature clusters of all classes and avoiding the problem of a surge in the number of feature clusters of key features due to an increase in the number of categories.

[0117] The candidate feature difference analysis and screening includes the following steps:

[0118] Select the feature f1 with the highest FL-Relrvance from the candidate features and use f1 as the feature cluster to be formed in this round i The first feature of , and f1 is called the selected feature;

[0119] Calculate the G-Differences between the selected feature f1 and all other candidate features, and store the calculated G-Differences in the list GAP_Array. G-Differences are the same class label labelc Next, two features f i and f j The difference between them is determined by the feature f i With class label label c FL-Relevance value and feature f i With class label label c FL-Relevance value The absolute value of the difference express,

[0120]

[0121] The larger the value of , the greater the difference between the two features. Conversely, the greater the correlation between the two features;

[0122] After completing the difference analysis between the selected feature f1 and the candidate features, sort the GAP_Array from small to large and obtain the GAP with the largest G-Differences value max ;

[0123] Set the difference threshold Flag based on the currently selected feature f1 to remove features that are too different from the selected feature f1 to avoid putting them into the same feature cluster.

[0124]

[0125] Among them, n is the number of current candidate features; it is worth noting that the features that are screened out at this time are not deleted as mentioned in other filtering algorithms, but are just not added to the feature cluster cluster of the current selected feature f1. i , and continue to participate in the subsequent clustering process. The advantage of this is that no matter how small the correlation is, the feature will be stored in a certain feature cluster, so that it still has a certain probability of being selected, avoiding the mistaken deletion of features that are effective in combination with other features but have weak correlation.

[0126] When each class performs clustering for the first time, it is also necessary to complete the initialization of the dynamic threshold th. The specific calculation process of th initialization is shown in the following formula. After that, each time a complete feature cluster is formed, th is updated using other methods:

[0127]

[0128] Where n is the number of all features except the separated key features.

[0129] Feature correlation analysis and feature cluster formation include the following steps:

[0130] Select the features that are strongly correlated with the selected feature f1 from the candidate features screened by differences, and cluster the features and the selected feature f1 into a feature cluster cluster i In the above example, the FF-Relevance between the selected feature and the candidate feature is calculated. FF-Relevance is the sum of the two features f i and f j The correlation between them is determined by the uncertainty value SU ij express,

[0131]

[0132] If the FF-Relevance between the selected feature f1 and the candidate feature is greater than th, it is considered that there is a strong correlation between the two features, and the candidate feature is stored in the feature cluster formed in this round. i On the contrary, it is considered that there is a weak correlation between the two, and the candidate feature is abandoned and stored in the feature cluster formed in this round, and is left for the subsequent clustering process;

[0133] After the complete execution, a new feature cluster based on the current class will be obtained. i , a dynamic mechanism is used to update the dynamic threshold th, and finally the candidate feature difference analysis and screening are returned to continue clustering the candidate features that have not yet been clustered;

[0134] After completing feature clustering, each class will form an ordered sequence of feature clusters, and the feature distribution within each feature cluster is also ordered, where the feature distribution within each feature cluster satisfies:

[0135] For class label c The i-th feature cluster in the ordered feature cluster sequence formed i ={f3, f6, ..., f i}, where the distribution of features within the cluster is based on each feature and class label c The FL-Relevance between them is sorted from large to small.

[0136] The sampling feature initialization strategy includes the following steps:

[0137] Arrange multiple ordered feature cluster sequences formed by all classes in rows. The first row is the feature cluster set formed by the first class, and so on, eventually forming n rows of feature cluster sequences, where n is the number of classes.

[0138] Cluster them vertically and ensure the order, that is, feature clusters with the same index in each row form a group and are sorted by row number;

[0139] The groups formed by vertical aggregation are horizontally spliced ​​and the order is guaranteed, that is, starting from the last column, the groups are gradually spliced ​​to the end of the groups in the previous column;

[0140] The feature cluster sequence formed by horizontal splicing is divided according to the number of categories n. The first n feature clusters are indexed as one group. By analogy, N groups will be obtained. The number of feature clusters in the first N-1 groups is n, and the number of feature clusters in the last group is in the interval [1, n].

[0141] Through the combination of vertical aggregation and horizontal splicing, the columns with feature clusters of each class are grouped by column. In the case of a large number of feature clusters formed by clustering of certain classes, they are also grouped reasonably to avoid independent clustering. In the end, not only the number of groups is reduced to a level close to the number of clusters generated by traditional classification-specific clustering methods, but also reasonable grouping provides a step for subsequent reduction of the number of features.

[0142] After the feature clusters are grouped, the dynamic threshold p of each feature cluster is calculated in each group. p acts on the probability of determining whether the feature cluster participates in the initialization of the current particle. Before determining whether a feature cluster participates in the initialization of the current particle, a random number r is generated. If the random number r is less than p of the corresponding cluster, the feature is selected in the cluster. Otherwise, the feature is not selected in the cluster. The dynamic threshold p is the FL-Relevance value of the first feature of each cluster calculated in the same feature cluster group. And the sum of the FL-Relevance values ​​of the first feature in all feature clusters in this group (SU all ), and use p as the dynamic threshold for selecting features in this cluster.

[0143]

[0144] Integer encoding is used to describe the selected feature subset, X = (x1, x2, ..., x M ), x i ∈{0,1,...,|cluster i |},

[0145] Among them, x i =a represents feature cluster i The feature with index a in is selected into the feature subset; but if x i = -1 means not from the feature cluster cluster i Select any feature in .

[0146] The dual-objective dual-population co-evolutionary PSO algorithm includes the following steps:

[0147] Divide the population into PureACC_pop and HybridFeature_pop populations;

[0148] The PureACC_pop population is optimized with the classification performance of the selected feature subset as the goal, and the objective function is:

[0149] Fitness1 = 1-BACC,

[0150] Among them, BACC is the balanced accuracy of the base classifier used,

[0151] Among them, c is the number of label categories, TP i is the number of samples correctly classified in the i-th class label, N i is the number of samples of the i-th label;

[0152] The HybridFeature_pop population takes the comprehensive score of the classification performance of the selected feature subset and the number of features as the optimization goal, and the objective function is:

[0153]

[0154] Among them, α is a weight ranging from 0 to 1, and BACC is the balanced accuracy of the base classifier used.

[0155] The dual-objective dual-population co-evolutionary PSO algorithm also includes an optimal solution update strategy based on archives and a dual elite fusion strategy based on a population stagnation detector. The optimal solution update strategy based on archives includes:

[0156] Each population sets the global optimal archive gb ={gb1, gb2, ..., gb n}, used to store optimal solutions with the same objective function score but different feature combinations;

[0157] If the objective function score of a particle exceeds the global optimum, the global optimum file will be cleared and the particle will be stored in it; if the objective function score of a particle is equal to the global optimum and is not in the global optimum file, it will be added to the global optimum file; the speed update formula is:

[0158] v ij =w*v ij +c1*r1*(gb rj -dp ij )+c2*r2*(pb ij -dp ij ),

[0159] Among them, w is the inertia weight, c1 and c2 are constants, r1 and r2 are random numbers in the interval [0, 1], and r is a random number in the interval [0, n-1], representing the number of samples from the archive. gb Randomly select particles to complete the velocity update.

[0160] By treating it as two sub-problems, and the two populations evolve in two different directions respectively, not only can the population diversity of the entire large population be improved, but also the population trapped in the local optimum can be assisted to escape through information interaction between the two populations with different target directions, so as to avoid premature convergence to the non-global optimum, thereby avoiding the problem of decreased population diversity and premature convergence of the algorithm in the single-objective problem.

[0161] The dual elite fusion strategy based on population stagnation detector includes:

[0162] Set up a population stagnation detector monitor stagntion , through the monitor of HybridFeature_pop and PureACC_pop stagntion The value of is used to determine whether a population has fallen into a local optimum. Once a population is found to have fallen into a local optimum, the population immediately requests to execute a dual elite fusion strategy with another population, and uses the communication between the two populations with different objective functions to update some particles in the population that has fallen into the local optimum.

[0163] monitor stagntion The evaluation criteria for whether the population evolution occurs are whether the objective function score of the global optimal particle in the PSO population is improved or the objective function remains the same and the number of features decreases. If, in a round of population iteration, the objective function score of a new particle in the HybridFeature_pop or PureACC_pop population is improved compared with the global optimal particle or the objective function remains the same as the global optimal particle and the number of features decreases, it is considered that the HybridFeature_pop or PureACC_pop population is optimized in this round, and the monitor of the population is stdgntion Perform the clear operation, otherwise, the monitor of the population stagntion Perform accumulation operations to record the optimization stagnation of this round.

[0164] monitor stagntion =0,

[0165] monitor stagntion +=1,

[0166] Among them, monitor stagntionRepresents the population stagnation detector of HybridFeature_pop or PureACC_pop. When the monitor of HybridFeature_pop or PureACC_pop stagntion If the value of is greater than the predetermined threshold qz, it is considered that HybridFeature_pop or PureACC_pop is trapped in the local optimum, and the population immediately initiates an information exchange request to PureACC_pop or HybridFeature_pop, executes the dual elite fusion strategy, and updates some particles of the population to escape the local optimum.

[0167] The dual elite fusion strategy based on population stagnation detector specifically includes the following steps:

[0168] According to the objective function of the population HybridFeature_pop or PureACC_pop that is trapped in the local optimum, obtain the particle dp with the best objective function score among the existing particles in HybridFeature_pop or PureACC_pop best , suboptimal dp sbest and worst dp worse , and the best particle gb recorded in the population best and the worst particle gw worse ; Using the same objective function, obtain the particle dp with the best objective function score among the existing particles from PureACC_pop or HybridFeature_pop best ′, suboptimal dp sbest ′ and worst dp worse ′, and the best particle gb recorded in the population best ′ and the worst particle gW worse ′;

[0169] The three high-quality particles dp obtained from the population HybridFeature_pop or PureACC_pop that are trapped in the local optimum best , dp sbest and gb best With 3 high-quality particles dp obtained from PureACC_pop or HybridFeature_pop best ′, dp sbest ′ and gb best 'Forming a combination of 6 high-quality particles; secondly, 3 high-quality particles dp best , dp sbest and gb bestEach particle will be hybridized with the other five particles in the high-quality particle combination except for itself. After hybridization, each particle will generate five new particles. Then, the particle with the best objective function score is selected from the five new particles, so three new particles are finally obtained. and

[0170] Two high quality particles dp i and dp j During hybridization, in order to further retain the genes of high-quality particles, the new particles formed by hybridization are dp new Each gene of is determined according to the ratio of the inverse objective function score,

[0171]

[0172] Among them, Finess i and Finess j dp respectively i and dp j The score calculated based on the objective function of the population trapped in the local optimum, r is a random number in the interval [0, 1];

[0173] First, the two inferior particles dp obtained from the population HybridFeature_pop or PureACC_pop that are trapped in the local optimum are worse and gw worse With 2 inferior particles dp obtained from PureACC_pop or HybridFeature_pop worse ′ and gw worse 'Forming a combination of 4 inferior particles;

[0174] Secondly, 2 inferior particles dp worse and gw worse Hybridize with the three particles in the inferior particle combination except for the particle itself. After hybridization, each particle will generate three new particles. Then, the particle with the worst objective function score is selected from the three new particles, so two new particles are finally obtained. and

[0175] Two inferior particles dp i and dp j During hybridization, in order to further retain the genes of inferior particles, the new particles formed by hybridization are dp new Each gene of is determined according to the ratio of the objective function score.

[0176]

[0177] Among them, Finess iand Finess j dp respectively i and dp j The score calculated based on the objective function of the population trapped in the local optimum;

[0178] Three high-quality particles and two low-quality particles will be formed. In the population HybridFeature_pop or PureACC_pop that falls into the local optimum, five particles are randomly selected except the particles used in the hybridization process and replaced with the five new particles formed by hybridization. stagntion The value is cleared to zero.

[0179] Experimental verification

[0180] Dataset

[0181] Experiments were conducted on 10 high-dimensional medical datasets that have been studied in the field of feature selection under the same conditions to verify the effectiveness of the proposed method. Prostate is the abbreviation of Prostate_Tumor. The 10 datasets used are all open source datasets from https: / / github.com / pri mekangkang / Genedata. The detailed information of the datasets is shown in Table 1, which lists the number of features, total number of instances, and number of label classes for each dataset. It can be found that the number of features of the datasets used ranges from 2308 to 12600, the number of instances ranges from 50 to 203, and the number of label classes ranges from 2 to 11, and the total number of instances is much smaller than the number of features.

[0182]

[0183] Table 1. Overview of the dataset

[0184] Comparison Programs

[0185] To verify the effectiveness of the algorithm, five state-of-the-art methods are used as baselines for comparison with the proposed method. Bi-DFS is a two-layer discrete feature selection method, which performs feature selection in the upper layer and discretization in the lower layer. PSO-FS is a two-stage feature selection method, while EPSO and PPSO are single-stage feature selection methods. CCDFS is a collaborative co-evolutionary algorithm based on genetic algorithm and particle swarm algorithm.

[0186] Due to the randomness of particle swarm initialization and PSO optimization, 50 experiments were performed on each data set. The accuracy of 50 experiments was taken as the final experimental result of the corresponding data set. In addition, in order to fairly compare the algorithm with Bi-DFS, PSO-FS, EPSO, PPSO and CCDFS, the same learner (KNN with K=5) was used as the base classifier. At the same time, all codes are implemented in Python and sklearn.

[0187] Since the algorithm has two populations, in order to avoid unfairness in comparison, the total size is set to 40 (the population size of the two PSO populations with different objective functions is half of the total population size, which is 20). Each baseline method will be trained for 50 rounds, and the number of iterations in each round of training is 100.

[0188] Performance Comparison

[0189] The performance of our algorithm will be comprehensively compared with other feature selection methods. Table 2 lists the performance results of various feature selection algorithms based on the 10 real data sets mentioned above. Among them, "#Feature" represents the number of features selected after the corresponding algorithm is executed, "BestBAcc" represents the optimal classification accuracy of the corresponding algorithm in 50 rounds, and "MedianBAcc" represents the average classification accuracy of the corresponding algorithm in 50 rounds.

[0190] In terms of the best classification accuracy and median classification accuracy, the proposed method outperforms all the compared methods on most datasets, especially on the 9_Tumor, 11_Tumor, Brain_Tumor1, Brain_Tumor2, and Lung_Cancer datasets, where the proposed method achieves significant improvements over other methods.

[0191] In terms of dimensionality reduction, although the method selected more features than other algorithms in some data sets, due to clustering for label classes, the difference between the number of features selected in the 9 data sets except DLBCL and the number of features selected by other algorithms is far less than the number of categories. In fact, the number of features selected in the SRBCT, Leukemia1, Brain_Tumor1, Lung_Cancer, and 9_Tumor data sets is far less than the number of features selected by other algorithms. In the DLBCL data set, although the number of features in the final feature subset of the method is large, the classification accuracy obtained by the method is 100, which is higher than the other five algorithms.

[0192] In a side-by-side comparison with state-of-the-art methods, our method achieves fewer features than the Bi-DFS method on five datasets and outperforms its classification performance on nine datasets, with the dataset with the most significant gap in best classification accuracy being 9_Tumors. On this dataset, the proposed method achieves the best classification accuracy, 31.46% higher than BiDFS. In addition, the dataset with the largest gap in median classification accuracy is still 9_Tumors, where the proposed method achieves a median classification accuracy of 86.92%, 24.91% higher than Bi-DFS. At the same time, the average number of features selected by the proposed method on this dataset is 50.34, which is much lower than the number of features selected by Bi-DFS. On the Prostate_Tumor dataset, the best classification accuracy is comparable to that of the proposed method, although its median classification accuracy is lower than that of Bi-DFS.

[0193] In a side-by-side comparison with the most advanced methods, compared with the two-stage PSO-FS method, the number of features obtained by the method on each dataset is far less than the number of categories than the number of features selected by PSO-FS. In 8 datasets, the number of features obtained is even far less than that of PSO-FS, which proves the success of the sampling strategy in the method and does not lead to a surge in the number of features due to targeted label clustering. The method achieves better classification accuracy than PSO-FS on all datasets, and on the dataset 9Tumor, the best classification accuracy of HDFSDC-IP is 45.56% higher than that of PSO-FS, and the median classification accuracy is 40.81% higher.

[0194] In a side-by-side comparison with state-of-the-art methods, our method outperforms single-stage EPSO and PPSO methods in terms of dimensionality reduction on seven and six datasets, respectively, and outperforms all datasets in terms of classification performance. In particular, our method outperforms them in terms of best classification accuracy on the 9Tumor dataset by 35.80% and 34.60%, respectively, while in terms of median classification accuracy, our method outperforms EPSO by 29.35% and PPSO by 28.79%, respectively.

[0195] In a side-by-side comparison with the state-of-the-art methods, our method outperforms CCDFS on all eight datasets in terms of dimensionality reduction and outperforms CCDFS on all datasets in terms of classification accuracy. In terms of classification accuracy, our method performs well on all datasets.

[0196]

[0197]

[0198]

[0199] Table 2 Experimental results

[0200] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A hybrid feature selection method based on an improved particle swarm algorithm, characterized in that: The following steps are involved: A dynamic clustering method based on unbalanced classes is used to reduce the dimensionality of high-dimensional data and divide high-dimensional features into feature clusters of different classes. Initialize two PSO populations with the same number of particles but different objective functions through sampling feature initialization strategy; The dual-objective dual-population co-evolution PSO algorithm is used to complete the screening of the final feature subset.

2. The hybrid feature selection method of the improved particle swarm algorithm according to claim 1 is characterized in that: The dynamic clustering method based on unbalanced classes comprises the following steps: MI is used to measure the degree of mutual dependence between features and target labels, capturing the linear or nonlinear relationship between features and target labels. The MI formula is: Where X and Y are two variables to be measured, p(x, y) is the joint probability distribution of X and Y, and p(x) and p(y) are their respective marginal probability distributions. If X and Y are independent, MI(X, Y) = 0. Otherwise, if X and Y are completely correlated, MI(x, Y) = 1. The SU value is used to compare the interdependence between different data sets or different variable pairs. The SU formula is: H(X)=-∑ x∈X p(x)log2p(x), H(Y)=-∑ y∈Y p(y)log2p(y), Among them, H(X) and H(Y) are the entropies of variables X and Y, respectively, which are used to measure the uncertainty of random variables, and p(x) and p(y) are the probabilities of variables X and Y taking x and y, respectively; Aggregate highly redundant features into the same feature cluster: Use an adaptive unbalanced class feature screening strategy to separate key features for the class, and each key feature is independently clustered; analyze and screen candidate feature differences; analyze feature correlations and form feature clusters.

3. The hybrid feature selection method of the improved particle swarm algorithm according to claim 2 is characterized in that: The screening of the adaptive imbalanced features comprises the following steps: In the initial stage of clustering each class, the FL-Relrvance between each candidate feature and the current class is calculated, and the candidate features are arranged in descending order according to the FL-Relrvance. FL-Relrvance is the feature f i With a certain label label c The correlation between the two is determined by the uncertainty value express, The number of key features that need to be separated is calculated based on the number of categories. The features with the highest relevance ranking of key_num are separated. The separated key features will form a cluster independently, and the remaining features will continue to enter the second stage as candidate features to complete clustering. Among them, x represents the number of categories, and feature_num represents the number of all candidate features.

4. The hybrid feature selection method of the improved particle swarm algorithm according to claim 3 is characterized in that: The candidate feature difference analysis and screening includes the following steps: Select the feature f1 with the highest FL-Relrvance from the candidate features and use f1 as the feature cluster to be formed in this round i The first feature of , and f1 is called the selected feature; Calculate the G-Differences between the selected feature f1 and all other candidate features, and store the calculated G-Differences in the list GAP_Array. G-Differences are the same class label label c Next, two features f i and f j The difference between them is determined by the feature f i With class label label c FL-Relevance value and feature f j With class label label c FL-Relevance value The absolute value of the difference express, The larger the value of , the greater the difference between the two features, and vice versa, the greater the correlation between the two features; After completing the difference analysis between the selected feature f1 and the candidate features, sort the GAP_Array from small to large and obtain the GAP with the largest G-Differences value max ; Set the difference threshold Flag based on the currently selected feature f1 to remove features that are too different from the selected feature f1 to avoid putting them into the same feature cluster. Where n is the number of current candidate features; When each class performs clustering for the first time, it is also necessary to complete the initialization of the dynamic threshold th. The specific calculation process of th initialization is shown in the following formula. After that, each time a complete feature cluster is formed, th is updated using other methods: Where n is the number of all features except the separated key features.

5. The hybrid feature selection method of the improved particle swarm algorithm according to claim 4 is characterized in that: The feature correlation analysis and feature cluster formation include the following steps: Select the features that are strongly correlated with the selected feature f1 from the candidate features screened by differences, and cluster the features and the selected feature f1 into a feature cluster cluster i In the above example, the FF-Relevance between the selected feature and the candidate feature is calculated. FF-Relevance is the sum of the two features f i and f j The correlation between them is determined by the uncertainty value SU ij express, If the FF-Relevance between the selected feature f1 and the candidate feature is greater than th, it is considered that there is a strong correlation between the two features, and the candidate feature is stored in the feature cluster formed in this round. i On the contrary, it is considered that there is a weak correlation between the two, and the candidate feature is abandoned and stored in the feature cluster formed in this round, and is left for the subsequent clustering process; After the complete execution, a new feature cluster based on the current class will be obtained. i , a dynamic mechanism is used to update the dynamic threshold th, and finally the candidate feature difference analysis and screening are returned to continue clustering the candidate features that have not yet been clustered; After completing feature clustering, each class will form an ordered sequence of feature clusters, and the feature distribution within each feature cluster is also ordered, where the feature distribution within each feature cluster satisfies: For class label c The i-th feature cluster in the ordered feature cluster sequence formed i ={f3,f6,…,f i }, where the distribution of features within the cluster is based on each feature and class label c The FL-Relevance between them is sorted from large to small.

6. The hybrid feature selection method of the improved particle swarm algorithm according to claim 1, characterized in that: The sampling feature initialization strategy includes the following steps: Arrange multiple ordered feature cluster sequences formed by all classes in rows. The first row is the feature cluster set formed by the first class, and so on, eventually forming n rows of feature cluster sequences, where n is the number of classes. Cluster them vertically and ensure the order, that is, feature clusters with the same index in each row form a group and are sorted by row number; The groups formed by vertical aggregation are horizontally spliced ​​and the order is guaranteed, that is, starting from the last column, the groups are gradually spliced ​​to the end of the groups in the previous column; The feature cluster sequence formed by horizontal splicing is divided according to the number of categories n. The first n feature clusters are indexed as one group. By analogy, N groups will be obtained. The number of feature clusters in the first N-1 groups is n, and the number of feature clusters in the last group is in the interval [1, n]. After the feature clusters are grouped, the dynamic threshold p of each feature cluster is calculated in each group. p acts on the probability of determining whether the feature cluster participates in the initialization of the current particle. Before determining whether a feature cluster participates in the initialization of the current particle, a random number r is generated. If the random number r is less than p of the corresponding cluster, features are selected in the cluster. Otherwise, features are not selected in the cluster. The dynamic threshold p is the FL-Relevance value of the first feature of each cluster in the same feature cluster group, and the proportion p of the sum of the FL-Relevance values ​​of the first features of all feature clusters in this group, and p is used as the dynamic threshold for selecting features in the cluster. Integer encoding is used to describe the selected feature subset, X = (x1, x2, ..., x M ),x i ∈{0,1,…,|cluster i ∣}, Among them, x i =a represents feature cluster i The feature with index a in is selected into the feature subset; but if x i = -1 means not from the feature cluster cluster i Select any feature in .

7. The hybrid feature selection method of the improved particle swarm algorithm according to claim 1, characterized in that: The dual-objective dual-population co-evolution PSO algorithm comprises the following steps: Divide the population into PureACC_pop and HybridFeature_pop populations; The PureACC_pop population is optimized with the classification performance of the selected feature subset as the goal, and the objective function is: Fitness1 = 1-BACC, Among them, BACC is the balanced accuracy of the base classifier used, Among them, c is the number of label categories, TP i is the number of samples correctly classified in the i-th label, N i is the number of samples of the i-th label; The HybridFeature_pop population takes the comprehensive score of the classification performance of the selected feature subset and the number of features as the optimization goal, and the objective function is: Among them, α is a weight ranging from 0 to 1, and BACC is the balanced accuracy of the base classifier used.

8. The hybrid feature selection method of the improved particle swarm algorithm according to claim 7, characterized in that: The dual-objective dual-population co-evolution PSO algorithm also includes an optimal solution update strategy based on archives and a dual elite fusion strategy based on a population stagnation detector. The optimal solution update strategy based on archives includes: Each population sets the global optimal archive gb ={gb1,gb2,…,gb n }, used to store optimal solutions with the same objective function score but different feature combinations; If the objective function score of a particle exceeds the global optimum, the global optimum file will be cleared and the particle will be stored in it; if the objective function score of a particle is equal to the global optimum and is not in the global optimum file, it will be added to the global optimum file; the speed update formula is: v ij =w*v ij +c1-r1*(gb rj -dp ij )+c2*r2-(pb ij -dp ij ), Among them, w is the inertia weight, c1 and c2 are constants, r1 and r2 are random numbers in the interval [0,1], and r is a random number in the interval [0,n-1], representing the number of samples from the archive. gb Randomly select particles to complete the velocity update.

9. The hybrid feature selection method of the improved particle swarm algorithm according to claim 8, characterized in that: The dual elite fusion strategy based on population stagnation detector includes: Set up a population stagnation detector monitor stagntion , through the monitor of HybridFeature_pop and PureACC_pop stagntion The value of is used to determine whether a population has fallen into a local optimum. Once a population is found to have fallen into a local optimum, the population immediately requests to execute a dual elite fusion strategy with another population, and uses the communication between the two populations with different objective functions to update some particles in the population that has fallen into the local optimum. monitor stagntion The evaluation criteria for whether the population evolution occurs are whether the objective function score of the global optimal particle in the PSO population is improved or the objective function remains the same and the number of features decreases. If, in a round of population iteration, the objective function score of a new particle in the HybridFeature_pop or PureACC_pop population is improved compared with the global optimal particle or the objective function remains the same as the global optimal particle and the number of features decreases, it is considered that the HybridFeature_pop or PureACC_pop population is optimized in this round, and the monitor of the population is stagntion Perform the clear operation, otherwise, the monitor of the population stagntion Perform accumulation operations to record the optimization stagnation of this round. monitor stagntion =0, monitor stagntion +=1, Among them, monitor stagntion Represents the population stagnation detector of HybridFeature_pop or PureACC_pop. When the monitor of HybridFeature_pop or PureACC_pop stagntion If the value of is greater than the predetermined threshold qz, it is considered that HybridFeature_pop or PureACC_pop is trapped in the local optimum, and the population immediately initiates an information exchange request to PureACC_pop or HybridFeature_pop, executes the dual elite fusion strategy, and updates some particles of the population to escape the local optimum.

10. The hybrid feature selection method of the improved particle swarm algorithm according to claim 9, characterized in that: The dual elite fusion strategy based on population stagnation detector specifically includes the following steps: According to the objective function of the population HybridFeature_pop or PureACC_pop that is trapped in the local optimum, obtain the particle dp with the best objective function score among the existing particles in HybridFeature_pop or PureACC_pop best , suboptimal dp sbest and worst dp worse , and the best particle gb recorded in the population best and the worst particle gw worse ; Using the same objective function, obtain the particle dp with the best objective function score among the existing particles from PureACC_pop or HybridFeature_pop best ′, suboptimal dp sbest ′ and worst dp worse ′, and the best particle gb recorded in the population best ′ and the worst particle gw worse ′; The three high-quality particles dp obtained from the population HybridFeature_pop or PureACC_pop that are trapped in the local optimum best , dp sbest and gb best With 3 high-quality particles dp obtained from PureACC_pop or HybridFeature_pop best ′, dp sbest ′ and gb best 'Forming a combination of 6 high-quality particles; secondly, 3 high-quality particles dp best , dp sbest and gb best Each particle will be hybridized with the other five particles in the high-quality particle combination except for itself. After hybridization, each particle will generate five new particles. Then, the particle with the best objective function score is selected from the five new particles, so three new particles are finally obtained. and Two high quality particles dp i and dp j During hybridization, in order to further retain the genes of high-quality particles, the new particles formed by hybridization are dp new Each gene of is determined according to the ratio of the inverse objective function score, Among them, Finess i and Finess j dp respectively i and dp j The score calculated based on the objective function of the population trapped in the local optimum, r is a random number in the interval [0,1]; First, the two inferior particles dp obtained from the population HybridFeature_pop or PureACC_pop that are trapped in the local optimum are worse and gw worse With 2 inferior particles dp obtained from PureACC_pop or HybridFeature_pop worse ′ and gw worse 'Forming a combination of 4 inferior particles; Secondly, 2 inferior particles dp worse and gw worse Hybridize with the three particles in the inferior particle combination except for the particle itself. After hybridization, each particle will generate three new particles. Then, the particle with the worst objective function score is selected from the three new particles, so two new particles are finally obtained. and Two inferior particles dp i and dp j During hybridization, in order to further retain the genes of inferior particles, the new particles formed by hybridization are dp new Each gene of is determined according to the ratio of the objective function score. Among them, Finess i and Finess j dp respectively i and dp j The score calculated based on the objective function of the population trapped in the local optimum; Three high-quality particles and two low-quality particles will be formed. In the population HybridFeature_pop or PureACC_pop that falls into the local optimum, five particles are randomly selected except the particles used in the hybridization process and replaced with the five new particles formed by hybridization. stagntion The value is cleared to zero.

Images