Dynamic compression and expansion feature selection algorithm for double-population double-measurement three-archive based on PSO (Particle Swarm Optimization)

Abstract

The invention discloses a feature selection algorithm for dynamic compression and expansion of double-population double-measurement three archives based on PSO, and the algorithm comprises the following steps: employing different measurement methods for two populations through a double-population double-measurement filtering strategy, and preliminarily completing the dimension reduction of high-dimensional data; performing preliminary filtering on features of the population Subs and the population SubM by adopting different measurement methods so as to obtain a high-quality initial solution; sU measurement is adopted for the population Subs, and MIC measurement is adopted for the population SubM; based on a cluster imbalance group sampling strategy, utilizing different measurement methods to respectively cluster and group the two populations, further reducing the data dimension and completing population sampling; the group sampling strategy based on cluster imbalance comprises a rapid clustering strategy, a self-adaptive division strategy and a group sampling strategy; and selecting an optimal feature subset in the reduced search space through a double-population three-file evolutionary strategy. And the search flexibility and coverage are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of feature selection, and in particular to a feature selection algorithm of dynamic compression and expansion of dual-population, dual-metric and three-file based on PSO. Background Art

[0002] With the rapid development of science and technology, massive high-dimensional data (HD) has been widely used in many fields such as bioinformatics, image processing and network security. Although high-dimensional data provides rich opportunities for more accurate and in-depth analysis, it also brings significant challenges, such as increased computational complexity, model overfitting and prolonged training time. Irrelevant or redundant features not only increase the burden on computing resources, but also may reduce the performance of machine learning algorithms. Feature selection (FS), as an effective dimensionality reduction technique, aims to extract representative feature subsets from the original feature set, thereby alleviating the "curse of dimensionality", improving model performance and retaining the interpretability of features. Common FS methods can be roughly divided into three categories: 1) filter methods; 2) embedded methods; 3) wrapper methods. Filter methods evaluate the importance of features based on the intrinsic properties of the data, independent of the specific learning algorithm. Embedded methods automatically perform feature selection during model training, embedding the feature selection process into the model, and taking into account both efficiency and performance. Wrapper methods regard FS as a search problem, and evaluate the pros and cons of feature subsets through specific classifiers during the search process. Wrapper methods usually show better performance. To further improve the FS effect, researchers have proposed a hybrid FS method, which combines the advantages of different types of FS methods and applies different FS methods at different stages to achieve efficient dimensionality reduction. Existing hybrid methods are mainly divided into two categories: filter-wrapper hybrid methods and embedded-wrapper hybrid methods. These methods usually adopt a two-stage strategy: the first stage reduces the search space by removing irrelevant or redundant features, and the second stage seeks the best feature subset in the reduced search space. Compared with the traditional three-category FS methods, hybrid FS methods can usually achieve better results. Traditional evolutionary algorithms (EC), such as ant colony optimization, gray wolf optimization, and genetic algorithms, have been proven to be able to effectively screen out feature subsets with high classification accuracy due to their excellent search capabilities. In recent years, many researchers have applied EC with global optimization capabilities to the field of FS. Among them, the particle swarm optimization algorithm (PSO) has been widely used to solve various feature selection problems due to its simple concept, convenient implementation, strong global search capability, and fast convergence speed.

[0003] In hybrid FS methods, a common approach to reducing the search space is to use a single correlation measure to measure the importance of features. However, most studies have not paid attention to the complexity within HD and the differences in information captured by different correlation measures. Different correlation measures have different focuses, and this difference can lead to significantly different effects on the distribution of features in the final search space. In addition, if only a fixed threshold is used to filter out features with low correlation, the dimensionality reduction effect obtained is very limited. Although the search space can be further reduced by clustering methods, common clustering algorithms need to calculate the correlation between all features, which often brings extremely high computational costs. In recent years, some researchers have used approximate clustering methods (such as FCFC) to reduce the search space, which significantly reduces the computational overhead. However, many studies usually randomly select features in feature clusters when sampling, ignoring the approximate clustering characteristics of FCFC and failing to fully consider the differences between features in the same cluster. Most current clustering-based wrapper methods usually cover all feature clusters in the initial sampling stage, and do not conduct in-depth evaluation of the importance of different clusters. In addition, when setting the subpopulation size, the common multi-population FS algorithm usually does not consider the impact of the size of the search space and the differences in its feature composition on the convergence speed. It simply sets the subpopulation size to the same, failing to fully tap the potential of each subpopulation.

[0004] Although PSO and its variants have made significant progress in the field of FS, they still face great challenges when applied to HD. In particular, when the search range is large, PSO is prone to fall into local optimality. How to effectively monitor the evolutionary state and jump out of the local optimal trap in time has become a key problem in the application of PSO to HD. In addition, as a wrapper method, PSO needs to frequently evaluate particle performance during the search process. Improving the effective utilization of evaluation results is also an important issue that needs to be solved in FS applications. Summary of the invention

[0005] The purpose of the present invention is to provide a dynamic compression and expansion feature selection algorithm of dual-population dual-metric three-file based on PSO in response to the above-mentioned problems. By applying different metric methods to two populations, the dimensionality reduction of high-dimensional data is preliminarily completed. Based on the group sampling strategy of cluster imbalance, different metric methods are used to cluster and group the two populations respectively to further reduce the data dimension and complete population sampling. Finally, a dual-population three-file evolution strategy is proposed to select the optimal feature subset in the reduced search space.

[0006] The technical solution of the present invention is as follows:

[0007] A dynamic compression and expansion feature selection algorithm based on PSO with dual populations, dual metrics and three archives includes the following steps:

[0008] Through the dual-population dual-metric filtering strategy, different metric methods are applied to the two populations respectively to preliminarily complete the dimensionality reduction of high-dimensional data; s and population Sub M Different metric methods are used to preliminarily filter the features to obtain high-quality initial solutions; s Using SU metric, the population Sub M Using MIC measurement;

[0009] The group sampling strategy based on cluster imbalance uses different measurement methods to cluster and group the two populations respectively, further reducing the data dimension and completing population sampling; the group sampling strategy based on cluster imbalance includes fast clustering strategy, adaptive partitioning strategy and group sampling strategy;

[0010] Through the dual-population three-file evolution strategy, the optimal feature subset is selected in the reduced search space; the dual-population three-file evolution strategy includes dual-population PSO evolution process, three-file assisted evolution and compression-expansion mechanism.

[0011] Furthermore, the population Sub s The SU measurement method is adopted, and the SU measurement includes the following steps:

[0012] The calculation formula of SU is:

[0013] Among them, X and Y are two variables to be measured, MI(X,Y) represents the mutual information between X and Y, H(X) and H(Y) are the entropies of X and Y respectively; the calculation formula of MI(X,Y) is:

[0014]

[0015] Where p(x,y) is the joint probability distribution of X and Y, and p(x) and p(y) are their marginal probability distributions respectively; the calculation formulas for H(X) and H(Y) are as follows:

[0016] H(X)=-∑ x∈X p(x)log2p(x),

[0017] H(Y)=-∑ y∈Y p(y)log2p(y),

[0018] Among them, p(x) and p(y) are their marginal probability distributions respectively.

[0019] Furthermore, the population Sub M The MIC measurement method is adopted. The MIC measurement includes the following steps:

[0020] The calculation formula of MIC is:

[0021]

[0022] Among them, X and Y are two variables to be measured, MI(X,Y) represents the mutual information between X and Y, m and n represent the number of grid divisions in the X and Y directions respectively, and min(n,m) represents the smaller value in the two directions.

[0023] Furthermore, the preliminary completion of dimensionality reduction of high-dimensional data also includes the following steps:

[0024] Sub s and population Sub M The filtering threshold is determined by the following function:

[0025] ρ su =θ su ·SU max ,

[0026] ρ mic =θ mic MIC max ,

[0027] Among them, θ su and θ mic is the correlation coefficient, SU max and MIC max are the maximum SU value and MIC value between all features and class labels respectively;

[0028] After determining the filtering threshold, the original data set S has D features, denoted as F = {f1,f2,f3,…,f D}, First, Sub s Calculate each feature f i The SU value between the class label and the filter threshold ρ is calculated according to the corresponding threshold formula su , then, Sub M Calculate each feature f i The MIC value between the class label and the filter threshold ρ is calculated mic Finally, Sub s and Sub M Remove SU values ​​lower than ρ su and MIC values ​​lower than ρ mic features to complete the initial dimensionality reduction.

[0029] Furthermore, the fast clustering strategy includes the following steps:

[0030] Using the FCFC framework to s Fast clustering of features in ;

[0031] The correlation measurement method in the FCFC framework is replaced by MIC for Sub M Fast clustering of features in ;

[0032] Get two sets of ordered feature clusters s and clusters M , including D s and D M feature clusters.

[0033] Furthermore, the adaptive partitioning strategy includes the following steps:

[0034] The size of different sub-populations is determined by the sub-population adaptive strategy. The specific formula is:

[0035]

[0036] N M =NN S ,

[0037] Among them, N is the total size of the two populations, D s and D M Clusters s and clusters M The length, N S and N M Represents Sub s and Sub m The size of both populations;

[0038] According to D s With D M The relative size is dynamically adjusted. The detailed process is shown in the following formula:

[0039]

[0040] Further, the group adoption strategy includes the following steps:

[0041] According to the order of the feature clusters, clusters s and clusters M The feature clusters in are grouped separately, and finally we get and

[0042] The number of feature clusters within each group is determined by the following formula:

[0043]

[0044] Where x represents the group number, D′ is the total number of feature clusters; Sub s and SubM Initially, only and sampling;

[0045] Sun s The sampling process includes: The feature clusters in are divided into different levels according to the number of features: the first level is composed of feature clusters of a single feature, and the second level is composed of feature clusters of multiple features; then the number of features selected from the two levels is dynamically determined, as shown in the following formula:

[0046]

[0047] size2=size-size1,

[0048] Where size is the preset length of the particle, D1 and D2 are the lengths of level1 and level2 respectively, size1 and size2 are the number of features selected from level1 and level2 respectively; Sub s It is composed of two subpopulations, sub1 and sub2, which are spliced ​​horizontally;

[0049] sub1: Before sampling, set the dynamically adaptive JS coefficient threshold ρ js ; All solutions in sub1 are obtained by randomly sampling size1 features from level1; except for the first solution, the remaining solutions need to calculate their maximum similarity with the solutions stored in sub1 before storing them in sub1; if the maximum similarity is less than or equal to ρ js , then store it in sub1; otherwise, regenerate; ρ js The initial settings are as follows:

[0050]

[0051] Among them, size1 is the number of features selected from level1, and D1 is the length of level1;

[0052] If the condition is not met after five consecutive attempts, it means that ρ js If it is too small, dynamically adjust ρ js as follows:

[0053] ρ js =ρ js +0.1*ρ js ,

[0054] The JS coefficient is used to measure the correlation between solutions; the calculation formula of the JS coefficient is as follows:

[0055]

[0056] Among them, set1 and set2 are both sets;

[0057] sub2: Randomly select size2 feature clusters from level2, randomly select a feature in each selected feature cluster, and find the feature with the lowest correlation with it in the cluster as the complementary feature. The correlation measurement method is consistent with the method used in population filtering and clustering.

[0058] Furthermore, the dual-population PSO evolution process includes the following steps:

[0059] The quality of particles is evaluated based on the weighted average of classification balance accuracy BACC and Fisher discriminant distance FDD. The fitness function is:

[0060] fitness=w*BACC+(1-w)*FDD,

[0061] Among them, w is the weight coefficient, the value range is [0,1], BACC represents the classification balance accuracy, and the calculation formula is:

[0062]

[0063] Where c is the total number of categories, TP i is the number of true positive samples of the i-th label, N i is the total number of samples in the i-th class; FDD is based on the inter-class distance D B and intra-class distance D w Calculate the Fisher discriminant distance, the expression is:

[0064]

[0065] Among them, D B is the average distance from the sample to the nearest sample of different categories, D w is the average distance from a sample to the farthest sample of the same category.

[0066] Furthermore, the three-file assisted evolution includes the following steps:

[0067] The three files include two local files Ach S and Ach M And a global file Ach G Among them, Ach S and Ach M Used to record Sub s and Sub M The importance of each feature in the respective search scope; Ach G Then Ach S and Ach M, records the global feature importance of the two populations; initially, the feature importance recorded in all archives is 0;

[0068] Through the change of fitness value before and after the particle position is updated, the importance of new or missing features in the particle position is reversely updated; Ach S The update process includes: s In the equation, each particle dp i Evolved into dp through location updates i ′, resulting in the fitness value from fitness i Become fitness i ′; and dp i In comparison, dp i ′ contains the newly added feature set F a and the deleted feature set F d ; For Ach S If the features in F a , its feature importance imp m Update as follows:

[0069] imp m =fitness i ′-fitness i ;

[0070] For the existence of F d , its feature importance imp n Update as follows:

[0071] imp n =fitness i -fitness i ′;

[0072] Local file auxiliary strategy: Ach S and Ach M By recording the importance of the features of their respective populations during the evolution process, it helps particles escape faster when they are trapped in the local optimum; dp i The escape process includes:

[0073] According to Ach S According to the importance of features in , the features are divided into supporting features and opposing features, forming supporting feature set S and opposing feature set O respectively; according to dp i The stagnation time determines the number of mutated features num, and the formula is:

[0074] num=min(max(tP k ,1),5),

[0075] Where t is dpi The number of consecutive stagnations; loop num times, each time with a probability of 0.5 from dp i Randomly delete a feature in O, or randomly add a feature from S that is not in dp i Features in;

[0076] Global archive auxiliary strategy: When one of the populations stops PSO search but has not reached the set number of iterations, and the other population is still searching normally, use Ach G Stop searching for each particle in the population dp i Optimize; according to Ach G The feature importance recorded in , delete dp i The least important feature in the equation; the specific process is as follows:

[0077] Extraction of Ach G All features whose importance is less than 0 are sorted in ascending order of importance and stored in the anti-feature set O g ; delete dp i In O g The top-ranked feature in dp; if the fitness decreases after deletion, restore the feature; otherwise, keep the new particle to replace the original dp i .

[0078] Furthermore, the compression-expansion mechanism includes two parts, SE-1 and SE-2, which dynamically adjust particles and search space, specifically including the following steps:

[0079] SE-1 mechanism: When the population evolution falls into a local optimum, it is preliminarily speculated that the particle length may be set too small, so the particle length needs to be dynamically expanded; in the process of expanding the particle length, each particle in the population is compressed using the S-1 strategy, and then the compressed particles are expanded using the E-1 strategy;

[0080] S-1 strategy: dp through local filings i The feature with the lowest feature importance and importance less than 0 is deleted; if the fitness of the particle does not decrease after deletion, the original dp is replaced with a new particle i ; Otherwise, abandon the deletion operation;

[0081] E-1 strategy: Determine the length that the particles in the population need to expand; calculate the particle expansion length using the following formula:

[0082]

[0083]

[0084] Where D represents the data set, c is the number of categories in the data set, and pi is the proportion of samples of the i-th category to the total samples, Ent(D) is the entropy of the data set; fitness gb is the global optimal fitness value of the population; the particle expansion strategy of the population is based on the group sampling strategy, and the following changes are made to it:

[0085] Modification of sampling level: The sampling level is adjusted from level 1 and level 2 to dp i The complements of these two layers are denoted as patches. i ′ and patches″ i ;

[0086] ρ js Initial setting and modification of supervision object: ρ js The initial formula is changed to:

[0087]

[0088] Among them, dp′1 represents the first particle in patches i ′ is the union of the sampled features and the features of level 1 in the original particle; ρ js The supervision object is adjusted from the original sampling feature set to the expanded feature set belonging to level 1;

[0089] Complementary particle generation strategy modification: when expanding, dp i+1 It is necessary to screen the complementary feature set and remove duplicate features; if dp i+1 If the number of features to be sampled at the second level does not exceed the number of complementary features after screening, they are directly selected; otherwise, the remaining features need to be randomly sampled;

[0090] SE-2 mechanism: If the population still cannot continue to evolve after executing SE-1, it is inferred that the reason why the population falls into the local optimum is not that the particle length is too small, but that the search algorithm has completed the search of the current search space; use the S-2 strategy to compress the current population, and then use the E-2 strategy to expand the search space first and then expand the population;

[0091] S-2 strategy: cancel the previous E-1 population expansion operation and restore the population to the state before the E-1 expansion;

[0092] E-2 strategy: expand the search space and add the next group to the current search space; if all groups have been included, abandon the execution of the E-2 strategy; expand the population based on the new search space according to the E-1 expansion strategy.

[0093] Compared with the prior art, the present invention has the following beneficial effects:

[0094] 1. A dynamic compression and expansion feature selection algorithm based on PSO with two populations, two metrics and three files, proposes a two-population and two-metric filtering strategy, which applies different correlation measurement methods in two populations to more comprehensively capture the data relationship in HD and complete the initial dimension reduction. This strategy not only fully extracts the data characteristics in HD during the dimension reduction process, but also provides differentiated search spaces for different populations, thereby improving the diversity of understanding;

[0095] 2. A dynamic compression and expansion feature selection algorithm based on PSO with two populations, two metrics and three archives is proposed, and a group sampling strategy based on cluster imbalance is proposed. By combining the fast clustering strategy with two correlation metrics, the search space of the two populations is further narrowed. In view of the difference in the size of the search space of the two populations, an adaptive partitioning strategy is proposed to dynamically determine the size of each sub-population. In view of the fact that the fast clustering strategy belongs to approximate clustering, in order to further improve the effectiveness of sampling, a group sampling strategy is proposed, which fully considers the importance of different clusters through grouping, and takes into account the diversity of the initial solution and the difference of the characteristics within the cluster in stratified sampling;

[0096] 3. A dynamic compression and expansion feature selection algorithm based on PSO's dual population, dual metric and three archives. A dual population and three archives evolution strategy based on PSO is proposed. The information obtained by evaluating particles in the two populations during the evolution process is used to design two local archives to record the excellent features of each population during the evolution process. When particles in a population fall into a local optimum, they can quickly jump out of the local optimum through the corresponding local archives. In addition, a global archive is generated by integrating two local archives to further optimize the converged population;

[0097] 4. A dynamic compression and expansion feature selection algorithm based on PSO with two populations, two metrics and three archives. Considering that the population only groups the most important feature clusters as search space sampling at the beginning, a compression-expansion mechanism is proposed to effectively improve the flexibility and coverage of the search by dynamically adjusting the length of particles in the population and the search range. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 This is the overall framework diagram of the algorithm of this application.

[0099] Figure 2 Schematic diagram of the imbalance of features within the feature cluster after approximate clustering.

[0100] Figure 3 This is a fitness convergence diagram. DETAILED DESCRIPTION

[0101] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0102] The features and performance of the present invention are further described in detail below in conjunction with the embodiments.

[0103] See also Figure 1-3 , a dynamic compression and expansion feature selection algorithm based on PSO with dual populations, dual metrics and three archives, such as Figure 1 As shown, the following steps are included:

[0104] Through the dual-population dual-metric filtering strategy, different metric methods are applied to the two populations respectively, and the dimensionality reduction of high-dimensional data is preliminarily completed; SU (Sub s ) and subpopulation MIC (Sub M ) use different metric methods to filter the features preliminarily to obtain high-quality initial solutions; s Using the Symmetricaluncertainty (SU) metric, the population Sub M The maximal information coefficient (MIC) is used for measurement;

[0105] The group sampling strategy based on cluster imbalance uses different measurement methods to cluster and group the two populations respectively, further reducing the data dimension and completing population sampling; the group sampling strategy based on cluster imbalance includes fast clustering strategy, adaptive partitioning strategy and group sampling strategy;

[0106] Through the dual-population three-file evolution strategy, the optimal feature subset is selected in the reduced search space; the dual-population three-file evolution strategy includes dual-population PSO evolution process, three-file assisted evolution and compression-expansion mechanism.

[0107] Sub sUsing the SU measurement method, SU can effectively describe the nonlinear relationship between random variables. As a standardized form of MI, it can eliminate the influence of the entropy size between variables, making it fairer when comparing the interdependence between different pairs of variables. The SU measurement includes the following steps:

[0108] The calculation formula of SU is:

[0109] Among them, X and Y are two variables to be measured, MI(X,Y) represents the mutual information between X and Y, H(X) and H(Y) are the entropies of X and Y respectively; the calculation formula of MI(X,Y) is:

[0110]

[0111] Where p(x,y) is the joint probability distribution of X and Y, and p(x) and p(y) are their marginal probability distributions respectively; the calculation formulas for H(X) and H(Y) are as follows:

[0112] H(X)=-∑ x∈X p(x)log2p(x),

[0113] H(Y)=-∑ y∈Y p(y)log2p(y),

[0114] Among them, p(x) and p(y) are their marginal probability distributions respectively.

[0115] Sub M Using the MIC metric, MIC can quantify the correlation between features and between features and class labels, whether the relationship is linear or nonlinear. Compared with other correlation metrics, MIC has advantages in capturing complex dependencies. The MIC metric includes the following steps:

[0116] The calculation formula of MIC is:

[0117]

[0118] Among them, X and Y are two variables to be measured, MI(X,Y) represents the mutual information between X and Y, m and n represent the number of grid divisions in the X and Y directions respectively, and min(n,m) represents the smaller value in the two directions. The logarithm is taken to standardize the mutual information to avoid the deviation of the MIC value due to the different number of grid divisions and make it more comparable.

[0119] The initial dimensionality reduction of high-dimensional data also includes the following steps:

[0120] Sub s and population SubM The filtering threshold is determined by the following function:

[0121] ρ su =θ su ·SU max ,

[0122] ρ mic =θ mic MIC max ,

[0123] Among them, θ su and θ mic is the correlation coefficient, SU max and MIC max are the maximum SU value and MIC value between all features and class labels respectively;

[0124] After determining the filtering threshold, the original data set S has D features, denoted as F = {f1,f2,f3,…,f D}, First, Sub s Calculate each feature f i The SU value between the class label and the filter threshold ρ is calculated according to the corresponding threshold formula su , then, Sub M Calculate each feature fi i The MIC value between the class label and the filter threshold ρ is calculated mic Finally, Sub s and Sub M Remove SU values ​​lower than ρ su and MIC values ​​lower than ρ mic features to complete the initial dimensionality reduction.

[0125] The dual-population dual-metric filtering strategy can quickly achieve dimensionality reduction requirements by removing low-correlation features or noise. However, this strategy only considers the correlation between features and class labels, and does not involve redundancy between features, which may lead to significant redundancy between the remaining features. s and Sub M The features in are clustered separately.

[0126] The fast clustering strategy includes the following steps:

[0127] Using the FCFC framework to s Fast clustering of features in ;

[0128] The correlation measurement method in the FCFC framework is replaced by MIC for Sub M Fast clustering of features in ;

[0129] Get two sets of ordered feature clusters s and clusters M , including D s and D M feature clusters.

[0130] The adaptive partitioning strategy includes the following steps:

[0131] The size of different sub-populations is determined by the sub-population adaptive strategy. The specific formula is:

[0132]

[0133] N M =NN S ,

[0134] Among them, N is the total size of the two populations, D s and D M Clusters s and clusters M The length, N S and N M Represents Sub s and Sub m The size of both populations;

[0135] According to D s With D M The relative size is dynamically adjusted. The detailed process is shown in the following formula:

[0136]

[0137] The group adoption strategy consists of the following steps:

[0138] According to the order of the feature clusters, clusters s and clusters M The feature clusters in are grouped separately, and finally we get and This grouping strategy not only fully considers the importance of feature clusters, enabling the evolutionary algorithm to prioritize feature clusters with higher importance, but also further compresses the search space, thereby improving the overall search efficiency.

[0139] The number of feature clusters within each group is determined by the following formula:

[0140]

[0141] Where x represents the group number, D′ is the total number of feature clusters; Sub s and Sub M Initially, only and sampling;

[0142] Existing FS algorithms based on the FCFC framework usually randomly sample from feature clusters during sampling, ignoring the approximate clustering nature of FCFC and failing to recognize the differences between features in the same cluster (e.g. Figure 2 ), thus affecting the quality of the initial population and the final solution.

[0143] Sub s The sampling process includes: The feature clusters in are divided into different levels according to the number of features: the first level (level 1) is composed of feature clusters of a single feature, and the second level (level 2) is composed of feature clusters of multiple features; then the number of features selected from the two levels is dynamically determined, as shown in the following formula:

[0144]

[0145] size2=size-size1,

[0146] Where size is the preset length of the particle, D1 and D2 are the lengths of level1 and level2 respectively, size1 and size2 are the number of features selected from level1 and level2 respectively; Sub s It is composed of two subpopulations, sub1 and sub2, which are spliced ​​horizontally;

[0147] sub1: Before sampling, set the dynamically adaptive JS coefficient threshold ρ js ; All solutions in sub1 are obtained by randomly sampling size1 features from level1; except for the first solution, the remaining solutions need to calculate their maximum similarity with the solutions stored in sub1 before storing them in sub1; if the maximum similarity is less than or equal to ρ js , then store it in sub1; otherwise, regenerate; ρ js The initial settings are as follows:

[0148]

[0149] Among them, size1 is the number of features selected from level1, and D1 is the length of level1;

[0150] If the condition is not met after five consecutive attempts, it means that ρ js If it is too small, dynamically adjust ρ js as follows:

[0151] ρ js =ρ js +0.1 / ρ js ,

[0152] The JS coefficient is used to measure the correlation between solutions; the calculation formula of the JS coefficient is as follows:

[0153]

[0154] Among them, set1 and set2 are both sets;

[0155] sub2: Randomly select size2 feature clusters from level2, randomly select a feature in each selected feature cluster, and find the feature with the lowest correlation with it in the cluster as the complementary feature. The correlation measurement method is consistent with the method used in population filtering and clustering.

[0156] The dual-population PSO evolution process includes the following steps:

[0157] The quality of particles is evaluated based on the weighted average of classification balance accuracy BACC and Fisher discriminant distance FDD. The fitness function is:

[0158] fitness=w*BACC+(1-w)*FDD,

[0159] Among them, w is the weight coefficient, the value range is [0,1], BACC represents the classification balance accuracy, and the calculation formula is:

[0160]

[0161] Where c is the total number of categories, TP i is the number of true positive samples of the i-th label, N i is the total number of samples in the i-th class; FDD is based on the inter-class distance D B and intra-class distance D w Calculate the Fisher discriminant distance, the expression is:

[0162]

[0163] Among them, D B is the average distance from the sample to the nearest sample of different categories, D w is the average distance from a sample to the farthest sample of the same category.

[0164] In order to simplify the representation of particles and avoid the extra conversion overhead caused by traditional binary encoding, the feature index set is directly used to represent particles, which is dp = {i,…,j,..}, where i and j represent the feature f respectively. i and f j Due to the change in particle representation, the particle position update strategy of traditional PSO is no longer applicable. Therefore, an explicit update strategy is adopted to update Sub s and SubM of particles.

[0165] For ease of explanation, the following is Sub s Take particle dp as an example s The update process. First, dp s To its local optimal solution pb s Close, keep and pb s The characteristics of the intersection. For dp s Relative to pb s The difference set is matched to pb one by one according to probability θ1. s Relative to dp s After the match is completed, dp s Continue to Sub s The global optimal solution gb s Close, strategy and close to pb s same.

[0166] In addition to the improved update strategy, considering Sub s and Sub M The search range of is gradually expanded with the iteration, and the relative importance of the newly added features is necessarily lower than the features in the original search range. A further optimization is proposed: the constraint of the number of consecutive stagnations of population evolution is introduced. s and Sub M The upper limit of the set number of iterations is reached, or the global optimal solution is found in the continuous G k If there is no change in the iteration, the PSO search of the population is stopped immediately. This strategy can effectively reduce unnecessary searches and save computing resources.

[0167] The three-file assisted evolution includes the following steps:

[0168] The three files include two local files Ach S and Ach M And a global file Ach G Among them, Ach S and Ach M Used to record Sub s and Sub M The importance of each feature in the respective search scope; Ach G Then Ach S and Ach M , records the global feature importance of the two populations; initially, the feature importance recorded in all archives is 0;

[0169] In order to avoid adding extra computational costs, the importance of new or missing features in the particle position is reversely updated through the change in the fitness value before and after the particle position is updated; S The update process includes: s In the equation, each particle dp i Evolved into dp through location updates i ′, resulting in the fitness value from fitness i Become fitness i ′; and dp i In comparison, dp i ′ contains the newly added feature set F a and the deleted feature set F d ; For Ach S If the features in F a , its feature importance imp m Update as follows:

[0170] imp m =fitness i ′-fitness i ;

[0171] For the existence of F d , its feature importance imp n Update as follows:

[0172] imp n = fiteness i -fitness i ′;

[0173] Local file auxiliary strategy: Ach S and Ach M By recording the importance of the characteristics of their respective populations during the evolution process, it helps particles to escape faster when they are trapped in the local optimum; specifically, when Sub s A particle dp i The number of consecutive stagnations of the local optimum exceeds the set threshold P k When it is considered to fall into the local optimum, at this time, using Ach S The information recorded in dp is deleted with a certain probability i Remove unimportant features or add important features to help it escape. i The escape process includes:

[0174] According to Ach S According to the importance of features in , the features are divided into supporting features and opposing features, forming supporting feature set S and opposing feature set O respectively; according to dp iThe stagnation time determines the number of mutated features num, and the formula is:

[0175] num=min(max(tP k ,1),5),

[0176] Where t is dp i The number of consecutive stagnations; loop num times, each time with a probability of 0.5 from dp i Randomly delete a feature in O, or randomly add a feature from S that is not in dp i Features in;

[0177] Global archive auxiliary strategy: When one of the populations stops PSO search but has not reached the set number of iterations, and the other population is still searching normally, use Ach G Stop searching for each particle in the population dp i Optimize; according to Ach G The feature importance recorded in , delete dp i The least important feature in the equation; the specific process is as follows:

[0178] Extraction of Ach G All features whose importance is less than 0 are sorted in ascending order of importance and stored in the anti-feature set O g ; delete dp i In O g The top-ranked feature in dp; if the fitness decreases after deletion, restore the feature; otherwise, keep the new particle to replace the original dp i .

[0179] When PSO performs a search task, the length of the particle has a crucial impact on its search effect. If the particle length is set too small, PSO will not be able to fully explore the entire search space, resulting in unsatisfactory search results. s and Sub M The preset particle length is used for both, and only the first group of its characteristic cluster is sampled.

[0180] The compression-expansion mechanism includes two parts, SE-1 and SE-2, which dynamically adjust particles and search space. Specifically, it includes the following steps:

[0181] SE-1 mechanism: When the population evolution falls into a local optimum (i.e., the global optimum of the population stagnates for k1 consecutive times), it is preliminarily speculated that the particle length may be set too small, so the particle length needs to be dynamically expanded; in the process of expanding the particle length, each particle in the population is compressed using the S-1 strategy, and then the compressed particles are expanded using the E-1 strategy;

[0182] S-1 strategy: dp through local filings i The feature with the lowest feature importance and importance less than 0 is deleted; if the fitness of the particle does not decrease after deletion, the original dp is replaced with a new particle i ; Otherwise, abandon the deletion operation;

[0183] E-1 strategy: Determine the length that the particles in the population need to expand; calculate the particle expansion length using the following formula:

[0184]

[0185]

[0186] Where D represents the data set, c is the number of categories in the data set, and p i is the proportion of samples of the i-th category to the total samples, Ent(D) is the entropy of the data set; fitness gb is the global optimal fitness value of the population; the particle expansion strategy of the population is based on the group sampling strategy, and the following changes are made to it:

[0187] Modification of sampling level: The sampling level is adjusted from level 1 and level 2 to dp i The complements of these two layers are denoted as patches. i ′ and patches″ i ;

[0188] ρ js Initial setting and modification of supervision object: ρ js The initial formula is changed to:

[0189]

[0190] Among them, dp′1 represents the first particle in patches i ′ is the union of the sampled features and the features of level 1 in the original particle; ρ js The supervision object is adjusted from the original sampling feature set to the expanded feature set belonging to level 1;

[0191] Complementary particle generation strategy modification: when expanding, dp i+1 It is necessary to screen the complementary feature set and remove duplicate features; if dp i+1 If the number of features to be sampled at the second level does not exceed the number of complementary features after screening, they are directly selected; otherwise, the remaining features need to be randomly sampled;

[0192] SE-2 mechanism: If the population still cannot continue to evolve after executing SE-1 (i.e. the population global optimum stagnates for k2 consecutive times), it is inferred that the reason why the population falls into the local optimum is not that the particle length is too small, but that the search algorithm has completed the search of the current search space; use the S-2 strategy to compress the current population, and then use the E-2 strategy to expand the search space first and then expand the population;

[0193] S-2 strategy: cancel the previous E-1 population expansion operation and restore the population to the state before the E-1 expansion;

[0194] E-2 strategy: expand the search space and add the next group to the current search space; if all groups have been included, abandon the execution of the E-2 strategy; expand the population based on the new search space according to the E-1 expansion strategy.

[0195] Experimental verification:

[0196] Dataset:

[0197] We use 10 public high-dimensional medical datasets that have been studied in the field of feature selection to conduct experiments to verify the effectiveness of the algorithm in this application. The detailed information of the datasets is shown in Table 1, which lists the abbreviations, number of features, total number of instances, and number of label categories of each dataset in detail. Among them, the number of features ranges from 2308 to 12600, the total number of instances ranges from 50 to 203, and the label categories range from 2 to 11.

[0198] Table 1 Dataset details table

[0199]

[0200] Evaluation setting, indicators and procedures:

[0201] 1) Foundation: In order to verify the performance of the method in this application, a total of 6 methods were used for comparison, including 4 state-of-the-art FS methods, 1 traditional FS method and 1 original feature set comparison method, as shown below:

[0202] ESAPSO, a hybrid filter-wrapper FS method based on PSO, adopts explicit representation and size adaptation strategy.

[0203] MOP-FCPSO, a discretized feature selection framework based on multi-objective optimization, utilizes flexible cut point PSO and Pareto integration method to find relevant feature subsets.

[0204] VLPSO, a FS method, introduces variable-length representation in PSO, reduces the search space, and improves the performance on high-dimensional data.

[0205] PSO-FS, a PSO-based FS method, is used to represent feature subsets and their discretization cut points, and adopts a new fitness function, which comprehensively considers classification accuracy and subset compactness, and combines discretization and feature selection in one stage.

[0206] LFS, a wrapper-based FS method, utilizes linear forward selection to reduce attribute expansion and prevent overfitting.

[0207] Full, original dataset, without any FS or dimensionality reduction processing.

[0208] Since three multi-objective optimization algorithms, namely MOEA / D-FCPSO, NSGA-FCPSO and SPEA-FCPSO, are proposed in the MOP-FCPSO method, but since the performance of these three methods are similar, only MOEA / D-FCPSO is selected as the benchmark method and FCPSO is used as the abbreviation of this method.

[0209] 2) Indicators: Based on the essential requirements of FS, this application evaluates the performance of the algorithm from two aspects, namely, the classification performance of the selected feature subset and the effect of its dimensionality reduction.

[0210] In order to measure the classification performance of the feature subset selected by the algorithm, this application uses BACC to measure the classification performance of the feature subset. Since the PSO algorithm is random and needs to be run multiple times, the average balanced classification accuracy (Mean), the optimal balanced accuracy (Best) and the standard deviation (std) are used to comprehensively measure the classification performance of the feature subset. In addition, this application also uses a statistical T-test significance test with a significance level of 5% to verify whether there is a significant difference in the classification performance between Ousr and other algorithms.

[0211] In order to measure the dimensionality reduction effect of the algorithm, this application measures the length d of the final feature subset selected by each algorithm. * To evaluate the dimensionality reduction effect of the algorithm.

[0212] 3) Procedure: In order to reduce experimental bias and improve the generalization ability of the model, ten-fold cross validation (TFCV) is used to verify the proposed algorithm. That is, the entire data set is divided into ten parts. When evaluating the algorithm, each part is used as a test set in turn, and the remaining nine parts are combined as a training set. On the training set, the algorithm will identify the optimal feature subset and use these features to train the base classifier. Subsequently, the classifier is used to evaluate the selected optimal feature subset on the test set, and its BACC is calculated to evaluate the overall effect of the algorithm. Therefore, one TFCV can generate 10 BACCs, and the average of the 10 BACCs is used as the result of this run. It should be noted that during each fold run, the test set does not participate in any FS process, but is only used to evaluate the performance of the final feature subset selected by the algorithm.

[0213] Moreover, considering the randomness of PSO, each algorithm conducted 30 independent experiments on each data set. All computational experiments were run on Intel Core I5-10400F CPU@2.90GHz, and the code was implemented based on Python and sklearn.

[0214] Parameter settings:

[0215] K-NN is a base classifier widely used in wrapper-based FS methods. This application also uses the K-NN classifier to analyze the performance of this algorithm, and sets K=1. In order to ensure fairness, all algorithms are executed using the same classifier 1-NN.

[0216] ESAPSO, FSPSO, VLPSO and PSO-FS all use the parameters recommended by their authors, while LFS uses the parameters recommended by Weka.

[0217] Since there are many data sets used and the number of features between data sets varies greatly, the maximum number of iterations T recommended by the ESAPSO author depends on the linear relationship between the number of features, that is, T = |F| / 50, where |F| is the feature dimension of the data set.

[0218] Results and analysis:

[0219] The proposed algorithm (Ours) is compared with the 6 comparison algorithms on 10 datasets. Table 2 lists the classification performance (i.e., Best, Mean, and std) of each method on all datasets in detail. Table 3 lists the dimensionality reduction capability (d *) and the difference between Ours and the baseline method. In Table 3, “S” is the T-test result, where “+” means that the classification performance of Ours is significantly better than the comparison algorithm, “=” means that there is no significant difference between the classification performance of Ours and the comparison algorithm, and “-” means that the classification performance of Ours is significantly worse than the comparison algorithm. Combining Tables 2 and 3, the classification performance of Ours is significantly better than other baseline algorithms on most data sets.

[0220] Compared with ESAPSO: From Table 2, we can see that Ours achieves the best Mean compared with ESAPSO on all datasets. Combined with the T-test results in Table 3, we can conclude that Ours’ classification performance is significantly better than ESAPSO on all 9 datasets except Lung_Cancer. In addition, for the Lung_Cancer dataset, the classification performance of Ours algorithm has no obvious advantage over ESAPSO. The Mean achieved by Ours is slightly higher than that of ESAPSO by 0.51%, but the d obtained by Ours is * It is significantly lower than ESAPSO, with a decrease of 20.31%. The main reason for this phenomenon is that Lung_Cancer contains many redundant features. The ESAPSO algorithm only has filtering operations but no clustering operations. By combining these features, the classification performance can also be improved, but this method inevitably leads to a longer length of the selected feature subset.

[0221] Ours achieved d on 6 datasets except SRBCT, Leukemia1, Brain_Tumor1 and 9_Tumors. * For the four datasets mentioned above, Ours’s d * Although higher than ESAPSO, the Mean of Ours has been significantly improved compared with ESAPSO, and the improvement is most obvious on the 9_Tumors dataset, which is an increase of 4.44%. The improvement is the least on the SRBCT dataset, although it is only 0.47%, but it is an increase from 99.50% to 99.97%, which is very important. This is mainly because Ours and ESAPSO use different sources of features when expanding the population. ESAPSO selects feature expansion populations from the entire current search range, but Ours selects feature expansion from the complement of each particle. Although the expansion strategy adopted by ESAPSO controls the length of particles to a certain extent, it is easy to lead to poor expansion effect, thus affecting the classification effect of the final selected feature subset.

[0222] Compared with FCPSO: Combining Table 2 and Table 3, it can be seen that only on the 11_Tumors dataset, the classification performance of Ours is significantly worse than that of FCPSO, but the d obtained by Ours is* It is significantly lower than FCPSO, with a decrease of 69.78%. The main reason for this is that Ours performs large-scale filtering and clustering on features, which narrows the PSO search range and has a certain error. In addition, on the Leukemia2 dataset, the classification performance of Ours has no obvious advantage over FCPSO. The Mean obtained by Ours is slightly higher than that of the FCPSO algorithm by 0.07%, but the d obtained by Ours is * It is significantly lower than FCPSO, with a reduction of 47.76%. On all datasets except the two mentioned above, the classification performance of Ours is significantly better than FCPSO.

[0223] Ours achieved d on four datasets: SRBCT, DLBCL, Leukemia1 and 9_Tumors. * The main reason for the above phenomenon is that FCPSO is a multi-objective FS algorithm, while Ours simplifies the multi-objective problem into a single-objective problem. However, Ours controls the particle length by performing operations such as feature filtering and clustering, and its control effect is very prominent on the other six data sets. The d obtained by Ours is * In addition, in the above datasets, the classification performance of Ours is significantly better than that of FCPSO, and the Mean is improved most significantly on the 9_Tumors dataset, which is increased by 11.21%.

[0224] Compared with other methods: As shown in Table 3, Ours achieves significantly better classification performance than VLPSO, PSO-FS and LFS on all datasets. VLPSO controls the particle length by using a particle length change mechanism, and achieves d * The mean of Ours is 17.07% higher than that of Ours, but the mean of VLPSO is much lower than that of Ours, which is especially obvious on the Brain_Tumor2 dataset. The mean of Ours is 17.07% higher than that of VLPSO. The effect of Ours algorithm is more in line with the requirements of FS problem.

[0225] Subsequently, PSO-FS selected a large number of features on all datasets, and the d * All are higher than Ours. Moreover, the classification performance of PSO-FS on all datasets is significantly lower than Ours. This phenomenon shows that PSO-FS has very limited ability to deal with high-dimensional FS problems. It also reflects the necessity of filtering and clustering strategies in Ours algorithm to greatly reduce the search space.

[0226] In addition, for the LFS algorithm, the d *All of them are less than 15, indicating that its dimensionality reduction capability is very good. However, FS requires reducing the feature dimension while ensuring the classification performance. Compared with the Ours algorithm, the classification performance achieved by the LFS algorithm is too obvious, and the classification performance achieved is too low, which is most obvious in the Brain_Tumor2 dataset. The Mean achieved by Ours is 30.52% higher than that of LFS. The Mean achieved by Ours is 17.13% higher than that of the LFS algorithm on average.

[0227] Finally, compared with Full, Ours achieves significantly higher classification performance on all datasets. This phenomenon not only highlights the necessity of FS for high-dimensional data, but also highlights the correctness of Ours algorithm.

[0228] Table 2 Classification performance of seven algorithms

[0229]

[0230]

[0231]

[0232] Table 3 Dimensionality reduction capabilities and T-test results of the seven algorithms

[0233]

[0234]

[0235]

[0236] Convergence performance comparison: To further explore the performance of Ours algorithm in the search process, the convergence curves of the global optimal solution (gbest) fitness value of Ours and the other three algorithms (ESAPSO, VLPSO and standard PSO) are plotted during the search process. These four algorithms use the same fitness calculation function, the same population size (to ensure fairness, the total size of the dual population in Ours is consistent with that of other algorithms), and set the same number of iterations (100 times).

[0237] from Figure 3It can be clearly seen that Ours has shown strong search capabilities on most datasets, especially on datasets such as Leukemia2, 9_Tumors, Prostate, and Brain_Tumor2. In terms of initial fitness values, Ours outperforms other algorithms on all 9 datasets except 11_Tumors. This phenomenon not only directly shows that the initial solution generated by Ours algorithm is better than that of other algorithms, but also indirectly shows that the strategy of Ours to screen features by using two different metrics is reasonable and effective.

[0238] In terms of the final fitness value, the fitness values ​​of Ours on the five datasets are better than those of other algorithms. In the 11_Tumors, Prostate, DLBCL, and Lung_Cancer datasets, the fitness value of Ours is slightly lower than that of ESAPSO, but the difference is very small, only a small difference of three decimal places; in the Leukemia1 dataset, the fitness value of Ours is slightly lower than that of VLPSO, and the difference is also small, only 0.0011. This shows that although the Ours algorithm is divided into two populations, each of which is smaller than other algorithms, its final fitness value is still relatively good, further verifying the effectiveness of its search strategy.

[0239] Compared with ESAPSO, Ours showed faster convergence speed in all 9 datasets except 11_Tumors, reaching convergence earlier. In the 11_Tumors dataset, since Ours adopted a dual population strategy, the size of each population was smaller, which may lead to a slightly slower convergence speed, but in the end Ours almost caught up with the performance of ESAPSO.

[0240] Compared with VLPSO, Ours converges earlier in all six datasets except SRBCT, 9_Tumors, 11_Tumors, and Leukemia1. In SRBCT, 9_Tumors, and 11_Tumors, VLPSO converges faster because it falls into the local optimal solution too early, while in Leukemia1, the population particle length of Ours gradually expands, resulting in a slower convergence speed until the iteration is nearly completed before gradually approaching the performance of VLPSO.

[0241] Compared with the standard PSO, although PSO shows earlier convergence, the reason for its early convergence is that it falls into the local optimal solution too early, which limits the further evolution of the population. The comparison with PSO further highlights the effectiveness of Ours in avoiding local optimal solutions.

[0242] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A dynamic compression and expansion feature selection algorithm based on PSO with dual populations, dual metrics and three archives, characterized in that: The following steps are involved: Through the dual-population dual-metric filtering strategy, different metric methods are applied to the two populations respectively to preliminarily complete the dimensionality reduction of high-dimensional data; s and population Sub M Different metric methods are used to preliminarily filter the features to obtain high-quality initial solutions; s Using SU metric, the population Sub M Using MIC measurement; Based on the cluster imbalance group sampling strategy, different measurement methods are used to cluster and group the two populations respectively, further reducing the data dimension and completing population sampling; The group sampling strategies based on cluster imbalance include fast clustering strategy, adaptive partitioning strategy and group sampling strategy; The optimal feature subset is selected in the reduced search space through the dual-population three-file evolutionary strategy; The dual-population three-file evolution strategy includes the dual-population PSO evolution process, three-file assisted evolution and compression-expansion mechanism.

2. The feature selection algorithm of dynamic compression expansion based on dual population, dual metric and three files of PSO according to claim 1 is characterized in that: The population Sub s The SU measurement method is adopted, and the SU measurement includes the following steps: The calculation formula of SU is: Among them, X and Y are two variables to be measured, MI(X, Y) represents the mutual information between X and Y, H(X) and H(Y) are the entropies of X and Y respectively; the calculation formula of MI(X, Y) is: Where p(x, y) is the joint probability distribution of X and Y, and p(x) and p(y) are their marginal probability distributions respectively; the calculation formulas for H(X) and H(Y) are as follows: H(X)=-∑ x∈x p(x)log2 p(x), H(Y)=-∑ y∈Y p(y)log2 p(y), Among them, p(x) and p(y) are their marginal probability distributions respectively.

3. The feature selection algorithm of dynamic compression expansion based on dual population, dual metric and three files of PSO according to claim 2 is characterized in that: The population Sub M The MIC measurement method is adopted. The MIC measurement includes the following steps: The calculation formula of MIC is: Where X and Y are two variables to be measured, MI(X, Y) represents the mutual information between X and Y, m and n represent the number of grid divisions in the X and Y directions respectively, and min(n, m) represents the smaller value in the two directions.

4. The feature selection algorithm of dynamic compression expansion based on dual population, dual metric and three files of PSO according to claim 3 is characterized in that: The preliminary completion of the dimensionality reduction of high-dimensional data also includes the following steps: Sub s and population Sub M The filtering threshold is determined by the following function: r su =θ su ·SU max , r mic =θ mic ·MIC max , Among them, θ su and θ mic is the correlation coefficient, SU max and MIC max are the maximum SU value and MIC value between all features and class labels respectively; After determining the filtering threshold, the original data set S has D features, denoted as F = {f1, f2, f3, ..., f D }, First, Sub s Calculate each feature f i The SU value between the class label and the filter threshold ρ is calculated according to the corresponding threshold formula su , then, Sub M Calculate each feature f i The MIC value between the class label and the filter threshold ρ is calculated mic Finally, Sub s and Sub M Remove SU values ​​lower than ρ su and MIC values ​​lower than ρ mic features to complete the initial dimensionality reduction.

5. The feature selection algorithm of dynamic compression expansion based on dual population, dual metric and three files of PSO according to claim 1 is characterized in that: The fast clustering strategy includes the following steps: Use FCFC framework to s Fast clustering of features in ; The correlation measurement method in the FCFC framework is replaced by MIC for Sub M Fast clustering of features in ; Get two sets of ordered feature clusters s and clusters M , including D s and D M feature clusters.

6. The feature selection algorithm of dynamic compression expansion based on dual population, dual metric and three files of PSO according to claim 5 is characterized in that: The adaptive partitioning strategy comprises the following steps: The size of different sub-populations is determined by the sub-population adaptive strategy. The specific formula is: NM=N-Ns, Among them, N is the total size of the two populations, D s and D M Clusters s and clusters M The length, N S and N M Represents Sub s and Sub m The size of both populations; According to D s With D M The relative size is adjusted dynamically. The detailed process is shown in the following formula:

7. The feature selection algorithm of dynamic compression expansion based on dual population, dual metric and three files of PSO according to claim 6 is characterized in that: The group adoption strategy includes the following steps: According to the order of the feature clusters, clusters s and clusters M The feature clusters in are grouped separately, and finally we get and The number of feature clusters within each group is determined by the following formula: Where x represents the group number, D′ is the total number of feature clusters; Sub s and Sub M Initially, only and sampling; Sub s The sampling process includes: The feature clusters in are divided into different levels according to the number of features: the first level is composed of feature clusters of a single feature, and the second level is composed of feature clusters of multiple features; then the number of features selected from the two levels is dynamically determined, as shown in the following formula: size2=size--size1, Where size is the preset length of the particle, D1 and D2 are the lengths of level1 and level2 respectively, size1 and size2 are the number of features selected from level1 and level2 respectively; Sub s It is composed of two subpopulations, sub1 and sub2, which are spliced ​​horizontally; sub1: Before sampling, set the dynamically adaptive JS coefficient threshold ρ js ; All solutions in sub1 are obtained by randomly sampling size1 features from level1; except for the first solution, the remaining solutions need to calculate their maximum similarity with the solutions stored in sub1 before storing them in sub1; if the maximum similarity is less than or equal to ρ js , then store it in sub1; otherwise, regenerate; ρ js The initial settings are as follows: Among them, size1 is the number of features selected from level1, and D1 is the length of level1; If the condition is not met after five consecutive attempts, it means that ρ js If it is too small, dynamically adjust ρ js as follows: r js =ρ js +0.1*p js , The JS coefficient is used to measure the correlation between solutions; the calculation formula of the JS coefficient is as follows: Among them, set1 and set2 are both sets; sub2: Randomly select size2 feature clusters from level2, randomly select a feature in each selected feature cluster, and find the feature with the lowest correlation with it in the cluster as the complementary feature. The correlation measurement method is consistent with the method used in population filtering and clustering.

8. The feature selection algorithm of dynamic compression expansion based on dual population, dual metric and three files of PSO according to claim 1 is characterized in that: The dual-population PS0 evolution process comprises the following steps: The quality of particles is evaluated based on the weighted average of classification balance accuracy BACC and Fisher discriminant distance FDD. The fitness function is: fitness=w*BACC+(1-w)*FDD, Among them, w is the weight coefficient, the value range is [0, 1], BACC represents the classification balance accuracy, and the calculation formula is: Where c is the total number of categories, TP i is the number of true positive samples of the i-th label, N i is the total number of samples in the i-th class; FDD is based on the inter-class distance D B and intra-class distance D w Calculate the Fisher discriminant distance, the expression is: Among them, D B is the average distance from the sample to the nearest sample of different categories, D w is the average distance from a sample to the farthest sample of the same category.

9. The feature selection algorithm of dynamic compression expansion based on dual population, dual metric and three files of PSO according to claim 1 is characterized in that: The three-file assisted evolution includes the following steps: The three files include two local files Ach S and Ach M And a global file Ach G Among them, Ach S and Ach M Used to record Sub s and Sub M The importance of each feature in the respective search scope; Ach G Then Ach S and Ach M , records the global feature importance of the two populations; initially, the feature importance recorded in all archives is 0; Through the change of fitness value before and after the particle position is updated, the importance of new or missing features in the particle position is reversely updated; Ach S The update process includes: s In the equation, each particle dP i Evolves into dp′ through position update i , resulting in the fitness value changing from fitness i Become fitness i ; and dp i In comparison, dp′ i Contains the newly added feature set F a and the deleted feature set F d ; For Ach S If the features in F a , its feature importance imp m Update as follows: imp m =fitness′ i -fitness i ; For the existence of F d , its feature importance imp n Update as follows: imp n =fitness i -fitness′ i ; Local file auxiliary strategy: Ach S and Ach M By recording the importance of the features of their respective populations during the evolution process, it helps particles escape faster when they are trapped in the local optimum; dp i The escape process includes: According to Ach S According to the importance of features in , the features are divided into supporting features and opposing features, forming supporting feature set S and opposing feature set O respectively; according to dp i The stagnation time determines the number of mutated features num, and the formula is: num=min(max(t-P k ,1),5), Where t is dp i The number of consecutive stagnations; loop num times, each time with a probability of 0.5 from dp i Randomly delete a feature in O, or randomly add a feature from S that is not in dp i Features in; Global archive auxiliary strategy: When one of the populations stops PSO search but has not reached the set number of iterations, and the other population is still searching normally, use Ach G Stop searching for each particle in the population dp i Optimize; according to Ach G The feature importance recorded in , removing dp i The least important feature in the equation; the specific process is as follows: Extraction of Ach G All features whose importance is less than 0 are sorted in ascending order of importance and stored in the anti-feature set O g ; delete dp i In O g The top-ranked feature in dp; if the fitness decreases after deletion, restore the feature; otherwise, keep the new particle to replace the original dp i .

10. The feature selection algorithm of dynamic compression expansion based on dual population, dual metric and three archives of PSO according to claim 1 is characterized in that: The compression-expansion mechanism includes two parts, SE-1 and SE-2, which dynamically adjust particles and search space, and specifically includes the following steps: SE-1 mechanism: When the population evolution falls into a local optimum, it is preliminarily speculated that the particle length may be set too small, so the particle length needs to be dynamically expanded; in the process of expanding the particle length, each particle in the population is compressed using the S-1 strategy, and then the compressed particles are expanded using the E-1 strategy; S-1 strategy: dp through local filings i The feature with the lowest feature importance and importance less than 0 is deleted; if the fitness of the particle does not decrease after deletion, the original dp is replaced with a new particle i ; Otherwise, abandon the deletion operation; E-1 strategy: Determine the length that the particles in the population need to expand; calculate the particle expansion length using the following formula: Where D represents the data set, c is the number of categories in the data set, and p i is the proportion of samples of the i-th category to the total samples, Ent(D) is the entropy of the data set; fitness gb is the global optimal fitness value of the population; the particle expansion strategy of the population is based on the group sampling strategy, and the following changes are made to it: Modification of sampling level: The sampling level is adjusted from level 1 and level 2 to dp i The complements of these two layers are denoted as patches′ i and patches i ; ρ js Initial setting and modification of supervision object: ρ js The initial formula is changed to: Among them, dp′1 represents the first particle in patches′ i The union of the sampled features and the features of level 1 in the original particle; ρ js The supervision object is adjusted from the original sampling feature set to the expanded feature set belonging to level 1; Complementary particle generation strategy modification: when expanding, dp i+1 It is necessary to screen the complementary feature set and remove duplicate features; if dp i+1 If the number of features to be sampled at the second level does not exceed the number of complementary features after screening, they are directly selected; otherwise, the remaining features need to be randomly sampled; SE-2 mechanism: If the population still cannot continue to evolve after executing SE-1, it is inferred that the reason why the population falls into the local optimum is not that the particle length is too small, but that the search algorithm has completed the search of the current search space; use the S-2 strategy to compress the current population, and then use the E-2 strategy to expand the search space first and then expand the population; S-2 strategy: cancel the previous E-1 population expansion operation and restore the population to the state before the E-1 expansion; E-2 strategy: expand the search space and add the next group to the current search space; if all groups have been included, abandon the execution of the E-2 strategy; expand the population based on the new search space according to the E-1 expansion strategy.

Images