High-dimensional data feature selection method fusing sample proxy and particle swarm optimization

Abstract

The invention provides a high-dimensional data feature selection method fusing sample proxy and particle swarm optimization, and belongs to the technical field of feature selection, and the method comprises the steps: dividing a data set into a complete training sample set and a test data set, and dividing a feature space into a plurality of feature clusters through employing a high-dimensional feature space clustering technology; in each feature cluster, through an agent auxiliary evolution feature selection module based on sample selection, a most representative feature is selected by applying an IBPSO algorithm, and an optimal feature subset is formed and applied to a test data set. The feature selection process comprises the steps of initializing a population, constructing an initial sample agent set guided by multi-source knowledge, evaluating particle fitness by using an agent model, updating according to individual and global extremum, and optimizing search by combining a local search strategy guided by feature clustering. Meanwhile, a sample agent set management strategy fusing historical information is adopted, and effective updating of the sample agent set is ensured. According to the invention, a good classification effect can be obtained with low calculation cost.

Description

Technical Field

[0001] The present invention relates to the technical field of feature selection, and particularly to a high-dimensional data feature selection method integrating sample surrogate and particle swarm optimization. Background Art

[0002] Feature selection (FS) is a key technology in data analysis, and is particularly significant in fields such as bioinformatics, text mining, image classification, and Android malware detection. The data in these fields generally have a large number of irrelevant or redundant features, and the existence of these features not only reduces the running efficiency of algorithms, but also affects the prediction ability of models.

[0003] In the field of high-dimensional data feature selection, as the feature dimension increases, traditional feature selection methods face challenges such as the "curse of dimensionality" and high computational costs. To address these challenges, scholars have proposed a variety of feature selection methods based on evolutionary algorithms (EA), among which particle swarm optimization (PSO) has attracted much attention due to its simple concept, fast convergence speed, and ease of implementation. However, existing EA-based FS methods usually use classification algorithms to evaluate individuals on the entire training sample set, which leads to high computational costs.

[0004] To reduce computational costs, surrogate-assisted evolutionary algorithms (SAEAs) are introduced, which use low-cost surrogate models to approximately evaluate fitness. However, when dealing with high-dimensional FS problems, regression surrogate models are difficult to accurately capture the interaction between features due to the high dimensionality and discreteness of the feature space, as well as the high correlation or redundancy between features, thus affecting the accuracy of FS. Therefore, some researchers have tried to use sample selection techniques to select some representative samples from the entire sample set to construct a sample surrogate set to train a classifier, so as to obtain an approximate fitness value of an individual at a lower computational cost.

[0005] Although this method reduces computational costs to a certain extent, there are still some problems. For example, existing sample selection-based surrogate-assisted evolutionary feature selection methods mainly use random non-repetitive uniform sampling techniques to construct sample surrogate sets, which may lead to the generated sample surrogate sets being difficult to accurately represent the entire training sample set and unable to achieve high-quality evaluation of individuals. In addition, the update of the sample surrogate set may also affect the reliability of the update of individual extreme points. Summary of the Invention

[0006] The purpose of the present invention is to provide a high-dimensional data feature selection method integrating sample surrogate and particle swarm optimization, which can obtain better classification effects at a lower computational cost.

[0007] To achieve the above object, the present invention provides a high-dimensional data feature selection method integrating sample proxy and particle swarm optimization, comprising the following steps:

[0008] Step S1, dividing the original data set into a complete training sample set and a test data set;

[0009] Step S2, executing a high-dimensional feature space clustering module to divide the original feature space into several feature clusters;

[0010] Step S3, executing a proxy-assisted evolutionary feature selection module based on sample selection, using the IBPSO algorithm to search for representative features in each feature cluster, outputting an optimal feature subset and applying it to the test set data;

[0011] Step S31, initializing the population, and the initialized population is respectively used as the input of the initial sample proxy set construction module and the population update module guided by multi-source knowledge;

[0012] Step S32, executing an initial sample proxy set construction module guided by multi-source knowledge, constructing a set of proxy units S_Sample based on the representative sample selection guided by multi-source knowledge, and executing an initial sample proxy set construction strategy based on the set of proxy units S_Sample to construct an initial sample proxy set S 0 ;

[0013] Step S33, executing a loop-running population update module, calculating the approximate fitness value of the particles using the proxy model trained by the initial sample proxy set S 0 Initializing the individual extreme value Pbest and the global extreme value Gbest of each particle, updating the individual extreme value Pbest and the global extreme value Gbest respectively using the individual extreme value point update mechanism and the global extreme value point update mechanism, and performing a local search strategy on the global extreme value Gbest based on the local search strategy guided by feature clustering to update the individual position;

[0014] During the evolutionary search process, the management of the sample proxy set is realized through a sample proxy set management strategy module integrating historical information.

[0015] Preferably, in the IBPSO algorithm, the particle update rule is as follows:

[0016]

[0017] Among them, represents the position of the i-th particle in the j-th dimension at the (t + 1)-th generation; represents the individual historical optimal position of the i-th particle in the j-th dimension at the t-th generation; represents the global optimal position of the entire population in the j-th dimension at the t-th generation; G(0, 1) represents a standard Gaussian distribution; Denotes the floor function.

[0018] Preferably, in step S32, the set of proxy units S_Sample = {S cr , S sim , S rand}, where S cr denotes the proxy unit selected based on the degree of crowding; S sim denotes the proxy unit selected based on the degree of similarity; S rand denotes the proxy unit selected based on randomness;

[0019] The proxy unit S selected based on the degree of crowding cr is calculated as follows:

[0020] Calculate the crowding degree value CR j of the sample x j :

[0021] CR j = η × Var j + (1 - η) × OL j (2);

[0022] where η is a parameter; Var j denotes the degree of fluctuation; OL j denotes the degree of outlier;

[0023] Var j is calculated as follows:

[0024]

[0025] where m represents the number of samples; D j,h denotes the Euclidean distance from the j-th sample to the h-th sample;

[0026] OL j is calculated as follows:

[0027]

[0028] Based on the crowding degree value CR j sort the samples in the sample cluster SC i in descending order, and select the top samples and add them to the proxy unit S selected based on the degree of crowding cr ; where P represents the total number of samples contained in the current sample proxy set; K represents the number of true class labels in the complete training sample set;

[0029] The proxy unit S selected based on the degree of similarity sim is calculated as follows:

[0030] When the sample cluster SCi When all samples in i are from the same category, randomly select samples from the sample cluster SC as representative samples and add them to the proxy unit S selected based on similarity sim ;

[0031] When all samples in the sample cluster SC i do not belong to the same category, calculate the similarity value SIM j of the sample x j . Based on the similarity value SIM j , sort the samples in the sample cluster SC i in descending order and select the top samples to add to the proxy unit S selected based on similarity sim ;

[0032] The calculation formula for the similarity value SIM j is as follows:

[0033]

[0034] where represents the value of the central sample μ of the c-th category in the sample cluster SC i on the i-th feature; c represents the value of the sample x on the i-th feature; c_k represents the number of categories included in the sample cluster SC j ; i The proxy unit S selected based on randomness is calculated as follows:

[0035] For each sample cluster with a single class label, randomly select rand samples and add them to the proxy unit S selected based on randomness

[0036] ; For a sample cluster with multiple class labels, according to the proportion r of the number of samples corresponding to each class label in each cluster, dynamically adjust the sampling quantity, and randomly select rand samples from each category and add them to the proxy unit S selected based on randomness

[0037] ; samples and add them to the proxy unit S selected based on randomness rand .

[0038] Preferably, in step S32, based on the proxy unit set S_Sample, execute the initial sample proxy set construction strategy to construct the initial sample proxy set S 0 , and the specific operation is as follows:

[0039] First, use S_Sample = {S cr, S sim , S rand The surrogate model trained by the surrogate unit in {} evaluates individuals, determines the optimal individual under each surrogate unit, and constitutes the optimal individual set S_X best ;

[0040] Then, calculate the approximate fitness values of the optimal individuals to form an approximate fitness value vector. a cr , a sim and a rand respectively represent the approximate fitness value vectors constructed by approximately evaluating the individuals in S_X cr , S sim , S rand using S best ; represents the approximate fitness value obtained by approximately evaluating the first individual in S_X using the surrogate model trained by the surrogate unit S selected based on crowding degree cr ; best represents the approximate fitness value obtained by approximately evaluating the second individual in S_X using the surrogate model trained by the surrogate unit S selected based on crowding degree represents the approximate fitness value obtained by approximately evaluating the second individual in S_X using the surrogate model trained by the surrogate unit S selected based on crowding degree cr ; best represents the approximate fitness value obtained by approximately evaluating the third individual in S_X using the surrogate model trained by the surrogate unit S selected based on crowding degree represents the approximate fitness value obtained by approximately evaluating the first individual in S_X using the surrogate model trained by the surrogate unit S selected based on similarity degree cr ; best represents the approximate fitness value obtained by approximately evaluating the second individual in S_X using the surrogate model trained by the surrogate unit S selected based on similarity degree represents the approximate fitness value obtained by approximately evaluating the second individual in S_X using the surrogate model trained by the surrogate unit S selected based on similarity degree sim ; best represents the approximate fitness value obtained by approximately evaluating the third individual in S_X using the surrogate model trained by the surrogate unit S selected based on similarity degree represents the approximate fitness value obtained by approximately evaluating the first individual in S_X using the surrogate model trained by the surrogate unit S selected based on similarity degree sim ; best represents the approximate fitness value obtained by approximately evaluating the second individual in S_X using the surrogate model trained by the surrogate unit S selected based on similarity degree represents the approximate fitness value obtained by approximately evaluating the second individual in S_X using the surrogate model trained by the surrogate unit S selected based on similarity degree sim ; best represents the approximate fitness value obtained by approximately evaluating the third individual in S_X using the surrogate model trained by the surrogate unit S selected based on similarity degree represents the approximate fitness value obtained by approximately evaluating the first individual in S_X using the surrogate model trained by the surrogate unit S selected based on randomness rand ; best represents the approximate fitness value obtained by approximately evaluating the second individual in S_X using the surrogate model trained by the surrogate unit S selected based on randomness represents the approximate fitness value obtained by approximately evaluating the second individual in S_X using the surrogate model trained by the surrogate unit S selected based on randomness rand ; best represents the approximate fitness value obtained by approximately evaluating the third individual in S_X using the surrogate model trained by the surrogate unit S selected based on randomness Indicates the use of the agent unit S selected based on randomness rand The trained agent model approximately evaluates S_X best The approximate fitness value obtained by the third individual in

[0041] Meanwhile, the classification model trained using the complete training sample set evaluates S_X best The individuals in Indicates the evaluation of S_X using the complete training sample set best The true fitness value obtained by the first individual in Indicates the evaluation of S_X using the complete training sample set best The true fitness value obtained by the second individual in Indicates the evaluation of S_X using the complete training sample set best The true fitness value obtained by the third individual in

[0042] Finally, measure the similarity between the true fitness value vector o and the approximate fitness value vectors a cr 、a sim and a rand and select the agent unit corresponding to the approximate fitness value vector with the greatest similarity as the initial sample agent set S 0 .

[0043] Preferably, in step S33, the individual extreme point update mechanism:

[0044] By utilizing the historical information of the particles and the individual extreme points, measure the superiority and inferiority relationship between the particles of the current generation and their individual extreme points, thereby determining the position of the individual extreme points and their fitness values at generation t;

[0045] First, give a judgment threshold as follows:

[0046]

[0047] Among them, f_s t-1 (Pbest i t-1 ) is the fitness value of Pbest i t-1 based on the sample agent set S t-1 at generation t - 1; f_s t (X i t ) is the fitness value of X i t based on the sample agent set S t at generation t; |·| represents taking the absolute value; θ f represents the threshold; N represents the population size; Pbest it-1 Denote the \(i\)-th individual in the individual extreme value Pbest at generation \(t - 1\); \(X\) i t Denote the \(i\)-th individual in the current population \(X\) at generation \(t\);

[0048] For each particle, calculate the difference Diff between the fitness value of the particle at generation \(t\) based on the sample proxy set \(S\) t and the fitness value of the individual extreme point at generation \(t - 1\) based on the sample proxy set \(S\) t-1 as Diff = \(f_s\) t (\(X\) i t ) - \(f_s\) t-1 (Pbest i t-1 );

[0049] Based on \(\theta\) f and Diff, the update mechanism of the individual extreme point is divided into the following three cases:

[0050] Case 1: When Diff > \(\theta\) f , update the position of the individual extreme point Pbest i t = \(X\) i t , and its fitness value \(f_s\) t (Pbest i t ) = \(f_s\) t (\(X\) i t );

[0051] Case 2: When -Diff > \(\theta\) f , update the position of the individual extreme point Pbest i t = Pbest i t-1 , and its fitness value \(f_s\) t (Pbest i t ) = \(f_s\) t-1 (Pbest i t-1 );

[0052] Case 3: When |Diff| ≤ \(\theta\) f , recalculate the fitness value \(f_s\) t of Pbest i t-1 based on \(S\): t (Pbest i t-1 );

[0053]

[0054] If \(f_s\) t (X i t ) is greater than \(f_s\) t (Pbest i t-1 ), then update the position of the individual extreme point Pbest i t = X i t , and update its fitness value \(f_s\) t (Pbest i t ) = \(f_s\) t (X i t );

[0055] If \(f_s\) t (X i t ) is less than \(f_s\) t (Pbest i t-1 ), then update the position of the individual extreme point Pbest i t = Pbest i t-1 , and update its fitness value \(f_s\) t (Pbest i t ) = \(f_s\) t (Pbest i t-1 );

[0056] If the fitness values of the two are equal, then according to the number of features included in X i t and Pbest i t-1 , select the one with fewer features as the individual extreme point at generation t.

[0057] Preferably, in step S33, the local search strategy guided by feature clustering is as follows:

[0058] First, store the clustering result of the feature space clustering stage as knowledge in the feature cluster set G;

[0059] Next, map Gbest to the feature space, and further perform feature deletion and addition operations in the feature space using the importance and correlation - redundancy of features to obtain an improved feature subset F best_local ;

[0060] Finally, based on the feature cluster set G, represent F in the decision variable space best_local to obtain the improved Gbest;

[0061] Among them, during the representation process, if there are multiple features in F best_local belonging to the same feature cluster, further select the most important feature for representation by using the importance of the features;

[0062] The calculation formula for correlation - redundancy is as follows:

[0063]

[0064] Among them, C is the class label; |F| is the scale of the feature subset; I(f j , C) is the mutual information value between the feature f j and the class label C; I(f j , f i ) is the mutual information value between the feature f j and the feature f i .

[0065] Preferably, during the evolutionary search process, the management of the sample surrogate set is realized through the sample surrogate set management strategy module that fuses historical information, including two update times, namely update time 1 and update time 2;

[0066] Among them, update time 1:

[0067] When the relative prediction error value of the model is greater than 0.01, update time 1 is satisfied, and update rule 1 is executed;

[0068] The calculation formula for the relative prediction error value of the model is as follows:

[0069]

[0070] Among them, Gbest t represents the global extreme point obtained by the algorithm at the t - th iteration; F'_sur(Gbest t ) and F_real(Gbest t ) respectively represent the predicted fitness value and the real fitness value of Gbest t ;

[0071] When the global extreme point has not changed for consecutive t 0 generations, that is, update time 2 is satisfied, then update rule 2 is executed.

[0072] Preferably, update rule 1 is as follows:

[0073] The current sample surrogate set S t-1It evolved from one of the proxy units of S_Sample, and this proxy unit is deleted from S_Sample;

[0074] Subsequently, randomly select one from the remaining proxy units of S_Sample and denote it as S tem ;

[0075] Finally, classify S based on the optimal feature subset obtained during the current evolutionary search process tem , screen out the misclassified samples, and add these samples as representative samples to S t-1 , and at the same time remove duplicate samples.

[0076] It should be noted that misclassified samples are selected as representative samples because they usually contain unique information, are located in complex data regions, and may not be fully considered by the current classification model.

[0077] To ensure the rationality of the size of the filled samples, an upper limit l max and a lower limit l min are set. Specifically, the lower limit l min is set to the number of class labels included in S t-1 . The upper limit l max is determined based on the number of samples in S t-1 , and is usually set to P min *|S t-1 |. If the number of misclassified samples is less than l min , randomly select some samples from S_Whole to supplement to this number; if the number of samples exceeds l max , randomly delete the excess samples. At the same time, in order to avoid the sample proxy set approaching the entire sample set too early and increasing the calculation cost when continuously satisfying update condition 1 but not yet satisfying update condition 2, it is stipulated that the size of S t-1 shall not exceed P max times the size of S_Whole, that is, not exceeding |S t-1 |≤P max *|S_Whole|.

[0078] Preferably, the update rule 2 is as follows:

[0079] The current sample proxy set S t-1 evolved from one of the proxy units of S_Sample. Use the other two proxy units except this proxy unit to train the proxy model respectively, approximately evaluate the current population, obtain the optimal feature subsets corresponding to the proxy units respectively, and form the union F Gbest of the optimal feature subsets with F union , based on F unionRe - cluster the entire original sample space and execute the initial sample proxy set construction strategy to generate a new sample proxy set.

[0080] Therefore, the present invention adopts the above - mentioned high - dimensional data feature selection method that fuses sample proxy and particle swarm optimization, and the beneficial technical effects are as follows:

[0081] This method combines clustering technology and the idea of assisting with a proxy sample set, aiming to improve the efficiency and accuracy of feature selection. By designing an initial sample proxy set construction strategy guided by multi - source knowledge to construct an initial proxy sample set, and using the machine learning model trained on it as a proxy model to reduce the cost of subsequent evaluations. Utilize historical search information and specific update rules to maintain the quality and diversity of the proxy sample set. Propose an extreme point update strategy to reduce the deviation when comparing the individual fitness value and the fitness value of the individual extreme point. The local search strategy guided by feature clustering guides the addition and deletion of features to improve the global optimal solution and enhance the quality of the solution.

[0082] This algorithm was compared with six existing feature selection methods on 10 datasets. The results show that this algorithm can achieve the best classification accuracy at a relatively low computational cost in most cases, demonstrating its effectiveness in dealing with high - dimensional data.

[0083] In summary, by combining clustering technology and the idea of assisting with a proxy sample set, this algorithm provides an effective high - dimensional feature selection method, which can improve the classification accuracy and the quality of the solution while maintaining a relatively low computational cost. Brief Description of the Drawings

[0084] Figure 1 It is the basic framework diagram of the algorithm proposed in the present invention;

[0085] Figure 2 It is a schematic diagram of the local search process. Detailed Embodiments

[0086] The technical solution of the present invention will be further described below through the drawings and embodiments.

[0087] Unless otherwise defined, the technical terms or scientific terms used in the present invention should have the ordinary meaning understood by those of ordinary skill in the field to which the present invention belongs.

[0088] Embodiment 1

[0089] The purpose of FS is to select d features (where d ≤ D) from a dataset containing D - dimensional features and M instances through a certain strategy to form a subset to maximize the classification accuracy F(·), and the mathematical expression is as follows:

[0090]

[0091] Among them, x i ∈ {0, 1} indicates the value of x i Among them, x i = 0 represents that the i-th feature is not selected, and x i = 1 represents that the i-th feature is selected.

[0092] As Figure 1 shown, the basic framework of ISPSO-FS proposed by the present invention includes two modules:

[0093] 1) High-dimensional feature space clustering module;

[0094] 2) Agent-assisted evolutionary feature selection module based on sample selection.

[0095] The agent-assisted evolutionary feature selection module based on sample selection includes three sub-modules: 1) Initial sample agent set construction module guided by multi-source knowledge; 2) Population update module; 3) Sample agent set management strategy module integrating historical information.

[0096] Among them, the main purpose of the initial sample agent set construction module guided by multi-source knowledge is to construct a set of proxy units using the sample selection method on the reduced feature space when the iteration number t = 1 after executing the high-dimensional feature space clustering module, and construct the initial sample agent set (t is the current iteration number).

[0097] The purpose of the population update module is to continuously generate candidate feature subsets through an improved integer particle swarm algorithm. Among them, the evaluation of individuals is based on a machine learning model (i.e., the proxy model) trained on the sample agent set rather than the complete training sample set.

[0098] When the iteration number t > 1, the sample agent set management strategy module integrating historical information is to update the sample agent set in a timely manner during the evolution process to ensure the prediction accuracy of the proxy model.

[0099] The method proposed by the present invention includes the following process:

[0100] Step S1, divide the original data set into a complete training sample set and a test data set, and perform the subsequent FS process on the complete training sample set;

[0101] Step S2: Execute the high-dimensional feature space clustering module to divide the original feature space into several feature clusters. Specifically, use the high-dimensional feature clustering mentioned in

Song, Xian-Fang, et al. "A fast hybrid feature selection based on correlation-guided clustering and particle swarm optimization for high-dimensional data." IEEE Transactions on Cybernetics 52.9 (2021): 9573-9586.

[0102] Step S3: Execute the proxy-assisted evolutionary feature selection module based on sample selection, use the IBPSO algorithm to search for representative features in each feature cluster, output the optimal feature subset and apply it to the test set data;

[0103] Step S31: Initialize the population, and the initialized population is used as the input of the multi-source knowledge-guided initial sample proxy set construction module and the population update module respectively;

[0104] Step S32: Execute the multi-source knowledge-guided initial sample proxy set construction module. Based on the representative sample selection guided by multi-source knowledge, construct a set of proxy units S_Sample, and execute the initial sample proxy set construction strategy based on the set of proxy units S_Sample to construct the initial sample proxy set S 0 ;

[0105] Step S33: Execute the loop-running population update module, use the proxy model trained by the initial sample proxy set S 0 to calculate the approximate fitness value of the particles, initialize the individual extreme value Pbest and the global extreme value Gbest of each particle, use the individual extreme value point update mechanism and the global extreme value point update mechanism to update the individual extreme value Pbest and the global extreme value Gbest respectively, and perform a local search strategy on the global extreme value Gbest based on the feature clustering-guided local search strategy to update the individual position.

[0106] During the evolutionary search process, the management of the sample proxy set is realized through the sample proxy set management strategy module that fuses historical information.

[0107] It should be noted that in the later stage of the search of the ISPSO-FS algorithm, the true fitness value of an individual is calculated by using the classification model trained with the complete training sample set to ensure the effectiveness of the evolutionary search direction of the algorithm. According to the method in

Nguyen, Hoai Bach, et al. "Surrogate-model based particle swarm optimisation with local search for feature selection in classification." Applications of Evolutionary Computation: 20th European Conference, EvoApplications 2017, Amsterdam, The Netherlands, April 19-21, 2017, Proceedings, Part I 20. Springer International Publishing, 2017.

[0108] The following is a specific description of the construction module of the initial sample surrogate set guided by multi-source knowledge.

[0109] The construction strategy of the initial sample surrogate set guided by multi-source knowledge.

[0110] First, use K-means clustering to divide the sample space into K clusters (K is the number of true class labels in the complete training sample set); then, comprehensively consider the crowding degree, similarity degree and randomness of the samples, and select representative samples from each sample cluster from multiple perspectives to construct a set of surrogate units; finally, construct the initial sample surrogate set based on the true fitness value of the individual and the approximate fitness value under different surrogate units.

[0111] The method for selecting representative samples considering multiple knowledge characteristics.

[0112] The sample size of the initial sample surrogate set is a key decision factor. If the sample size is too small, it may not fully reflect the distribution of the real data, while if it is too large, it will increase the running cost. In addition, a dataset with a large number of classes requires more representative samples to ensure comprehensiveness. Therefore, the value of the sample size S_P of the initial sample surrogate set for each dataset is as follows:

[0113] S_P = (P min +(P max -Pmin ) * log 10 (k)) * |S_Whole| (2);

[0114] Among them, |S_Whole| represents the size of the complete training sample set; k represents the number of categories. P max and P min both belong to [0, 1] and can be set according to the preference of the decision maker. P max and P min are the upper and lower limit values of the proportion of the sample size of the initial sample proxy set to the size of the complete training sample set. In this embodiment, P max is set to 0.5, and P min is set to 0.1.

[0115] (1) Representative sample selection strategy based on crowding degree.

[0116] The degree of fluctuation is the variance of the Euclidean distance between a sample and other samples, describing the degree of difference between samples. The degree of outlier is the average Euclidean distance between a sample and other samples, evaluating the degree of dispersion between samples. By calculating the degree of fluctuation and the degree of outlier, the crowding degree of samples can be evaluated, so as to effectively identify representative samples with differences and diversities. Suppose a sample cluster contains m samples, x 1 , x 2 ,..., x m , and each sample has n features. For the Euclidean distance D j,h from the j-th sample to the h-th sample. First, the definitions of the degree of fluctuation and the degree of outlier are given.

[0117] Degree of fluctuation: For the j-th sample in a certain sample cluster, its degree of fluctuation is defined as Var j .

[0118] The greater the degree of fluctuation of a sample, the more obvious the difference between this sample and other samples.

[0119]

[0120] Degree of outlier: For the j-th sample in a certain sample cluster, its degree of fluctuation is defined as OL j .

[0121] The greater the degree of outlier of a sample, the more obvious the discreteness between this sample and other samples.

[0122]

[0123] For any sample, the degree of fluctuation value and the degree of outlier value are weighted and summed to approximately characterize the crowding degree of the sample.

[0124] Degree of crowding: For the j-th sample in a certain sample cluster, its degree of crowding is defined as CR j .

[0125] CR j = η * Var j +(1 - η) * OL j (5);

[0126] Among them, the importance of Var j and OL j is controlled by the parameter η. The larger η is, the more important the degree of sample fluctuation is. On the contrary, the smaller η is, the more important the degree of sample outlier is. The value of η can be set according to the preference of the decision maker. In this embodiment, η is set to 0.5.

[0127] Based on the degree of crowding value CR j sort the samples in the sample cluster SC i in descending order, and select the top samples and add them to the proxy unit S cr selected based on the degree of crowding; where P represents the total number of all samples included in the current sample proxy set; K represents the number of true class labels in the complete training sample set.

[0128] (2) Representative sample selection strategy based on similarity degree.

[0129] When all samples in the sample cluster SC i belong to the same category, randomly select i samples from the sample cluster SC as representative samples and add them to the proxy unit S sim ;

[0130] When all samples in the sample cluster SC i do not belong to the same category, calculate the similarity degree value SIM j of the sample x j , sort the samples in the sample cluster SC j in descending order based on the similarity degree SIM i , and select the top samples and add them to the proxy unit S sim ;

[0131] The calculation formula of the similarity degree value SIM j is as follows:

[0132]

[0133] Among them, represents the central sample μ of the c-th category in the sample cluster SC i ​c The value on the i-th feature; Denote the sample x j The value on the i-th feature; c_k represents the sample cluster SC i The number of categories included;

[0134] (3) Representative sample selection strategy based on randomness.

[0135] This strategy adopts random balanced sampling technology.

[0136] For each sample cluster with a single class label, by randomly selecting samples and adding them to the proxy unit S selected based on randomness rand ;

[0137] For sample clusters with multiple class labels, according to the proportion r of the number of samples corresponding to each class label in each cluster, dynamically adjust the sampling quantity, and randomly select samples from each category and add them to the proxy unit S selected based on randomness rand ;

[0138] Finally, form the proxy unit S rand , thus ensuring sample diversity and class balance, and providing a high-quality representative sample set for the subsequent evolution process.

[0139] Construction strategy of the initial sample proxy set.

[0140] First, use the proxy models trained by the proxy units in S_Sample = {S cr , S sim , S rand} to evaluate individuals, determine the optimal individuals under each proxy unit, and form the optimal individual set S_X best . Then, calculate the approximate fitness values of the optimal individuals to form an approximate fitness value vector. Meanwhile, use the classification model trained with the complete training sample set to evaluate the individuals in S_X best to obtain the true fitness value vector Finally, measure the similarity between the true fitness value vector o and the approximate fitness value vectors a cr , a sim and a rand (in this embodiment, cosine similarity is adopted), and select the proxy unit corresponding to the approximate fitness value vector with the greatest similarity as the initial sample proxy set.

[0141] Sample proxy set management strategy integrating historical information.

[0142] In order to ensure the prediction accuracy of the proxy model, the sample proxy set needs to be updated in a timely manner, but frequent updates will increase the computational cost. On the one hand, when the model predicts a large difference between the fitness value and the true value, representative samples can be filled in to improve the prediction accuracy; on the other hand, when the particle swarm is close to convergence, filling in samples alone may not be enough to improve the effect. At this time, samples should be reselected based on the optimal feature subset to construct a new sample proxy set. The present invention determines the update timing based on the model relative prediction error and population convergence. Among them, the definition of the model relative prediction error is as follows:

[0143] Model relative prediction error: For a proxy model trained on a sample proxy set, the calculation expression of its model relative prediction error is as follows:

[0144]

[0145] Among them, Gbest t Indicates the global extreme point obtained by the algorithm at the tth iteration; F'_sur(Gbest t ) and F_real(Gbest t ) represent Gbest t If the RPE value is small, the predicted fitness value of the proxy model is close to the true value, indicating that the sample proxy set is effective; if the RPE value is large, the sample proxy set needs to be updated.

[0146] When the global extreme point is continuous t 0 In summary, in each iteration of the algorithm, if the model relative prediction error value is greater than a certain threshold, the update timing 1 is met, and the update rule 1 is executed at this time; when the global extreme point is continuous t 0 When the generation does not change, update timing 2 is met, and update rule 2 is executed.

[0147] Update rule 1: Filling of sample proxy set. In each iteration, when update opportunity 1 is met, representative samples are selected from the remaining samples according to the filling rule and added to the current sample proxy set.

[0148] The filling rules are as follows: First, clarify the current sample proxy set S t-1 Based on which proxy unit is it evolved? If it is S sim , then from S_Sample = {S cr ,S sim ,S rand} to delete S sim ; Then, in S_Sample = {S cr ,S rand} randomly selects an agent unit and records it as Stem ; Finally, classify S based on the optimal feature subset obtained during the current evolutionary search process tem to screen out misclassified samples, and add these samples as representative samples to S t-1 , while removing duplicate samples. It should be noted that misclassified samples are selected as representative samples because they usually contain unique information, are located in complex data regions, and may not be fully considered by the current classification model.

[0149] To ensure the rationality of the size of the filled samples, an upper limit l max and a lower limit l min are set. Considering the class labels covered by the samples in S t-1 , set l min to the number of class labels in S t-1 . The upper limit is based on the number of samples in S t-1 , and usually set l max to P min *|S t-1 |. If the number of misclassified samples is less than l min , randomly select some samples from S_Whole to supplement to this number; if it is greater than l max , randomly delete the excess samples. It should be noted that when continuously meeting update opportunity 1 and not yet meeting update opportunity 2, to avoid the sample proxy set approaching the entire sample set too early and increasing the calculation cost, it is stipulated that the size of S t-1 shall not exceed P max times the size of S_Whole, that is, |S t-1 |≤P max *|S_Whole|.

[0150] Update rule 2: Rebuilding the sample proxy set. In each iteration, when update opportunity 2 is met, the sample proxy set needs to be rebuilt. The specific process is as follows:

[0151] First, save the corresponding optimal feature subset F Gbest in the current population, and clarify which proxy unit S t-1 evolved from. Suppose it is S sim ; then, use the proxy units S sim other than S cr and S rand to train the proxy models respectively, approximately evaluate the current population, and obtain the corresponding optimal feature subsets F cr and F rand of the proxy units respectively, and form the union F Gbest of the optimal feature subsets with F union =(F Gbest ∪Fcr ∪F rand ) and based on F union Re - cluster the entire original sample space and execute the initial sample surrogate set construction strategy to generate a new sample surrogate set.

[0152] Agent - assisted integer particle swarm optimization algorithm.

[0153] Since the traditional BBPSO algorithm is no longer applicable to process feature clusters, an integer particle swarm optimization algorithm (IBPSO) is proposed. To reduce the search space of the algorithm, IBPSO is introduced to search for the optimal feature subset. Taking the i - th particle as an example, the particle update rule is as follows:

[0154]

[0155] where G(0, 1) is the standard Gaussian distribution; is the floor function.

[0156] It should be noted that during the execution of IBPSO, to reduce the computational cost, the above - proposed sample surrogate set is used to replace the complete training sample set to train the classification model to calculate the fitness value of the particle.

[0157] Extreme point update strategy.

[0158] In the traditional extreme point update strategy, if the current fitness of the particle is better than its historical best value, then update the individual extreme point; if the individual extreme point of this particle is better than the current global extreme point, then update the global extreme point. In the agent - assisted particle swarm optimization algorithm, approximate fitness values are usually used. However, this method has deficiencies. After updating the sample surrogate set in the t - th generation, the approximate fitness value of the individual extreme point of the particle in the (t - 1) - th generation is obtained based on the sample surrogate set in the (t - 1) - th generation, while the fitness value of the current particle is based on the sample surrogate set in the t - th generation. Therefore, directly comparing the individual approximate fitness value in the t - th generation with the individual extreme point in the (t - 1) - th generation and then updating the individual extreme point may be unfair and affect the accuracy. In addition, if the surrogate model is re - trained based on the sample set in the t - th generation to evaluate the individual extreme point in the (t - 1) - th generation, the computational cost will be significantly increased.

[0159] In view of this, an individual extreme point update mechanism is introduced: by using the historical information of the particle and the individual extreme point, the superiority - inferiority relationship between the particle in the current generation and its individual extreme point is measured, so as to determine the position of the individual extreme point and its fitness value at the t - th generation. First, a judgment threshold is given as follows:

[0160]

[0161] In the formula, f_s t-1 (Pbesti t-1 ) represents Pbest at t - 1 i t-1 Based on the sample surrogate set S t-1 The fitness value; f_s t (X i t ) represents X at generation t i t Based on the sample surrogate set S t The fitness value; |·| represents taking the absolute value.

[0162] For each particle, calculate the fitness value of the particle at generation t based on S t and the fitness value of the individual extreme point at generation t - 1 based on S t -1 The difference Diff = f_s t (X i t ) - f_s t-1 (Pbest i t-1 ). Based on θ f and Diff, the update mechanism of the individual extreme point can be roughly divided into the following 3 cases:

[0163] Case 1: If Diff > θ f , then f_s t (X i t ) is significantly greater than f_s t-1 (Pbest i t-1 ). Most likely, f_s t (X i t ) is also greater than f_s t (Pbest i t-1 ). Then update the position of the individual extreme point Pbest i t = X i t , and its fitness value f_s t (Pbest i t ) = f_s t (X i t ).

[0164] Case 2: If -Diff > θ f , then f_s t-1 (Pbest i t-1 ) is significantly greater than f_st (X i t ), most likely \(f_s\) t (Pbest i t-1 ) is also greater than \(f_s\) t (X i t ), then update the position of the individual extreme point Pbest i t = Pbest i t-1 , and its fitness value \(f_s\) t (Pbest i t ) = \(f_s\) t-1 (Pbest i t-1 ).

[0165] Case 3: If \(|Diff| \leq \theta\) f , then the gap between \(f_s\) t-1 (Pbest i t-1 ) and \(f_s\) t (X i t ) is relatively small, and it is difficult to directly distinguish the relationship between \(f_s\) t (Pbest i t-1 ) and \(f_s\) t (X i t ). Then it is necessary to recalculate the fitness value \(f_s\) of Pbest t based on S again i t-1 , see formula (10) for details. t (Pbest i t-1 ).

[0166]

[0167] If \(f_s\) t (X i t ) is greater than \(f_s\) t (Pbest i t-1 ), then update the position of the individual extreme point Pbest i t = X i t , and update its fitness value \(f_s\) t (Pbest i t) = f_s t (X i t )); If f_s t (X i t ) is less than f_s t (Pbest i t-1 ), then update the position of the individual extreme value point Pbest i t = Pbest i t-1 , and update its fitness value f_s t (Pbest i t ) = f_s t (Pbest i t-1 ); If the fitness values of the two are equal, then according to X i t and Pbest i t-1 's included feature numbers, select the one with fewer feature numbers as the individual extreme value point at generation t.

[0168] Furthermore, in order to improve the performance of ISPSO-FS, the global extreme value point update rule proposed in

Song, Xianfang, et al. "Surrogate sample-assisted particle swarm optimization for feature selection on high-dimensional data." IEEE transactions on evolutionary computation 27.3 (2022): 595-609.

[0169] Feature clustering-guided local search strategy.

[0170] Most existing local search strategies usually add or delete features by adjusting one or more decision variables. However, in the proposed algorithm, each decision variable corresponds to a feature cluster. Using the existing methods will directly affect the entire feature cluster and ignore the feature correlation relationships within the feature cluster.

[0171] In view of this, a feature clustering-guided local search strategy is proposed. First, store the clustering results of the feature space clustering stage as knowledge into the feature cluster set G; then, map Gbest to the feature space, and further perform feature deletion and addition operations in the feature space using the importance degree and correlation-redundancy of features to obtain an improved feature subset F best_local; Finally, based on the feature cluster set G, represent F in the decision variable space best_local to obtain the improved Gbest. Among them, during the representation process, if there are multiple features in F best_local belonging to the same feature cluster, further select the most important feature for representation by using the importance of the features. In this embodiment, the symmetric uncertainty SU C between the feature and the class label is used to measure the importance degree of the feature, and a correlation-redundancy index is defined. The definition of this index is as follows:

[0172] Correlation-redundancy value: For the j-th feature in the feature subset, its correlation-redundancy value is denoted as

[0173]

[0174] where C represents the class label; |F| represents the scale of the feature subset; I(f j , C) represents the mutual information value between the feature f j and the class label C; I(f j , f i ) represents the mutual information value between the feature f j and the feature f i .

[0175] The local search strategy is mainly applied to the proxy-assisted evolutionary search stage of ISPSO-FS. When Gbest has not changed for t 0 consecutive generations, this strategy can be executed to help the algorithm jump out of the local optimum. In this embodiment, α is taken as 0.125. It should be noted that the greater the maximum number β of attempts to develop Gbest, the higher the possibility of improving Gbest, but it will also increase the computational cost. Considering that as the algorithm iterates, Gbest gets closer and closer to the optimal solution, the results of the subsequent local searches are usually closer to the optimal solution than the previous ones. In view of this, β is set to increase as the number of times the local search strategy is executed increases, that is, β = t 0 × flag (where flag is the value of the number of times the local search strategy is currently executed), and its upper limit is N.

[0176] Figure 2 is a schematic diagram of the execution of the local search process. Assume that the original feature space is {f 1 , f 2 , f 3 , f 4 , f 5 , f 6 , f 7 , f 8}}, and the feature clusters in G are {Cluster 1 , Cluster 2,Cluster 3 ,Cluster 4}, respectively contain the features {f 1}, {f 2 , f 3 , f 4}, {f 5 , f 6}, and {f 7 , f 8}. (1) The corresponding process of Gbest in the feature space: Gbest = (1, 3, 2, 0), indicating that the first feature f 1 in the feature cluster Cluster 1 , the third feature f 2 in the feature cluster Cluster 4 , and the second feature f 3 in the feature cluster Cluster 6 are selected, and no feature in the feature cluster Cluster 4 is selected, forming the optimal feature subset F best = {f 1 , f 4 , f 6}. (2) Perform the development operation: Perform the feature deletion operation on F best . Suppose the feature f 1 is deleted, and F best_dele = {f 4 , f 6} is obtained. Then, perform the feature addition operation on F best_dele . Suppose the features f 2 and f 8 are added to F best_dele , and F best_local = {f 2 , f 4 , f 6 , f 8} is obtained. (3) The representation process of F best_local in the decision space: No feature in the feature cluster Cluster 1 is selected, so the first dimension of Gbest_new is 0. The second feature f 3 and the second feature f 4 in the feature clusters Cluster 6 and Cluster 8 are selected respectively, so the third and fourth dimensions of Gbest_new are both 2. In addition, for the feature cluster Cluster 2 , since the features f 2 and f 4 belong to the same feature cluster, at this time, it is necessary to calculate the feature f 2 and f4 The symmetric uncertainty value SU between the class label C C For comparison, if SU C (f 4 )>SU C (f 2 ), select SU C The feature f with the largest value 4 The position number 3 is used as the value of the second dimension of Gbest_new. In summary, after Gbest=(1,3,2,0) executes the local search strategy, the improved Gbest_new=(0,3,2,2) is obtained.

[0177] Computational complexity analysis.

[0178] The main components of ISPSO-FS time complexity include: high-dimensional feature space clustering and evolutionary feature selection assisted by sample selection agent. Assume that the complete training sample set S_Whole contains M samples, D features, and the maximum number of iterations is T max , the size of the sample proxy set S is S_P. The time complexity of the algorithm is as follows: (1) The time complexity of high-dimensional feature space clustering is (2) The agent-assisted evolutionary feature selection based on sample selection mainly includes the construction strategy of the initial sample agent set, the sample agent set management strategy integrating historical information, and the agent-assisted integer particle swarm optimization algorithm. Its time complexity is mainly concentrated on the agent-assisted integer particle swarm optimization algorithm. Since the computational cost of the EA-based encapsulated FS method mainly comes from individual evaluation when processing data sets with high-dimensional features, the time complexity of the algorithm is mainly concentrated in the main loop. The main loop needs to execute T max ×N individual evaluations. Among them, 0.75×T max ×N individual evaluations used a classifier trained on a sample proxy set (i.e., a proxy model), 0.25×T max The ×N individual evaluations used a classifier trained based on the complete training sample set (i.e., the true fitness function).

[0179] For ease of understanding, assume that a single cross-validated k-NN is used to evaluate individuals. The time complexity of evaluating an individual based on the true fitness function is O(M×(M-1)). In view of this, the time complexity of evaluating an individual based on the agent model is O(S_P×(S_P-1)). Therefore, the agent-assisted integer particle swarm optimization algorithm performs T max The total computational complexity required for ×N individual evaluations is O((S_P×(S_P-1)×0.75+M×(M-1)×0.25))×T max× N, while the particle swarm optimization algorithm using the true fitness function to evaluate individuals executes T max The total computational complexity required for × N individual evaluations is O(M × (M - 1)) × T max × N. Since the size S_P of S is less than the size M of S_Whole, therefore, O((S_P × (S_P - 1) × 0.75 + M × (M - 1) × 0.25)) × T max × N < O(M × (M - 1)) × T max × N.

[0180] In summary, the time complexity of ISPSO - FS for evaluating individuals is less than that of evaluating individuals based on the complete training sample set.

[0181] The following is a further illustration of the present invention through specific examples.

[0182] Experiments and Analysis

[0183] Datasets

[0184] To verify the effectiveness of ISPSO - FS, it is applied to 10 typical datasets. Table 1 lists the basic information of the datasets. These datasets are downloaded from https: / / archive - beta.ics.uci.edu / and https: / / jundongl.github.io / scikit - feature / datasets.html, covering different sample sizes, feature dimensions, and number of classes. The number of samples ranges from 203 to 2000, the number of features ranges from 180 to 15154, and the number of classes ranges from 2 to 20. Their diversity can fully demonstrate the effectiveness and applicability of the algorithm of the present invention. To reduce the feature selection bias, double five - fold cross - validation is adopted in this embodiment. In the outer loop, the dataset is divided into five mutually exclusive subsets, which are taken turns as the test set, and the rest as the training set to evaluate the feature subsets obtained by the FS algorithm. In addition, during the search process of the feature selection algorithm, the training set is further divided into five subsets for inner - loop cross - validation to evaluate the fitness values of new candidate solutions. This method ensures an objective evaluation of the performance of feature subsets, reduces the bias caused by uneven data division, and improves the reliability and generalization ability of the results.

[0185] Table 1 Basic Information of Datasets

[0186]

[0187]

[0188] Experimental Settings.

[0189] To evaluate the performance of ISPSO-FS, two groups of experiments will be conducted. Experiment I aims to verify the effectiveness of the key strategies. Experiment II aims to confirm the superiority of ISPSO-FS by comparing it with a variety of advanced comparison algorithms. Among them, in Experiment II, 6 evolutionary FS algorithms are selected as comparison algorithms, and the descriptions of these comparison algorithms are as follows:

[0190] (1) An improved FS algorithm based on bare bones PSO (BBPSO); (2) Self-adaptive PSO FS algorithm for large-scale data (SaPSO); (3) Hybrid PSO FS with spiral-shaped mechanism (HPSO-SSM); (4) PSO and evolutionary multitasking-based FS algorithm (PSO-EMT); (5) Hybrid FS algorithm based on correlation-guided clustering and PSO (HFS-C-P); (6) A Simple, Fast, and Efficient Feature Selection Algorithm for High-Dimensional Data (SFE-PSO).

[0191] Support vector machine is an excellent classification algorithm and has been widely used in various FS methods. Therefore, the proposed algorithm and the six evolutionary FS algorithms all use SVM as the classifier to evaluate the pros and cons of individuals.

[0192] All experiments are conducted on a computer with an Intel(R) Xeon(R) Gold 5222 CPU@3.80GHz and implemented in the MATLAB 2022a environment. Each experiment runs 30 times, and the average value is taken as the final experimental result. For all evolutionary algorithms, the parameters are set to N = 50, T Max = 100. The parameters of the remaining comparison algorithms are set according to the recommendations in the corresponding literature, as shown in Table 2.

[0193] Table 2 Parameter settings

[0194]

[0195] Experiment I: Analysis of the effectiveness of the key strategies.

[0196] To evaluate the effectiveness of the key strategies in ISPSO-FS, four groups of ablation experiments were conducted to analyze the effectiveness of sample surrogate set construction, sample surrogate set update, extreme point update, and local search strategies respectively. Tables 3 - 6 show the performance of these algorithms on 10 datasets, including the average classification accuracy (Ac), the average number of selected features (d * ) and the average running time (t * ).

[0197] Effectiveness analysis of the initial sample surrogate set construction strategy guided by multi-source knowledge.

[0198] To comprehensively evaluate the effectiveness of the initial sample surrogate set construction strategy guided by multi-source knowledge, three variant algorithms (ISPSO-FS-1, ISPSO-FS-2, and ISPSO-FS-3) were designed for ablation experiments. The three variants respectively adopted representative sample selection strategies based on the crowding degree, similarity degree, and randomness of samples to generate surrogate units, which were directly used as the initial sample surrogate set. In subsequent updates, samples were randomly selected from the complete training sample set for supplementation, and the step of re-clustering to update the sample surrogate set was ignored. Table 3 presents the Ac, d * and t * values of ISPSO-FS-1, ISPSO-FS-2, ISPSO-FS-3, and ISPSO-FS on 10 datasets. It can be observed that: (1) In terms of classification accuracy, compared with the three variant algorithms, ISPSO-FS obtained the highest Ac value on all datasets. In particular, for PCAMA, compared with the average Ac value of the three variant algorithms, the increased value of the ISPSO-FS algorithm was 6.27%. (2) In terms of the average number of selected features, ISPSO-FS obtained the smallest d * value on DNA and COIL20. Slightly larger d * values were obtained on semeion, MFD, QSAR, MicroMass, PCAMA, and Ovarian. Larger d * values were obtained on GFE01 and Lung. Although the variant algorithms obtained slightly smaller or smaller d * values on most datasets, the Ac values they obtained were significantly lower than that of ISPSO-FS. (3) In terms of running time, ISPSO-FS obtained the smallest t * value on COIL20, the second smallest t * value on PCAMA, the third smallest t * value on QSAR and Ovarian, and larger t *Values. For all datasets, compared with the average Ac values of the three variant algorithms, the increased values are: 0.69%, 2.62%, 2.27%, 0.98%, 0.06%, 1.19%, 1.61%, 6.27%, 2.43%, and 1.64%, respectively. This is due to the fact that ISPSO-FS can select the most advantageous surrogate unit from multiple surrogate units during the surrogate construction process by utilizing the information in the evolutionary search process. Meanwhile, during the surrogate update process, it makes full use of the information of other surrogate units to ensure the surrogate quality. However, since multiple surrogate units need to be generated and the information of other surrogate units is utilized, it slightly increases the computational cost. To sum up, with the help of the initial sample surrogate set construction strategy, although ISPSO-FS slightly increases the computational cost, it significantly improves the classification performance of the algorithm.

[0199] Table 3 Comparison of ISPSO-FS with ISPSO-FS-1, ISPSO-FS-1, and ISPSO-FS-3 (t * Unit: minute)

[0200]

[0201] Effectiveness analysis of the sample surrogate set management strategy integrating historical information.

[0202] Through experiments, analyze the performance impact of the sample surrogate set management strategy integrating historical information on ISPSO-FS. Remove the sample surrogate set management strategy in ISPSO-FS as the comparison method, and mark this algorithm as ISPSO-FS-D. It can be seen from Table 4 that: (1) In terms of classification accuracy, ISPSO-FS obtained higher Ac values on all datasets. Especially on MicroMass, the Ac value obtained by the ISPSO-FS algorithm reached 98.13%, which was 3.41% higher than that of ISPSO-FS-D. (2) In terms of the number of selected features, on DNA, semeion, GFE01, MFD, COIL20, and QSAR, ISPSO-FS obtained smaller d * values. For the remaining 4 datasets, although ISPSO-FS-D obtained smaller d * values, the Ac values it obtained were significantly lower than those of ISPSO-FS. (3) In terms of running time, except for COIL20, ISPSO-FS obtained larger t * values on the remaining datasets. This is because this strategy needs to judge the update timing of the surrogate and execute the corresponding update rules during each iteration of the algorithm, which will increase the time cost of the algorithm. Although ISPSO-FS-D obtained smaller t *The value, however, obtained a relatively small Ac value on all datasets. In particular, for GFE01 and MicroMass, compared with ISPSO-FS-D, the Ac values obtained by ISPSO-FS increased by 4.54% and 3.41%. This strategy ensured the prediction accuracy of the surrogate model, thereby improving the classification performance of ISPSO-FS.

[0203] Table 4 Comparison between ISPSO-FS and ISPSO-FS-D (t * Unit: minute)

[0204]

[0205] Effectiveness analysis of the extreme point update strategy.

[0206] Verify the effectiveness of the extreme point update strategy through experiments. Use the traditional extreme point update strategy mentioned above to replace the update strategy proposed in ISPSO-FS as a variant algorithm, and name it ISPSO-FS-G. As can be seen from Table 5: (1) In terms of classification accuracy, ISPSO-FS obtained higher Ac values on all datasets. In particular, for GFE01, MicroMass, PCAMA, Lung, and Ovarian, the Ac values obtained by ISPSO-FS increased by 4.34%, 8.59%, 4.34%, 3.12%, and 6.89%. (2) In terms of the number of selected features, ISPSO-FS obtained smaller d * values on GFE01, MFD, and COIL20. Although ISPSO-FS-G obtained smaller d * values on the remaining 7 datasets, the Ac values it obtained on all datasets were lower. Especially for DNA and PCAMA, compared with ISPSO-FS-G, when the d * values were relatively close, the Ac values obtained by ISPSO-FS increased by 1.52% and 4.34%. (3) In terms of running time, ISPSO-FS obtained smaller t * values on semeion, MFD, and COIL20, but obtained larger t * values on the remaining datasets. This may be because this strategy needs to perform real evaluations on all global extreme points and re-approximate evaluations on the extreme points of some individuals, thus increasing the computational cost. However, on most datasets, ISPSO-FS can still maintain a similar computational cost while significantly improving the classification performance of the algorithm.

[0207] Table 5 Comparison between ISPSO-FS and ISPSO-FS-G (t * Unit: minute)

[0208]

[0209] Effectiveness analysis of the local search strategy.

[0210] The influence of the local search strategy on the algorithm performance was evaluated through experiments. The local search strategy in ISPSO-FS was removed as a comparative algorithm, denoted as ISPSO-FS-LS. As can be seen from Table 6: (1) In terms of classification accuracy, ISPSO-FS obtained higher Ac values on all datasets, which benefited from its embedding of the local search strategy to improve the global optimal solution. (2) In terms of the number of selected features, ISPSO-FS only obtained a smaller d * value on GFE01. Although ISPSO-FS-LS obtained smaller d * values on the remaining 9 datasets, its Ac values obtained on all datasets were lower. This may be because this strategy can find some valuable features from the unselected feature set and add them to the optimal feature subset, which not only increases the number of selected features but also improves the classification performance. In addition, when the d * values were similar, compared with ISPSO-FS-LS, ISPSO-FS also showed significantly improved classification performance on some datasets, further verifying the effectiveness of the local search strategy. (3) In terms of the average running time, ISPSO-FS-LS obtained smaller t * values on 8 out of 10 datasets. This is because ISPSO-FS-LS lacks processes such as feature addition / removal and real evaluation of the global optimal solution. Taking GFE01 and PCAMA as examples, the t * values obtained by ISPSO-FS were slightly higher, but the Ac values it obtained increased by 4.01% and 2.99%. Generally speaking, although the running time of ISPSO-FS increased with the help of this strategy, it showed significant advantages in classification accuracy, highlighting the ability of the local search strategy to discover valuable features.

[0211] Table 6 Comparison between ISPSO-FS and ISPSO-FS-LS (unit of t * : minute)

[0212]

[0213] Experiment II: Comparative experiment and analysis.

[0214] It was compared with 6 FS algorithms on 10 datasets. To evaluate the performance of the algorithms, the average classification accuracy (Ac), the average number of selected features (d * ), and the average running time (t * ) were used as evaluation indicators.

[0215] Classification accuracy

[0216] The ISPSO-FS was compared with six comparison algorithms using the classification accuracy. In addition, a t-test with a significance level of 0.5 was employed to investigate the statistical differences between the ISPSO-FS and each comparison algorithm. Table 7 presents the Ac values of the ISPSO-FS and the other six comparison algorithms on 10 datasets. Among them, the bold numbers indicate that the corresponding algorithms achieved the best results. “+” indicates that the ISPSO-FS is significantly better than the corresponding comparison algorithm, “=” indicates that there is no significant difference between the ISPSO-FS and the corresponding comparison algorithm, and “-” indicates that the corresponding comparison algorithm is significantly better than the ISPSO-FS.

[0217] As can be seen from Table 7, the ISPSO-FS obtained the highest Ac values on nine datasets (DNA, semeion, GFE01, MFD, COIL20, QSAR, MicroMass, PCAMA, and Ovarian), which were 96.19%, 94.86%, 82.53%, 98.42%, 100.00%, 90.80%, 98.13%, 86.86%, and 98.72%, respectively. For Lung, the ISPSO-FS obtained the second-highest Ac value. Further, from the results of their t-tests, it can be known that: (1) Compared with BBPSO and SFE-PSO, the ISPSO-FS achieved significantly better Ac values on 9 and 1 datasets, respectively, and there was no significant difference in the Ac value on 1 dataset. (2) Compared with HPSO-SSM and PSOEMT, the ISPSO-FS achieved significantly better Ac values on 8 and 2 datasets, respectively, and there was no significant difference in the Ac value on 2 datasets. (3) Except for the four datasets of semeion, MFD, COIL20, and QSAR, the Ac values obtained by the ISPSO-FS on the remaining datasets were significantly better than those of SaPSO. (4) On the five datasets of DNA, GFE01, QSAR, MicroMass, and Ovarian, the ISPSO-FS was significantly better than HFS-C-P. In addition, on the four datasets of semeion, MFD, COIL20, and PCAMA, there was no significant difference in the Ac values obtained by the above two algorithms. For Lung, compared with the ISPSO-FS, the Ac value obtained by the ISPSO-FS was slightly worse. Since the individual evaluation in the ISPSO-FS is based on the surrogate model trained by the sample surrogate set, there may be a little error in the evaluation results compared with HFS-CP. In summary, the ISPSO-FS can effectively identify high-quality feature subsets on most datasets and demonstrates strong competitiveness.

[0218] Table 7 Ac values (%) obtained by the ISPSO-FS and six comparison algorithms

[0219] Dataset BBPSO SaPSO HPSO-SSM PSOEMT HFS-C-P SFE-PSO ISPSO-FS DNA 81.71(+) 94.70(+) 94.45(+) 89.61(+) 95.48(+) 83.10(+) 96.19 semeion 92.64(+) 94.50(＝) 94.31(+) 94.76(＝) 94.40(＝) 93.79(+) 94.86 GFE01 77.50(+) 79.42(+) 76.24(+) 81.41(+) 80.95(+) 78.94(+) 82.53 MFD 97.20(+) 98.18(＝) 97.76(+) 96.75(+) 97.90(＝) 95.80(+) 98.42 COIL20 99.93(＝) 99.97(＝) 99.79(＝) 96.04(+) 99.73(＝) 99.58(＝) 100.00 QSAR 88.18(+) 90.53(＝) 89.28(+) 88.59(+) 89.75(+) 89.11(+) 90.80 MicroMass 94.20(+) 97.17(+) 97.25(+) 95.56(+) 97.25(+) 93.40(+) 98.13 PCAMA 77.43(+) 84.37(+) 83.79(+) 84.02(+) 86.53(＝) 77.44(+) 86.86 Lung 89.80(+) 81.52(+) 93.50(+) 90.13(+) 95.14(-) 90.69(+) 94.16 Ovarian 95.64(+) 93.02(+) 98.54(＝) 98.62(＝) 97.62(+) 98.07(+) 98.72

[0220] The number of selected features.

[0221] Compare the number of selected features of ISPSO-FS with six comparison algorithms. Table 8 shows the d * values obtained by ISPSO-FS and the comparison algorithms on 10 datasets, where A_d * represents the average value of the d * values obtained by the algorithm on all datasets. It can be observed that: (1) ISPSO-FS obtains the smallest d * values on semeion and QSAR. (2) For all datasets, ISPSO-FS obtains the second smallest A_d * value. Although SFE-PSO obtains the smallest A_d * value, as shown in Table 7, its classification performance is not ideal, and the Ac values obtained on all datasets are lower than those of ISPSO-FS. The main reason is that SFE-PSO deletes a large number of features in the first stage, resulting in the subsequent algorithm search space being based on the reduced feature space, thus selecting the smallest number of features and losing a lot of important information. (3) Compared with BBPSO, SaPSO, HPSO-SSM, and PSOEMT, ISPSO-FS obtains a significantly smaller A_d * value. At the same time, as can be seen from Table 7, ISPSO-FS also obtains significantly better Ac values. In summary, ISPSO-FS can maintain a small feature scale while obtaining a high classification accuracy, further demonstrating the superiority of the proposed algorithm.

[0222] Table 8 d * values obtained by ISPSO-FS and six comparison algorithms

[0223] Dataset BBPSO SaPSO HPSO-SSM PSOEMT HFS-C-P SFE-PSO ISPSO-FS DNA 38.7 155.1 122.2 10.0 19.8 35.8 17.7 semeion 52.5 169.6 62.2 16.0 8.7 11.4 5.8 GFE01 66.9 143.1 106.3 111.0 104.4 24.0 101.9 MFD 140.4 493.5 239.8 102.0 102.3 95.8 105.0 COIL20 210.9 735.2 174.8 118.0 161.0 84.7 167.8 QSAR 216.17 873.0 451.2 140.0 74.2 103.7 59.1 MicroMass 266.8 1102.9 671.1 110.0 511.9 194.5 453.4 PCAMA 632.0 2793.8 1985.9 101.0 64.4 520.1 68.2 Lung 2657.6 5018.8 1064 1557.0 609.3 200.0 632.5 Ovarian 3619.0 6075.5 377.3 1473.0 14.0 183.0 27.2 <![CDATA[A_d * > 790.1 1756.1 525.5 373.8 167.0 145.3 163.9

[0224] Running time.

[0225] Compare the running time of ISPSO-FS with six comparison algorithms to evaluate the relationship between performance improvement and computational cost. Table 9 shows the t * values obtained by ISPSO-FS and the comparison algorithms on 10 datasets, where [·] represents the rank of the t * values obtained by each algorithm on the same dataset, and "Avg.ranks" represents the average value of the ranks of the t * values obtained by a certain algorithm on all datasets. It can be seen that: (1) Except for COIL20, ISPSO-FS obtains the smallest t *Value. (2) For all datasets, compared with all the comparative algorithms, ISPSO-FS obtained the smallest Avg.ranks value. (3) For COIL20, compared with BBPSO and PSOEMT, ISPSO-FS obtained a smaller t * Value. Although the t * Value obtained by ISPSO-FS is greater than that of SaPSO, HPSO-SSM, HFS-C-P and SFE-PSO, the Ac values obtained by it are all higher than those of these 4 comparative algorithms. (4) HFS-C-P obtained the second smallest Avg.ranks value, because the clustering technique adopted by HFS-C-P can significantly reduce the search space of the evolutionary algorithm. However, its Avg.ranks value is still significantly greater than that of ISPSO-FS, because HFS-C-P still trains the classification model based on the complete training sample set to evaluate individuals, thus inevitably increasing the computational cost. Therefore, by combining the clustering technique with the idea of sample surrogate set assistance, ISPSO-FS successfully reduces the computational cost of the algorithm and shows high competitiveness in terms of classification performance, which is an excellent FS algorithm.

[0226] Table 9 t * Values (unit: minute) obtained by ISPSO-FS and 6 comparative algorithms

[0227] Dataset BBPSO SaPSO HPSO-SSM PSOEMT HFS-C-P SFE-PSO ISPSO-FS DNA 104.72[4] 121.61[5] 151.61[6] 281.33[7] 23.16[2] 77.57[3] 13.61[1] semeion 53.5[5] 42.69[4] 63.50[6] 182.60[7] 6.97[2] 16.63[3] 4.09[1] GFE01 38.77[4] 39.25[5] 41.23[6] 74.83[7] 31.99[3] 17.62[2] 14.55[1] MFD 107.70[6] 103.65[5] 103.06[4] 297.28[7] 34.25[2] 38.88[3] 22.16[1] COIL20 218.16[6] 194.90[4] 180.83[2] 1026.88[7] 185.15[3] 75.05[1] 204.48[5] QSAR 204.10[5] 251.97[6] 170.09[4] 449.67[7] 49.40[2] 60.10[3] 22.61[1] MicroMass 29.03[4] 34.31[5] 35.42[6] 56.45[7] 21.75[3] 13.09[2] 12.21[1] PCAMA 376.85[4] 516.16[5] 526.07[6] 1163.50[7] 31.92[2] 202.94[3] 25.10[1] Lung 148.31[6] 123.22[5] 122.91[4] 412.49[7] 11.40[3] 9.22[2] 7.50[1] Ovarian 221.91[6] 177.50[4] 187.12[5] 627.07[7] 4.59[2] 7.77[3] 2.21[1] Avg.ranks 5 4.8 4.9 7 2.4 2.5 1.4

[0228] Therefore, by adopting the above high-dimensional data feature selection method integrating sample surrogate and particle swarm optimization, the present invention can obtain a better classification effect at a lower computational cost.

[0229] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that they can still modify or equivalently replace the technical solutions of the present invention, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A high-dimensional data feature selection method integrating sample proxy and particle swarm optimization, characterized in that: The following steps are involved: Step S1, dividing the original data set into a complete training sample set and a test data set; Step S2, executing a high-dimensional feature space clustering module to divide the original feature space into a number of feature clusters; Step S3, executing the agent-assisted evolutionary feature selection module based on sample selection, using the IBPSO algorithm to search for representative features in each feature cluster, outputting the optimal feature subset and applying it to the test set data; Step S31, initializing the population, and the initialized population is used as the input of the initial sample proxy set construction module and the population update module guided by multi-source knowledge; Step S32, executing the initial sample proxy set construction module guided by multi-source knowledge, constructing a proxy unit set S_Sample based on the representative sample selection guided by multi-source knowledge, executing the initial sample proxy set construction strategy based on the proxy unit set S_Sample, and constructing the initial sample proxy set S0; Step S33, execute the population update module in a loop, use the proxy model trained by the initial sample proxy set S0 to calculate the approximate fitness value of the particle, initialize the individual extreme value Pbest and the global extreme value Gbest of each particle, use the individual extreme value point update mechanism and the global extreme value point update mechanism to update the individual extreme value Pbest and the global extreme value Gbest respectively, and perform a local search strategy on the global extreme value Gbest based on the local search strategy guided by feature clustering to update the individual position; In the process of evolutionary search, the sample proxy set is managed through the sample proxy set management strategy module which integrates historical information.

2. A high-dimensional data feature selection method integrating sample proxy and particle swarm optimization according to claim 1, characterized in that: In the IBPSO algorithm, the particle update rules are as follows: in, represents the position of the i-th particle in the j-th dimension in the t+1th generation; It represents the individual historical optimal position of the i-th particle in the j-th dimension in the t-th generation; Indicates the global optimal position of the entire population in the jth dimension in the tth generation; G(0,1) represents the standard Gaussian distribution; Represents the floor function.

3. The high-dimensional data feature selection method integrating sample proxy and particle swarm optimization according to claim 2, characterized in that: In step S32, the proxy unit set S_Sample = {S cr ,S sim ,S rand }, where S cr represents the agent unit selected based on the degree of crowding; S sim represents the agent unit selected based on similarity; S rand represents an agent unit selected based on randomness; Agent unit S is selected based on the degree of crowding cr The calculation is as follows: Calculate sample x j The crowding degree value CR j : CR j =η×Var j +(1-η)×OL j (2); Among them, η is a parameter; Var j Indicates the degree of fluctuation; OL j Indicates the degree of outlier; Var j The calculation formula is as follows: Where m represents the number of samples; D j,h represents the Euclidean distance from the jth sample to the hth sample; OL j The calculation formula is as follows: Based on the crowding degree value CR j For sample cluster SC i Sort the samples in descending order and select the first samples are added to the proxy unit S selected based on the degree of crowding cr ; Where P represents the total number of samples contained in the current sample proxy set; K represents the number of true class labels in the complete training sample set; Agent unit S is selected based on similarity sim The calculation is as follows: When the sample cluster SC i When all samples in belong to the same category, i Random selection samples as representative samples and add them to the proxy unit S selected based on similarity sim ; When the sample cluster SC i When all samples in do not belong to the same category, calculate the sample x j The similarity value SIM j , based on the similarity value SIM j For sample cluster SC i Sort the samples in descending order and select the first samples are added to the proxy unit S selected based on similarity sim ; Similarity value SIM j The calculation formula is as follows: in, Represents sample cluster SC i The center sample μ of the cth category in c The value on the i-th feature; Represents sample x j The value on the i-th feature; c_k represents the sample cluster SC i The number of categories included; Agent unit S based on random selection rand The calculation is as follows: For each sample cluster with a single class label, we randomly select samples and add them to the randomly selected agent S rand ; For sample clusters with multiple labels, the number of samples is dynamically adjusted according to the ratio r of the number of samples corresponding to each type of label in each cluster, and a random sample is selected from each category. samples and add them to the randomly selected agent S rand .

4. The high-dimensional data feature selection method integrating sample proxy and particle swarm optimization according to claim 3 is characterized in that: In step S32, the initial sample proxy set construction strategy is executed based on the proxy unit set S_Sample to construct the initial sample proxy set S0. The specific operations are as follows: First, use S_Sample = {S cr ,S sim ,S rand The agent model trained by the agent unit in} evaluates the individual, determines the optimal individual under each agent unit, and constitutes the optimal individual set S_X best ; Then, the approximate fitness value of the optimal individual is calculated to form an approximate fitness value vector. a cr , a sim and a rand Respectively represent the use of S cr ,S sim ,S rand For S_X best The approximate fitness value vector constructed by approximate evaluation of the individuals in ; represents the use of agent units S selected based on the degree of congestion cr The trained proxy model approximates the evaluation S_X best The approximate fitness value obtained by the first individual in ; represents the use of agent units S selected based on the degree of congestion cr The trained proxy model approximates the evaluation S_X best The approximate fitness value obtained by the second individual in ; represents the use of agent units S selected based on the degree of congestion cr The trained proxy model approximates the evaluation S_X best The approximate fitness value obtained by the third individual in ; Represents the use of proxy units S selected based on similarity sim The trained proxy model approximates the evaluation S_X best The approximate fitness value obtained by the first individual in ; Represents the use of proxy units S selected based on similarity sim The trained proxy model approximates the evaluation S_X best The approximate fitness value obtained by the second individual in ; Represents the use of proxy units S selected based on similarity sim The trained proxy model approximates the evaluation S_X best The approximate fitness value obtained by the third individual in ; Represents the use of agent units S based on random selection rand The trained proxy model approximates the evaluation S_X best The approximate fitness value obtained by the first individual in ; Represents the use of agent units S based on random selection rand The trained proxy model approximates the evaluation S_X best The approximate fitness value obtained by the second individual in ; Represents the use of agent units S based on random selection rand The trained proxy model approximates the evaluation S_X best The approximate fitness value obtained by the third individual in ; At the same time, the classification model trained using the full training sample set is evaluated S_X best The individuals in the get the true fitness value vector Indicates that the complete training sample set is used to evaluate S_X best The true fitness value of the first individual in ; Indicates that the complete training sample set is used to evaluate S_X best The true fitness value obtained by the second individual in ; Indicates that the complete training sample set is used to evaluate S_X best The true fitness value obtained by the third individual in ; Finally, measure the true fitness value vector o and the approximate fitness value vector a cr 、a sim and a rand The similarity between them is calculated, and the proxy unit corresponding to the approximate fitness value vector with the largest similarity is selected as the initial sample proxy set S0.

5. The high-dimensional data feature selection method integrating sample proxy and particle swarm optimization according to claim 4, characterized in that: In step S33, individual extreme point update mechanism: By using the historical information of particles and individual extreme points, the relationship between the current generation of particles and their individual extreme points is measured, so as to determine the position of the individual extreme point and its fitness value in the t generation; First, a judgment threshold is given as follows: Among them, f_s t-1 (Pbest i t-1 ) is Pbest at generation t-1 i t-1 Based on the sample proxy set S t-1 The fitness value of t (X i t ) is in generation t i t Based on the sample proxy set S t The fitness value of |·| means taking the absolute value; θ f represents the threshold; N represents the population size; Pbest i t-1 represents the i-th individual in the individual extreme value Pbest at generation t-1; X i t represents the i-th individual in the current population X at the tth generation; For each particle, calculate the t-th generation particle based on the sample proxy set S t The fitness value and the extreme point of the t-1 generation individual are based on the sample proxy set S t-1 The difference in fitness values ​​Diff = f_s t (X i t )-f_s t-1 (Pbest i t-1 ); Based on θ f and Diff, the update mechanism of individual extreme points is divided into the following three cases: Case 1: When Diff>θ f When the individual extreme point position Pbest is updated i t =X i t , and its fitness value f_s t (Pbest i t )=f_s t (X i t ); Case 2: When -Diff>θ f When the individual extreme point position Pbest is updated i t =Pbest i t-1 , and its fitness value f_s t (Pbest i t )=f_s t-1 (Pbest i t-1 ); Case 3: When |Diff|≤θ f When re-based on S t Calculate Pbest i t-1 The fitness value f_s t (Pbest i t-1 ): If f_s t (X i t ) is greater than f_s t (Pbest i t-1 ), then update the individual extreme point position Pbest i t =X i t , and update its fitness value f_s t (Pbest i t )=f_s t (X i t ); If f_s t (X i t ) is less than f_s t (Pbest i t-1 ), then update the individual extreme point position Pbest i t =Pbest i t-1 , and update its fitness value f_s t (Pbest i t )=f_s t (Pbest i t-1 ); If the fitness values ​​of the two are equal, then according to X i t and Pbest i t-1 The number of features included, select the one with fewer features as the individual extreme point in generation t.

6. The high-dimensional data feature selection method integrating sample proxy and particle swarm optimization according to claim 5, characterized in that: In step S33, the local search strategy guided by feature clustering is as follows: First, the clustering results of the feature space clustering stage are stored as knowledge in the feature cluster set G; Next, Gbest is mapped to the feature space, and the importance and relevance-redundancy of the features are further utilized to perform feature deletion and addition operations in the feature space to obtain an improved feature subset F best_local ; Finally, based on the feature cluster set G, F is transformed into best_local Express it and obtain the improved Gbest; Among them, in the representation process, if F best_local There are multiple features belonging to the same feature cluster, and the most important features are further selected for representation based on the importance of the features; The calculation formula of relevance-redundancy is as follows: Where C is the category label; |F| is the size of the feature subset; I(f j ,C) is the feature f j The mutual information value between the class label C; I(f j ,f i ) is the feature f j With feature f i The mutual information value between .

7. The high-dimensional data feature selection method integrating sample proxy and particle swarm optimization according to claim 6, characterized in that: In the process of evolutionary search, the sample proxy set management strategy module is used to integrate historical information to achieve management of the sample proxy set, including two update timings, namely update timing 1 and update timing 2; Among them, update timing 1: When the model relative prediction error value is greater than 0.01, update timing 1 is met and update rule 1 is executed; The calculation formula of the model relative prediction error value is as follows: Among them, Gbest t Indicates the global extreme point obtained by the algorithm at the tth iteration; F'_sur(Gbest t ) and F_real(Gbest t ) represent Gbest t The predicted fitness value and the true fitness value; When the global extreme point has not changed for t0 consecutive generations, that is, update timing 2 is met, update rule 2 is executed.

8. The high-dimensional data feature selection method integrating sample proxy and particle swarm optimization according to claim 7, characterized in that: Update rule 1 is as follows: Current sample proxy set S t-1 It evolves from one of the proxy units of S_Sample, and deletes the proxy unit from S_Sample; Then, a random selection is made from the remaining proxy units in S_Sample and recorded as S tem ; Finally, based on the optimal feature subset obtained in the current evolutionary search process, S tem Perform classification, filter out misclassified samples, and add these samples as representative samples to S t-1 In the process, duplicate samples are removed.

9. The high-dimensional data feature selection method integrating sample proxy and particle swarm optimization according to claim 8, characterized in that: Update rule 2 is as follows: Current sample proxy set S t-1 It is evolved from one of the proxy units in S_Sample. The other two proxy units are used to train the proxy model respectively, and the current population is approximately evaluated to obtain the optimal feature subsets corresponding to the proxy units respectively, and then compared with F Gbest The union of the optimal feature subsets F union , based on F union The original entire sample space is re-clustered, and the initial sample proxy set construction strategy is executed to generate a new sample proxy set.

Images