Optimization algorithm based on rough set and cockroach cluster

Abstract

The invention belongs to the technical field of data algorithms, and particularly relates to an optimization algorithm based on a rough set and a cockroach cluster, and the optimization algorithm comprises the following steps: 1, obtaining a rough set; step 2, optimizing cockroach groups; step 3, feature filtering; 4, expressing a solution and performing population initialization in combination with reverse learning; step 5, performing a domain search strategy based on pbest and gbest; step 6, performing taboo search; and 7, evaluating a feature subset and a population. According to the method, representative attributes can be selected from an incoming attribute set according to mutual information of condition attributes and decision attributes of a rough set, a generated new rough decision system is introduced into a cockroach swarm optimization algorithm, random selection is performed to find an optimal feature subset, features with high mutual information are selected from the incoming feature set, and the optimal feature subset is obtained. Under the condition that good classification precision is obtained, calculation operation time can be effectively shortened, and performance consumption is reduced.

Description

Technical Field

[0001] The invention belongs to the technical field of data algorithms, and in particular relates to an optimization algorithm based on rough sets and cockroach clusters. Background Art

[0002] Big data is growing explosively in all walks of life. A large amount of data brings data uncertainty and redundancy, which brings huge challenges to almost all algorithms in the field of machine learning and data mining, and often leads to poor results when performing learning tasks. Traditional data mining algorithms will show pressure when processing big data because they require more storage space and computing performance. Therefore, many scholars have proposed various feature selection algorithms to remove irrelevant and redundant features in the data set and obtain key feature subsets. The key feature subsets have a great improvement in the computing time and classification accuracy of subsequent data set processing. Therefore, an effective feature selection algorithm is very necessary to select key information features from the original data.

[0003] Existing data mining algorithms take a long time to calculate and consume a lot of performance for high-dimensional data sets, and swarm intelligence algorithms select too many irrelevant and redundant features. To this end, we propose an optimization algorithm based on rough sets and cockroach swarming to solve the above problems. Summary of the invention

[0004] The purpose of the present invention is to provide an optimization algorithm based on rough sets and cockroach swarms, which can select representative attributes from an input attribute set according to the mutual information between the conditional attributes and decision attributes of the rough set, and pass the generated new rough decision system into the cockroach swarm optimization algorithm for random selection to find the optimal feature subset, and select features with high mutual information from the input feature set. While obtaining better classification accuracy, the calculation running time can be effectively reduced and the performance consumption can be reduced.

[0005] The technical solution adopted by the present invention is as follows:

[0006] An optimization algorithm based on rough sets and cockroach clustering, the optimization algorithm comprises the following steps:

[0007] Step 1. Rough set;

[0008] Step 2. Cockroach colony optimization;

[0009] Step 3. Feature filtering;

[0010] Step 4. Expression of solution and population initialization combined with reverse learning;

[0011] Step 5. Field search strategy based on pbest and gbest;

[0012] Step 6. Taboo search;

[0013] Step 7. Feature subset and population evaluation;

[0014] Step 8. Dynamic threshold adjustment;

[0015] Step 9. Ensemble learning strategy.

[0016] In a preferred solution, the rough set describes knowledge as an information system, which consists of row vectors and column vectors. The row vectors represent objects in the domain, and the column vectors represent different attributes. There is a type of information system called a decision table, which consists of conditional attributes and decision attributes. Each attribute has a corresponding attribute value. In a decision table, unnecessary and redundant attributes are removed while ensuring that the classification capability of the information system remains unchanged.

[0017] Assume that the decision table is a non-empty finite set (U, C∪D), where U = {x1, x2, ..., xn} is called the domain, C = {a1, a2, ..., an} is called the condition attribute set, and D is the decision attribute set. As shown in Table 1, the decision table consists of 4 samples, 3 condition attributes {a1, a2, a3} and one decision attribute {d};

[0018] Table 1 Example of decision table

[0019]

[0020] Before calculating the mutual information, it is necessary to calculate the information entropy, conditional entropy or joint entropy of the discrete random variable a. First, the definition of information entropy is given;

[0021] Shannon defined information entropy as the probability of occurrence of discrete random events. The larger the probability, the more opportunities for occurrence and the smaller the uncertainty; conversely, the uncertainty is large. Let information entropy be H(X), and define it as follows:

[0022]

[0023] In formula (1), |U| represents the cardinality of U, which represents the probability of occurrence of discrete values ​​of variable x. Assuming variables a1 and a2,

[0024] a1=[1,0,1,1]; a2=[0,1,1,0];

[0025] According to probability

[0026] p(a1=0)=1 / 4; p(a1=1)=3 / 4;

[0027] p(a2=0)=2 / 4; p(a2=1)=2 / 4;

[0028]

[0029] Similarly, H(a2)=0.6931;

[0030] The joint information entropy of two variables X and Y is defined as:

[0031]

[0032] where x and y are specific values ​​of X and Y, respectively, and P(x,y) is the joint probability of these values ​​occurring together;

[0033] Table 2 Example:

[0034]

[0035] According to Table 2, P(a1,a2)

[0036] P(a1=0,a2=0)=0

[0037] P(a1=0,a2=1)=1 / 4

[0038] P(a1=1,a2=0)=2 / 4

[0039] P(a1=1,a2=0)=1 / 4

[0040] Calculate H(a1,a2) and get:

[0041]

[0042] Conditional entropy H(Y|X) represents the uncertainty of random variable Y under the condition of known random variable X, which is defined as:

[0043] H(Y|X)=-Σ x＝X P(x)Σ y＝Y P(y|x)log2P(y|x)

[0044] ⑶

[0045] According to the occurrence probability of (a1, a2):

[0046] Table 3 Example:

[0047]

[0048] Where P(a1=1|a2=1) represents the probability that a1=0 when a2=0, which is calculated as follows:

[0049]

[0050] Similarly, P(a1=0|a2=0), P(a1=0|a2=1), P(a1=1|a2=0)

[0051] Then the conditional entropy H(a1|a2) is calculated as:

[0052]

[0053] Mutual information is a useful information measure. It can be regarded as the amount of information contained in the random variable X about the random variable Y. Suppose the joint distribution of two random variables (X, Y) is p(x, y), and the marginal distribution is p(x), p(y). The mutual information I(X, Y) is the relative entropy of the joint distribution p(x)p(y) and the marginal distribution p(x), p(y), that is:

[0054]

[0055] The information calculation formula is:

[0056] I(X;Y)=H(X)+H(Y)-H(X,Y)⑸.

[0058] In a preferred embodiment, the cockroach swarm optimization is established by imitating the chasing behavior of individual cockroaches in searching for the global optimum, and the optimal solution is searched through continuous cooperation and information sharing among the swarms;

[0059] After a set of solutions are randomly initialized, the optimal solution is searched iteratively. The cockroaches continuously update their positions toward the two extreme values ​​pbest and gbest through aggregation behavior. The update rules are as follows:

[0060]

[0061] Where yr (r = 1, 2, 3, ..., n) is the position of the rth cockroach, a represents the step length, which is a fixed value, rand takes any value between (0, 1), pr and pg are the optimal position of the rth cockroach and the global optimal cockroach position respectively, and the calculation formula is:

[0062]

[0063] visual represents the cockroach's visual field, which is a constant, s = 1, 2, 3, ... n. After each round of foraging, each individual is randomly dispersed through the following model to maintain the diversity of the current individual;

[0064] y r =y r +rand(1,E),r=1,2,...,N ⑻

[0065] Among them, rand(1,E) is an E-dimensional (problem space dimension) random vector that can be set within a certain range. Finally, the global optimum is used to replace the randomly selected individuals. The model is as follows:

[0066] y n =ρ g , n=1,2,...,N ⑼。

[0067] In a preferred solution, the feature filtering is to use the mutual information of the rough set to perform preliminary filtering on the feature set. For two given features a and d, the difference between the information entropy H(a) of feature a and its conditional information entropy H(a|d) is less than a threshold. The input conditional attributes do not need to participate in feature selection and are filtered to select representative features.

[0068] In a preferred embodiment, the expression of the solution and the population initialization combined with reverse learning are as follows: assuming that the population size is N, the feature dimension is D, and a D-bit binary string is used to represent a solution. The binary string corresponds to a feature set, 1 indicates that the feature is selected, and 0 indicates that the feature is not selected. In this way, an iterative feature subset is obtained. There is a feature set {a, b, c}. If the final solution is {1, 0, 1}, the corresponding feature subset is {a, c}. The reverse population initialization is applied to the RSICSO algorithm, and an N*D-dimensional feasible solution matrix X is randomly created. For the value of any solution i (i = 1, 2, ..., N) in the g-th generation population, X is used. g i It represents a random number between 0 and 1. The corresponding reverse solution is generated according to formula (10). The solution is selected according to formula (11). The iterative optimal solution of each feasible solution is recorded as pbest, and the global optimal solution is recorded as gbest.

[0069] X o p=1-X⑽

[0070]

[0071] In a preferred solution, the domain search strategy based on pbest and gbest is that in each iteration of the algorithm, cockroach i will continuously update its position according to pbest and gbest. The aggregation behavior formula of the cockroach swarm algorithm is modified and applied to the algorithm model. A solution is randomly selected. According to the fitness of the i-th cockroach, if the fitness is small, a local search is performed relative to pbest. If the fitness is large, a global search is performed relative to gbest to generate a new solution. The formula is as follows:

[0072]

[0073] In the formula, ρ iis the optimal solution for the current individual of the i-th cockroach, threshold is the fitness threshold, y k , k∈[1,D] is the kth solution selected randomly.

[0074] In a preferred solution, the taboo search uses some mechanisms to avoid repeated searches, thereby ensuring the construction of a global solution. In RSICSO, a taboo search is used for cockroaches. First, a taboo table is generated to save the initial solution. Then, in the iterative process, a new solution is generated according to the search strategy. Then, according to the threshold and the taboo table, the solution is mutated or gradually changed to generate a new solution, which is then written into the taboo table, evaluated, and the algorithm is continued.

[0075] In a preferred embodiment, the feature subset and population evaluation is evaluated by using the prediction rate of the kNN classifier in the relevant standard. The classification model and feature selection are interdependent, and the obtained feature subset has a good accuracy rate. For each individual in the population, the fitness value is the prediction rate of the classifier, and the fitness value is used to evaluate the quality of the corresponding solution of the cockroach. If the fitness value of cockroach i during the search process is greater than the fitness value of cockroach i before, the individual optimal solution, individual fitness and current cockroach position are updated. In the feature subset and population evaluation, all solutions are evaluated using the classifier kNN (k=7), the fitness and the global optimal solution are updated, and N better solutions are selected according to the fitness to form an initial population. In the feature subset and population evaluation, the fitness of the solution is judged to mutate or gradually change the solution, and then a new solution is generated.

[0076] In a preferred embodiment, the dynamic threshold is adjusted to set an initial fitness threshold threshold_init at the beginning of the algorithm, which is set according to the specific characteristics of the problem, or initialized according to empirical values ​​or experimental results;

[0077] threshold=threshold_init;

[0078] At each iteration, the diversity of the current population is calculated:

[0079] diversity=1 / N*sum(||y_i-mean(Y)||)for i=1to N;

[0080] Where N is the population size, y_i is the position of the i-th cockroach, and mean(Y) is the mean value of the population position;

[0081] Dynamically adjust the fitness threshold according to changes in population diversity;

[0082] The population diversity has decreased significantly, so the threshold is relaxed to increase the exploration ability;

[0083] threshold=threshold*adjust_factor;

[0084] The diversity of populations is maintained well, the threshold is tightened, and the utilization capacity is increased;

[0085] threshold=threshold / adjust_factor;

[0086] To avoid the threshold being too large or too small, set the upper and lower limits of the threshold:

[0087] threshold=max(threshold_min,min(threshold_max,threshold));

[0088] Among them, threshold_min and threshold_max are the minimum and maximum values ​​of the threshold respectively;

[0089] Pseudo code for dynamic threshold adjustment:

[0090]

[0091]

[0092] In a preferred embodiment, the ensemble learning strategy is that after the population initialization of the cockroach colony optimization algorithm is completed, each individual corresponds to a feature subset, and each feature subset is trained and predicted using each classifier. Each classifier can be evaluated multiple times by cross-validation or multiple random data segmentation, and the prediction accuracy of each evaluation is recorded;

[0093]

[0094] For each feature subset, the prediction results of multiple classifiers are combined to calculate the comprehensive fitness value. The prediction accuracy of different classifiers is combined into one fitness value by weighted average.

[0095]

[0096]

[0097] According to the comprehensive fitness value, update the fitness value of each individual.

[0098] The technical effects achieved by the present invention are:

[0099] The problem of long calculation time and high performance consumption of swarm intelligence algorithm in high-dimensional data sets is improved; the problem of too many irrelevant and redundant features selected by swarm intelligence algorithm is improved. The weakly relevant and redundant features are filtered out by calculating the mutual information of conditional feature subsets and decision features, leaving the key features, and the fitness function of the swarm algorithm is defined in combination with the selected feature subsets. In order to enhance the global search capability and make RSICSO get rid of the local optimum, the CSO algorithm is modified by adopting reverse learning, taboo search and intergenerational random allocation mechanism. The proposed model is tested on 10 benchmark data sets and compared with 3 representative swarm intelligence feature selection methods. The cardinality and running time of the selected feature subsets are greatly reduced while ensuring good classification accuracy, especially on high-dimensional data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Figure 1 It is a schematic diagram of a first algorithm for feature filtering of an optimization algorithm based on rough sets and cockroach clustering according to the present invention;

[0101] Figure 2 It is a schematic diagram of a taboo search process of an optimization algorithm based on rough sets and cockroach clustering according to the present invention;

[0102] Figure 3 It is a schematic diagram of a second algorithm for feature subset and population evaluation of an optimization algorithm based on rough sets and cockroach swarming of the present invention;

[0103] Figure 4 It is a schematic diagram of the second algorithm operation flow of the feature subset and population evaluation of the optimization algorithm based on rough sets and cockroach clusters of the present invention. DETAILED DESCRIPTION

[0104] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0105] See also Figure 1-4 As shown, the present invention provides an optimization algorithm based on rough sets and cockroach clustering, and the optimization algorithm includes the following steps:

[0106] Step 1. Rough set;

[0107] Step 2. Cockroach colony optimization;

[0108] Step 3. Feature filtering;

[0109] Step 4. Expression of solution and population initialization combined with reverse learning;

[0110] Step 5. Field search strategy based on pbest and gbest;

[0111] Step 6. Taboo search;

[0112] Step 7. Feature subset and population evaluation;

[0113] Step 8. Dynamic threshold adjustment;

[0114] Step 9. Ensemble learning strategy.

[0115] Rough set describes knowledge as an information system, which consists of row vectors and column vectors. Row vectors represent objects in the domain, and column vectors represent different attributes. There is a type of information system called decision table, which consists of conditional attributes and decision attributes. Each attribute has a corresponding attribute value. In a decision table, unnecessary and redundant attributes are removed while ensuring that the classification ability of the information system remains unchanged.

[0116] Assume that the decision table is a non-empty finite set (U, C∪D), where U = {x1, x2, ..., xn} is called the domain, C = {a1, a2, ..., an} is called the condition attribute set, and D is the decision attribute set. As shown in Table 1, the decision table consists of 4 samples, 3 condition attributes {a1, a2, a3} and one decision attribute {d};

[0117] Table 1 Example of decision table

[0118]

[0119] Before calculating the mutual information, it is necessary to calculate the information entropy, conditional entropy or joint entropy of the discrete random variable a. First, the definition of information entropy is given;

[0120] Shannon defined information entropy as the probability of occurrence of discrete random events. The larger the probability, the more opportunities for occurrence and the smaller the uncertainty; conversely, the uncertainty is large. Let information entropy be H(X), and define it as follows:

[0121]

[0122] In formula (1), |U| represents the cardinality of U, which represents the probability of occurrence of discrete values ​​of variable x. Assuming variables a1 and a2,

[0123] a1=[1,0,1,1]; a2=[0,1,1,0];

[0124] According to probability

[0125] p(a1=0)=1 / 4; p(a1=1)=3 / 4;

[0126] p(a2=0)=2 / 4; p(a2=1)=2 / 4;

[0127]

[0128] Similarly, H(a2)=0.6931.

[0129] The joint information entropy of two variables X and Y is defined as:

[0130]

[0131] where x and y are specific values ​​of X and Y, respectively, and P(x,y) is the joint probability of these values ​​occurring together;

[0132] Table 2 Example:

[0133]

[0134] According to Table 2, P(a1,a2)

[0135] P(a1=0,a2=0)=0

[0136] P(a1=0,a2=1)=1 / 4

[0137] P(a1=1,a2=0)=2 / 4

[0138] P(a1=1,a2=0)=1 / 4

[0139] Calculate H(a1,a2) and get:

[0140]

[0141] Conditional entropy H(Y|X) represents the uncertainty of random variable Y under the condition of known random variable X, which is defined as:

[0142] H(Y|X)=-Σ x＝X P(x)Σ y＝Y P(y|x)log2P(y|x) ⑶

[0144] According to the occurrence probability of (a1, a2):

[0145] Table 3 Example:

[0146]

[0147] Where P(a1=1|a2=1) represents the probability that a1=0 when a2=0, which is calculated as follows:

[0148]

[0149] Similarly, P(a1=0|a2=0), P(a1=0|a2=1), P(a1=1|a2=0)

[0150] Then the conditional entropy H(a1|a2) is calculated as:

[0151]

[0152] Mutual information is a useful information measure. It can be regarded as the amount of information contained in the random variable X about the random variable Y. Suppose the joint distribution of two random variables (X, Y) is p(x, y), and the marginal distribution is p(x), p(y). The mutual information I(X, Y) is the relative entropy of the joint distribution p(x)p(y) and the marginal distribution p(x), p(y), that is:

[0153]

[0154] The information calculation formula is:

[0155] I(X;Y)=H(X)+H(Y)-H(X,Y)⑸.

[0157] Cockroach swarm optimization is established by imitating the pursuit behavior of individual cockroaches in search of the global optimal value, and searches for the optimal solution through continuous cooperation and information sharing among the swarm;

[0158] After a set of solutions are randomly initialized, the optimal solution is searched iteratively. The cockroaches continuously update their positions toward the two extreme values ​​pbest and gbest through aggregation behavior. The update rules are as follows:

[0159]

[0160] Where yr (r = 1, 2, 3, ..., n) is the position of the rth cockroach, a represents the step length, which is a fixed value, rand takes any value between (0, 1), pr and pg are the optimal position of the rth cockroach and the global optimal cockroach position respectively, and the calculation formula is:

[0161]

[0162] visual represents the cockroach's visual field, which is a constant, s = 1, 2, 3, ... n. After each round of foraging, each individual is randomly dispersed through the following model to maintain the diversity of the current individual;

[0163] yr=yr+rand(1,E), r=1,2,...N ⑻

[0164] Where rand(1,E) is a random vector of dimension E (problem space dimension) that can be set within a certain range.

[0165] Finally, the global optimum is used to replace the randomly selected individuals, and the model is as follows:

[0166] y n =ρ g , n=1,2,...,N ⑼。

[0167] Feature filtering is to use the mutual information of rough sets to perform preliminary filtering on the feature set. For two given features a and d, the difference between the information entropy H(a) of feature a and its conditional information entropy H(a|d) is less than a threshold. The incoming conditional attributes do not need to participate in feature selection and are filtered to select representative features. The feature filtering process is based on the first algorithm.

[0168] The expression of the solution and the population initialization combined with reverse learning are as follows: Assume that the population size is N, the feature dimension is D, and a D-bit binary string is used to represent a solution. The binary string corresponds to the feature set. 1 indicates that the feature is selected, and 0 indicates that the feature is not selected. In this way, an iterative feature subset is obtained. There is a feature set {a, b, c}. If the final solution is {1, 0, 1}, the corresponding feature subset is {a, c}. The reverse population initialization is applied to the RSICSO algorithm. A N*D-dimensional feasible solution matrix X is randomly created. For the value of any solution i (i = 1, 2, ..., N) in the g-th generation population, X is used. g i It represents a random number between 0 and 1. The corresponding reverse solution is generated according to formula (10). The solution is selected according to formula (11). The iterative optimal solution of each feasible solution is recorded as pbest, and the global optimal solution is recorded as gbest.

[0169] X o p=1-X⑽

[0170]

[0171] The domain search strategy based on pbest and gbest is that in each iteration of the algorithm, cockroach i will continuously update its position according to pbest and gbest. The aggregation behavior formula of the cockroach swarm algorithm has been modified and applied to the algorithm model. A solution is randomly selected. According to the fitness of the i-th cockroach, if the fitness is small, a local search is performed relative to pbest. If the fitness is large, a global search is performed relative to gbest to generate a new solution. The formula is as follows:

[0172]

[0173] In the formula, ρ i is the optimal solution for the current individual of the i-th cockroach, threshold is the fitness threshold, y k , k∈[1,D] is the kth solution selected randomly.

[0174] Taboo search is to use some mechanisms to avoid repeated searches, so as to ensure the construction of the global solution. In RSICSO, taboo search is used for cockroaches. First, a taboo table is generated to save the initial solution. Then, in the iterative process, a new solution is generated according to the search strategy. Then, according to the threshold and taboo table, the solution is mutated or gradually changed to generate a new solution, which is then written into the taboo table, evaluated, and the algorithm continues to execute.

[0175] The feature subset and population evaluation is evaluated by the prediction rate of the kNN classifier in the relevant standards. The classification model and feature selection are interdependent, and the obtained feature subset has a good accuracy rate. For each individual in the population, the fitness value is the prediction rate of the classifier. The fitness value is used to evaluate the quality of the corresponding solution of the cockroach. If the fitness value of cockroach i during the search process is greater than the previous fitness value of cockroach i, the individual optimal solution, individual fitness and current cockroach position are updated. The feature subset and population evaluation are based on the second algorithm.

[0176] In the feature subset and population evaluation, all solutions are evaluated using the classifier kNN (k=7), the fitness and global optimal solution are updated, and N better solutions are selected according to the fitness to form the initial population.

[0177] In the feature subset and population evaluation, the fitness of the solution is judged to be a mutation or a gradual change of the solution, and then a new solution is generated.

[0178] Dynamic threshold adjustment is to set the initial fitness threshold threshold_init at the beginning of the algorithm, which is set according to the specific characteristics of the problem, or initialized according to empirical values ​​or experimental results;

[0179] threshold=threshold_init;

[0180] At each iteration, the diversity of the current population is calculated:

[0181] diversity=1 / N*sum(||y_i-mean(Y)||)for i=1to N;

[0182] Where N is the population size, y_i is the position of the i-th cockroach, and mean(Y) is the mean value of the population position;

[0183] Dynamically adjust the fitness threshold according to changes in population diversity;

[0184] The population diversity has decreased significantly, so the threshold is relaxed to increase the exploration ability;

[0185] threshold=threshold*adjust_factor;

[0186] The diversity of populations is maintained well, the threshold is tightened, and the utilization capacity is increased;

[0187] threshold=threshold / adjust_factor;

[0188] To avoid the threshold being too large or too small, set the upper and lower limits of the threshold:

[0189] threshold=max(threshold_min,min(threshold_max,threshold));

[0190] Among them, threshold_min and threshold_max are the minimum and maximum values ​​of the threshold respectively;

[0191] Pseudo code for dynamic threshold adjustment:

[0192]

[0193]

[0194] The ensemble learning strategy is that after the population initialization of the cockroach swarm optimization algorithm is completed, each individual corresponds to a feature subset. For each feature subset, each classifier is used for training and prediction. For each classifier, cross-validation or multiple random data splits can be used for multiple evaluations, and the prediction accuracy of each evaluation is recorded.

[0195]

[0196] For each feature subset, the prediction results of multiple classifiers are combined to calculate the comprehensive fitness value. The prediction accuracy of different classifiers is combined into one fitness value by weighted average.

[0197]

[0198] According to the comprehensive fitness value, update the fitness value of each individual.

[0199] Using 6 data sets, the optimization algorithm based on rough sets and cockroach swarming is compared with the experimental results of the most representative algorithms in the other three swarm optimizations; the experimental results are discussed. All numerical experiments are implemented in MATLAB R2022b, and the equipment used is MacBook Air, Apple M1 chip, running frequency 3.20GHz, 8GB memory and arm64 architecture MacOS13.2beta system. The algorithms all use KNN classifier as the evaluation criterion, and it is decided to set k=7 based on preliminary experimental classification.

[0200] Data preprocessing and experimental design

[0201] Six datasets downloaded from UCI were used in the experiment. The table provides basic statistics of these datasets.

[0202] Table 4 Data description

[0203]

[0204] The sample set, condition attribute set, and decision attribute in each data set are represented as U, C, and D respectively. The attribute values ​​of some data sets are text type and continuous variable type, which are all normalized to discrete variable type. For null value data, the mode of column vector is used for replacement. In the algorithm, 70% of the instances of data set U (C; D) are selected as training samples, and the remaining 30% are used as test samples. In each iteration, the data set is randomly shuffled. In order to use the cross-validation method, the group optimization algorithms ABC, ACO, PSO, and the optimization algorithm based on rough set and cockroach swarm are executed on the decision table. The classifier KNN (k=7) in the wrapper technology is used as the evaluation standard to calculate the classification accuracy achieved by the feature subset.

[0205] Parameter settings

[0206] There are some important parameter settings in the algorithm. There is currently no theoretical proof of the parameter setting rules. In this article, they are all set based on experience.

[0207] Mutual information filtering threshold 1: mi, the setting of this parameter has a great influence on the running time and accuracy of the algorithm model. The mutual information size distribution of conditional features and decision features in each data set is different. Some distributions are relatively concentrated, and some have a large difference between the maximum mutual information and the minimum mutual information. In the experiment, mi = 0.01 is used.

[0208] Mutual information filtering threshold 2: mi2, which is set to 0.2 in the experiment, which means sorting from small to large according to the mutual information size, filtering out the features corresponding to the first 20% of the mutual information. Both thresholds 1 and 2 have a great impact on the model running time.

[0209] Feature selection threshold: threshold, according to the experiment, is set to 0.5. When the constructed solution vector is greater than or equal to 0.5, the feature is selected, and if it is less than 0.5, it is not selected.

[0210] Table 5 RSICSO parameter settings

[0211]

[0212] Comparison with other feature selection models

[0213] Feature selection is a very important step before data preprocessing. Many scholars have studied different methods and applied them to it. This paper mainly compares several representative swarm intelligence optimization algorithms, ABC bee colony algorithm, ACO ant colony algorithm, and PSO particle swarm algorithm, in terms of the cardinality, running time and accuracy of the selected feature subsets. The following is a brief description of these three swarm algorithms.

[0214] 1.ABC: A relatively new swarm intelligence optimization algorithm. The bee colony consists of nectar sources, employed bees, and unemployed bees. Through the local optimization behavior of each artificial bee individual, it finally achieves global optimization and finds the optimal feature subset.

[0215] 2.ACO: A probabilistic algorithm for finding optimization paths, with the characteristics of distributed computing, positive information feedback and heuristic search. Through the positive feedback mechanism, the search process continues to converge and eventually approaches the optimal solution.

[0216] 3. PSO: It searches for the area around the bird closest to the object, and continuously updates its speed and position during the search process to find the optimal solution.

[0217] As mentioned above, the experiments were repeated many times for the three aspects. Table 8 describes the average number of optimal feature subsets selected by the algorithm RSCSO in 10 data sets. In addition, the average running time and average classification accuracy of the optimal feature subsets obtained by the algorithm are described in Tables 6 and 7, respectively. It should be pointed out here that the running time of RSCSO includes the rough set reduction time and the cockroach swarm optimization random feature selection time.

[0218] Table 6 Comparison of classification accuracy using classifier kNN on 10 data sets

[0219]

[0220] Table 7 Comparison of running time on 10 datasets

[0221]

[0222] Table 8 Comparison of cardinality of selected feature subsets

[0223]

[0224] Table 9 Comparison of classification accuracy and feature subset cardinality using classifier SVM on 6 data sets

[0225]

[0226] As can be seen from the above table, in the classifier SVM, 4 data sets reported errors, so there is no test data. Compared with the original data set, the proposed RSICSO feature selection model can achieve better classification performance on all data sets, except the isolet data set. In addition, compared with the original data, the RSICSO model improves the classification accuracy of the classifier kNN and SVM by 19.27% ​​and 13.88%, respectively. This proves that removing weakly related or redundant features can improve the classification performance of information systems. Compared with other feature selection models, RSICSO can obtain fewer feature subsets and shorter running time while ensuring good classification accuracy. As can be seen from the above table, the cardinality of the selected feature subset increases significantly with the increase of the random feature dimension, and the running time increases with the number of instances. For the data set internet ads, the operation takes the longest time, and the cardinality of the selected feature subset is large. The number of selected results of the ABC and PSO algorithms reaches more than 700, and the time is relatively long. The cardinality of the feature subset selected by the algorithm RSICSO is only one-third of that of other algorithms and one-sixth of that of the data set while ensuring good classification accuracy. Table 7 shows that RSICSO takes less time on all high-dimensional datasets. Especially for datasets with a large number of features. In addition, for the large sample dataset clean, the average running time of RSICSO is almost the same as that of PSO due to the calculation of the mutual information between the conditional attributes and the decision attributes. Furthermore, it can be seen from Tables 6 and 7 that the average cardinality and average accuracy of the best feature subset obtained by RSICSO are relatively good. Under ultra-high-dimensional datasets, by selecting features with high mutual information from the incoming feature set through Algorithm 2, while obtaining better classification accuracy, it can effectively reduce the calculation running time and reduce performance consumption.

[0227] In the present invention, the problem of long calculation time and high performance consumption of swarm intelligence algorithm in high-dimensional data sets is improved; and the problem of too many irrelevant and redundant features selected by swarm intelligence algorithm is improved. The weakly relevant and redundant features are filtered out by calculating the mutual information of conditional feature subsets and decision features, leaving the key features, and the fitness function of the swarm algorithm is defined in combination with the selected feature subsets. In order to enhance the global search capability and make RSICSO get rid of the local optimum, the CSO algorithm is modified by adopting reverse learning, taboo search and intergenerational random allocation mechanism. The proposed model is tested on 10 benchmark data sets and compared with 3 representative swarm intelligence feature selection methods. The experimental results show that the cardinality and running time of the selected feature subsets are greatly reduced while ensuring good classification accuracy, especially on high-dimensional data sets.

[0228] The above is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention shall be implemented according to the conventional means in the art unless otherwise specified and limited.

Claims

1. An optimization algorithm based on rough sets and cockroach clustering, characterized in that: The optimization algorithm comprises the following steps: Step 1. Rough set; Step 2. Cockroach colony optimization; Step 3. Feature filtering; Step 4. Expression of solution and population initialization combined with reverse learning; Step 5. Field search strategy based on pbest and gbest; Step 6. Taboo search; Step 7. Feature subset and population evaluation; Step 8. Dynamic threshold adjustment; Step 9. Ensemble learning strategy.

2. The optimization algorithm based on rough sets and cockroach clustering according to claim 1, characterized in that: The rough set describes knowledge as an information system, which consists of row vectors and column vectors. The row vectors represent objects in the domain, and the column vectors represent different attributes. There is a type of information system called a decision table, which consists of conditional attributes and decision attributes. Each attribute has a corresponding attribute value. In a decision table, unnecessary and redundant attributes are removed while ensuring that the classification ability of the information system remains unchanged. Assume that the decision table is a non-empty finite set (U, C∪D), where U = {x1, x2, ..., xn} is called the domain, C = {a1, a2, ..., an} is called the condition attribute set, and D is the decision attribute set. As shown in Table 1, the decision table consists of 4 samples, 3 condition attributes {a1, a2, a3} and one decision attribute {d}; Table 1 Example of decision table Before calculating the mutual information, it is necessary to calculate the information entropy, conditional entropy or joint entropy of the discrete random variable a. First, the definition of information entropy is given; Shannon defined information entropy as the probability of occurrence of discrete random events. The larger the probability, the more opportunities for occurrence and the smaller the uncertainty; conversely, the uncertainty is large. Let information entropy be H(X), and define it as follows: In formula (1), |U| represents the cardinality of U, which represents the probability of occurrence of discrete values ​​of variable x. Assuming variables a1 and a2, a1=[1,0,1,1]; a2=[0,1,1,0]; According to probability p(a1=0)=1 / 4; p(a1=1)=3 / 4; p(a2=0)=2 / 4; p(a2=1)=2 / 4; Similarly, H(a2)=0.6931; The joint information entropy of two variables X and Y is defined as: where x and y are specific values ​​of X and Y, respectively, and P(x,y) is the joint probability of these values ​​occurring together; Table 2 Example: According to Table 2, P(a1,a2) P(a1=0,a2=0)=0 P(a1=0,a2=1)=1 / 4 P(a1=1,a2=0)=2 / 4 P(a1=1,a2=0)=1 / 4 Calculate H(a1,a2) and get: Conditional entropy H(Y|X) represents the uncertainty of random variable Y under the condition of known random variable X, which is defined as: H(Y|X)=-∑ x＝X P(x)∑ y＝Y P(y|x)log2P(y|x) ⑶According to the occurrence probability of (a1, a2): Table 3 Example: Where P(a1=1|a2=1) represents the probability that a1=0 when a2=0, which is calculated as follows: Similarly, P(a1=0|a2=0), P(a1=0|a2=1), P(a1=1|a2=0) Then the conditional entropy H(a1|a2) is calculated as: Mutual information is a useful information measure. It can be regarded as the amount of information contained in the random variable X about the random variable Y. Suppose the joint distribution of two random variables (X, Y) is p(x, y), and the marginal distribution is p(x), p(y). The mutual information I(X, Y) is the relative entropy of the joint distribution p(x)p(y) and the marginal distribution p(x), p(y), that is: The information calculation formula is: I(X;Y)=H(X)+H(Y)-H(X,Y)⑸.

3. The optimization algorithm based on rough sets and cockroach clustering according to claim 1, characterized in that: The cockroach swarm optimization is established by imitating the pursuit behavior of individual cockroaches to find the global optimal value, and the optimal solution is searched through continuous cooperation and information sharing among the groups; After a set of solutions are randomly initialized, the optimal solution is searched iteratively. The cockroaches continuously update their positions toward the two extreme values ​​pbest and gbest through aggregation behavior. The update rules are as follows: Where yr (r = 1, 2, 3, ..., n) is the position of the rth cockroach, a represents the step length, which is a fixed value, rand takes any value between (0, 1), pr and pg are the optimal position of the rth cockroach and the global optimal cockroach position respectively, and the calculation formula is: visual represents the cockroach's visual field, which is a constant, s = 1, 2, 3, ... n. After each round of foraging, each individual is randomly dispersed through the following model to maintain the diversity of the current individual; the r =y r +rand(1,E),r=1,2,...,N ⑻ Among them, rand(1,E) is an E-dimensional (problem space dimension) random vector that can be set within a certain range. Finally, the global optimum is used to replace the randomly selected individuals. The model is as follows: y n =ρ g ,n=1,2,...,N ⑼.

4. The optimization algorithm based on rough sets and cockroach clustering according to claim 1, characterized in that: The feature filtering is to use the mutual information of the rough set to perform preliminary filtering on the feature set. For two given features a and d, the difference between the information entropy H(a) of feature a and its conditional information entropy H(a|d) is less than a threshold. The input conditional attributes do not need to participate in feature selection and are filtered to select representative features.

5. The optimization algorithm based on rough sets and cockroach clustering according to claim 1, characterized in that: The expression of the solution and the population initialization combined with reverse learning are as follows: Assume that the population size is N, the feature dimension is D, and a D-bit binary string is used to represent a solution. The binary string corresponds to the feature set. 1 indicates that the feature is selected, and 0 indicates that the feature is not selected. In this way, an iterative feature subset is obtained. There is a feature set {a, b, c}. If the final solution is {1, 0, 1}, the corresponding feature subset is {a, c}. The reverse population initialization is applied to the RSICSO algorithm, and a N*D-dimensional feasible solution matrix X is randomly created. For the value of any solution i (i = 1, 2, ..., N) in the g-th generation population, X is used. g i It represents a random number between 0 and 1. The corresponding reverse solution is generated according to formula (10). The solution is selected according to formula (11). The iterative optimal solution of each feasible solution is recorded as pbest, and the global optimal solution is recorded as gbest. X o p=1-X ⑽ 6. The optimization algorithm based on rough sets and cockroach clustering according to claim 1, characterized in that: The domain search strategy based on pbest and gbest is that in each iteration of the algorithm, cockroach i will continuously update its position according to pbest and gbest. The aggregation behavior formula of the cockroach swarm algorithm is modified and applied to the algorithm model. A solution is randomly selected. According to the fitness of the i-th cockroach, if the fitness is small, a local search is performed relative to pbest. If the fitness is large, a global search is performed relative to gbest to generate a new solution. The formula is as follows: In the formula, ρ i is the optimal solution for the current individual of the i-th cockroach, threshold is the fitness threshold, y k , k∈[1,D] is the kth solution selected randomly.

7. The optimization algorithm based on rough sets and cockroach clustering according to claim 1, characterized in that: The taboo search adopts some mechanisms to avoid repeated searches, thereby ensuring the construction of a global solution. In RSICSO, a taboo search is used for cockroaches. First, a taboo table is generated to save the initial solution. Then, in the iterative process, a new solution is generated according to the search strategy. Then, according to the threshold and the taboo table, the solution is mutated or gradually changed to generate a new solution, which is then written into the taboo table, evaluated, and the algorithm is continued.

8. The optimization algorithm based on rough sets and cockroach clustering according to claim 1, characterized in that: The feature subset and population evaluation is evaluated by using the prediction rate of the kNN classifier in the relevant standards. The classification model and feature selection are interdependent, and the obtained feature subset has a good accuracy rate. For each individual in the population, the fitness value is the prediction rate of the classifier, and the fitness value is used to evaluate the quality of the corresponding solution of the cockroach. If the fitness value of cockroach i during the search process is greater than the fitness value of cockroach i before, the individual optimal solution, individual fitness and current cockroach position are updated. In the feature subset and population evaluation, all solutions are evaluated using the classifier kNN (k=7), and the fitness and global optimal solution are updated. N better solutions are selected according to the fitness to form an initial population. In the feature subset and population evaluation, the fitness of the solution is judged to mutate or gradually change the solution, and then a new solution is generated.

9. The optimization algorithm based on rough sets and cockroach clustering according to claim 1, characterized in that: The dynamic threshold adjustment is to set an initial fitness threshold threshold_init at the beginning of the algorithm, which is set according to the specific characteristics of the problem, or initialized according to empirical values ​​or experimental results; threshold=threshold_init; At each iteration, the diversity of the current population is calculated: diversity=1 / N*sum(||y_i-mean(Y)||)for i=1to N; Where N is the population size, y_i is the position of the i-th cockroach, and mean(Y) is the mean value of the population position; Dynamically adjust the fitness threshold according to changes in population diversity; The population diversity has decreased significantly, so the threshold is relaxed to increase the exploration ability; threshold=threshold*adjust_factor; The diversity of populations is maintained well, the threshold is tightened, and the utilization capacity is increased; threshold=threshold / adjust_factor; To avoid the threshold being too large or too small, set the upper and lower limits of the threshold: threshold=max(threshold_min,min(threshold_max,threshold)); Among them, threshold_min and threshold_max are the minimum and maximum values ​​of the threshold respectively; Pseudo code for dynamic threshold adjustment:

10. The optimization algorithm based on rough sets and cockroach clustering according to claim 1, characterized in that: The ensemble learning strategy is that after the population initialization of the cockroach swarm optimization algorithm is completed, each individual corresponds to a feature subset, and each feature subset is trained and predicted using each classifier. Each classifier can be evaluated multiple times using cross-validation or multiple random data splits, and the prediction accuracy of each evaluation is recorded; For each feature subset, the prediction results of multiple classifiers are combined to calculate the comprehensive fitness value. The prediction accuracy of different classifiers is combined into one fitness value by weighted average. According to the comprehensive fitness value, update the fitness value of each individual.

Images