Genetic programming text data oversampling method based on multi-layer selection double-criterion evaluation mechanism

Abstract

The invention belongs to the field of unbalanced data classification, and relates to a genetic programming text data oversampling method based on a multi-layer selection double-criterion evaluation mechanism. According to the method, firstly, an existing unbalanced data set is divided, the number of samples needing to be generated is calculated, and a target majority class and a target minority class are obtained; secondly, an evolution process of genetic programming needs to be designed, and each sub-population independently evolves a target sample by using the designed evolution process; a fitness function adopts a dual-criterion evaluation function based on distance and angle, and a selection operator adopts a selection algorithm which is improved based on a tournament selection method and is suitable for dual-criterion evaluation; the objective of sub-population evolution is to obtain an optimal synthetic instance. Then, selecting an optimal individual from each evolved sub-population as a synthetic instance, and finally, combining the selected individual with an original data set to obtain a complete balanced data set.

Description

Technical Field

[0001] The invention belongs to the field of imbalanced data classification and relates to a genetic programming text data oversampling method based on a multi-layer selection double-criteria evaluation mechanism. Background Art

[0002] Classification is a fundamental task in machine learning and plays a vital role in fields such as medical diagnosis, fraud detection, and spam judgment. In these tasks, the model must learn the data distribution from the input and accurately classify it. Traditional classification algorithms usually rely on balanced data sets, which enable the model to effectively learn the patterns of each category. However, in many real-world scenarios, data sets are often unbalanced, and some categories are significantly underrepresented. This imbalance can cause the model to be biased, thereby ignoring underrepresented categories. In scenarios such as rare disease diagnosis and fraudulent activity detection, whether minority categories can be accurately classified is the most interesting issue. Therefore, it is of great significance to study how to solve the problem that existing classification algorithms have difficulty in extracting effective information in imbalanced data for classifying minority categories.

[0003] To date, methods for solving the imbalanced classification problem can be roughly divided into two categories: algorithm-level methods and data-level methods. The former does not directly change the data distribution, but trains the classifier by weighing the losses associated with different types of errors and assigning unequal costs to each error. In this way, the classifier can discover hidden patterns in the imbalanced data and effectively distinguish different categories. However, the mechanism for changing different classification methods usually varies from case to case, and such methods still face huge challenges in terms of complexity, computational cost, and parameter adjustment. In contrast, data-level methods mainly refer to sampling methods that solve class imbalance problems by changing data distribution. The purpose is to provide a balanced data set for classifier training by increasing or decreasing the number of instances. Because it solves imbalanced classification by changing the data set itself, it solves the problem that algorithm-level methods vary due to different classification algorithm mechanisms, and is more universal.

[0004] At present, the sampling method in the dataset method has become the mainstream technology for solving the imbalanced classification problem. Common sampling methods mainly include undersampling, oversampling and mixed sampling. Taking binary classification as an example, the class containing most instances in the dataset is called the majority class (Maj), and the remaining instances are the minority class (Min). The undersampling method reduces the number of instances in Maj by removing a subset of instances, thereby balancing the dataset, while the oversampling rule compensates for the lack of data in Min by increasing the number of instances in Min. Undersampling may delete useful instances from Maj, resulting in the loss of some effective information, especially when the dataset is extremely unbalanced. The problem of losing instances will become more serious. The mixed sampling rule combines these two methods, first generates new samples through the oversampling method, and then selects high-quality samples for preservation. Due to the obvious defects of the undersampling method and the limitations of most scenarios, the oversampling algorithm, which is the basis of the mixed sampling algorithm, has become the main research area. SMOTE (Synthetic Minority Oversampling Technique) is one of the most classic oversampling algorithms. Most of the existing oversampling methods are evolved from SMOTE. However, existing oversampling methods always face some challenges: First, existing oversampling methods usually rely on neighborhood information to synthesize data, using linear interpolation of neighborhood samples. This method limits its ability to capture complex data structures, resulting in its lack of adaptability and flexibility. In cases involving complex category boundaries or nonlinear distributions, simple neighborhood interpolation may not be able to effectively generate representative minority category samples, thereby hindering the classifier's ability to identify minority categories. In addition, traditional methods also lack effective guidance in selecting appropriate neighborhood samples. Selecting inappropriate neighborhoods may introduce unnecessary noise, thereby reducing the quality of synthesized samples. These low-quality synthetic samples can cause classification models to overfit or generalize poorly.

[0005] Genetic programming is a heuristic algorithm derived from the field of evolutionary computing. The purpose of genetic programming is to evolve computer programs. Each program is usually represented by a tree, and the evolved tree is also considered as a candidate solution to the problem. In genetic programming, the leaf nodes and non-leaf nodes of each tree are automatically selected from the pre-designed terminal set and function set, respectively. At the same time, the genetic programming algorithm also builds a model structure to combine the selected nodes to form genetic programming individuals. This makes it possible for genetic programming to generate new data by automatically selecting and combining existing high-quality data and evolving structures to perform a series of operations. This is of great significance for solving the problem that existing oversampling methods generally require predefined linear structures to generate data and are greatly affected by noise data in the original data set. However, to date, there are few studies on how to use genetic programming to develop effective data generation methods. Therefore, it is of great significance to study how to combine genetic programming to solve the problem of imbalanced classification. Summary of the invention

[0006] In view of the problems that the existing oversampling methods generally have the risk of overfitting, require a predefined structure, and are greatly affected by the noise data in the original data set, the present invention proposes a genetic programming text data oversampling method based on a multi-layer selection dual-criteria evaluation mechanism, the purpose of which is to study the potential of using genetic programming to develop oversampling methods, construct an efficient oversampling algorithm to help solve the classification task of unbalanced data, and provide new ideas for the development of oversampling methods by introducing genetic programming. Through the method of the present invention, firstly, a partitioning operation is performed on the existing unbalanced data set, the number of samples to be generated is calculated, and the target majority class and the target minority class are obtained. Then, the evolution process of genetic programming needs to be designed, and each subpopulation uses the designed evolution process to independently evolve the target sample; the fitness function adopts a dual-criteria evaluation function based on distance and angle, and the selection operator adopts a selection algorithm suitable for dual-criteria evaluation based on the tournament selection method; the goal of subpopulation evolution is to obtain the best synthetic instance. Then, a best individual is selected from each evolved subpopulation as a synthetic instance, and finally the selected individual is combined with the original data set to obtain a complete balanced data set.

[0007] The present invention mainly includes two parts:

[0008] (1) Imbalanced data processing: preprocessing the existing imbalanced data set and calculating the information needed in the subsequent evolution process. First, calculate the number of samples that need to be generated, then calculate the mean of the minority class and majority class samples in each dimension in the data set, obtain the center points of the minority class and majority class, calculate the distance from the majority class sample to the cluster center, and combine the minority class samples from small to large distances to obtain multiple pairs of target majority class and target minority class.

[0009] (2) Data generation, to achieve the generation and supplement of unbalanced data sets. First, it is necessary to design the evolution process of genetic programming, and give the information obtained by the data processing module to each sub-population of genetic programming. Each sub-population uses the designed evolution process to independently evolve the target sample; the fitness function adopts a dual-criteria evaluation function based on distance and angle, and the selection operator adopts a selection algorithm based on the tournament selection method that is improved and suitable for dual-criteria evaluation. The goal of sub-population evolution is to obtain the best synthetic instance. From each evolved sub-population, an optimal individual is selected as the synthetic instance, and the selected individual is combined with the original data set to obtain a complete balanced data set.

[0010] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0011] A genetic programming text data oversampling method based on a multi-layer selection dual-criteria evaluation mechanism, the specific steps are as follows:

[0012] Step (1): Divide the population into subpopulations according to the number of samples required to be generated, and assign target minority class samples and target majority class samples to each subpopulation;

[0013] Step (2): Design the evolution process of genetic programming. Each subpopulation uses the designed evolution process to independently evolve the target sample. The fitness function adopts a dual-criteria evaluation function based on distance and angle. The selection operator adopts a selection algorithm suitable for dual-criteria evaluation based on the tournament selection method. The goal of subpopulation evolution is to obtain the best synthetic instance.

[0014] Step (3): Select an optimal individual from each evolved sub-population as a synthetic instance;

[0015] Step (4): Combine the selected individuals with the original dataset to obtain a complete balanced dataset.

[0016] The step (1) is specifically as follows:

[0017] (1.1) The data set contains N minority class samples and M majority class samples. MN samples need to be generated, and the population is divided into MN sub-populations accordingly;

[0018] (1.2) Calculate the mean of the minority class and majority class samples in each dimension in the data set respectively, and obtain the center points of the minority class and majority class, denoted as PC and NC;

[0019] (1.3) Calculate the Euclidean distance from the majority class sample to the NC, and select MN majority class samples and minority class samples in ascending order of distance to obtain the MN group instance information;

[0020] (1.4) Each subpopulation is assigned a set of target instance information from the original minority class and majority class, denoted as (Mint, Majt).

[0021] The step (2) is specifically as follows:

[0022] (2.1) Initialize the subpopulation, set the number of subpopulations to MN and the size to S; set the function set F = {+, -, *, / }, the termination set T = {Min, random (0, 1)}, Min is the minority class sample; set the number of iterations to G, the individual crossover probability to Cxpb, the mutation probability to Mutpb, and limit the individual tree height to no higher than H;

[0023] (2.2) Use the ramped half-and-half method to initialize the population individuals and obtain the initial population I;

[0024] (2.3) For each individual in the population, a dual-criteria evaluation function based on distance and angle is used to evaluate and calculate the individual's fitness value tuple (L1, L2);

[0025] (2.3.1) For the distance metric L1 in the evaluation criteria: Assume that each individual is I s , L1 is the synthetic instance generated from I s The difference between the distance to Majt and the distance to Mint. The mathematical calculation method is as follows:

[0026]

[0027] (2.3.2) For the angle measurement in the evaluation criteria: According to Majt, Mint and I s The position in space can naturally form a triangle. In the triangle, b represents I s The distance between I and Mint, c represents I s and Majt, a represents the distance between Mint and Majt. According to the law of cosines, the angle between b and c can be calculated as follows:

[0028]

[0029] (2.4) For each individual in the population after evaluation, the improved tournament selection method is used to confirm whether the individual is selected based on the calculated fitness value. First, the distance metric L1 is used as a constraint, and only individuals with L1>0 have the possibility of being selected. Then, the individual with a larger L2 is selected based on the angle metric L2;

[0030] (2.5) First select the best individuals to be retained for the next generation, and perform crossover / mutation on the other individuals in the population with a probability of Cxpb / Mutpb. After all operations are completed, a new population is obtained;

[0031] (2.6) Repeat the operation (2.3);

[0032] (2.7) Loop through (2.4)-(2.6) G times

[0033] (2.8) After the evolution process is completed, a set of solutions is obtained;

[0034] The step (3) is specifically as follows:

[0035] (3.1) After step (2), the subpopulations evolved by the MN group are obtained and screened according to the L1 metric of the individuals. If the L1 metric of a subpopulation is less than 0, an individual is randomly selected from the original data set;

[0036] (3.2) Select the individual with the largest L2 metric in each subpopulation. If the L2 metrics are the same, select the individual with the largest L1. If they are the same, they are considered equal individuals and can be randomly selected.

[0037] (3.3) The representation of a selected individual is a tree, which is parsed into an expression. The result of the expression is the generated data.

[0038] The step (4) is specifically as follows:

[0039] (4.1) Combining the individuals selected in step (3) as a dataset consisting entirely of the minority class, where the dataset size is MN;

[0040] (4.2) Combine the dataset obtained in (4.1) with the dataset before processing in (1) to obtain a balanced dataset. The number of majority classes in the dataset before processing is M, the number of minority classes is N, and the size of the newly generated dataset is MN. Therefore, the size of the combined dataset is M+N+(MN)=2M, which is the final solution.

[0041] Compared with the existing method, the present invention has the following beneficial effects:

[0042] This method demonstrates its effectiveness in generating high-quality synthetic instances to address the class imbalance problem. This method can help multiple classification algorithms effectively overcome the performance bias problem, thereby improving their classification performance on imbalanced datasets. This method uses genetic programming to generate a set of new instances for the minority class in the imbalanced classification to fill the dataset to maintain the balance between different classes. It is the first oversampling method based on genetic programming. This method can automatically select high-quality instances from the minority class and automatically evolve the model structure of the synthetic instance, regardless of the manually defined structure and neighborhood information of the original instance. This can reduce the risk of generating noisy instances when there is noise in the original data. At the same time, through the fitness evaluation method we designed, it is possible to avoid pre-providing weights. Finally, this method overcomes the shortcomings of existing oversampling methods, generates high-quality datasets, and improves the accuracy of imbalanced data classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a specific flow chart of an embodiment of the present invention;

[0044] Figure 2 It is a schematic diagram of the central target point;

[0045] Figure 3 It is a schematic diagram of the fitness calculation method;

[0046] Figure 4 It is a schematic diagram of the selection operator in the evolution process;

[0047] Figure 5 It is a schematic diagram of a specific spanning tree;

[0048] Figure 6 Generate a distribution plot of the dataset. DETAILED DESCRIPTION

[0049] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0050] The present invention proposes to fill the data set to maintain the balance between different categories by using genetic programming to generate a set of new instances for the minority class in the imbalanced classification, aiming at the class imbalance problem, and proposes a genetic programming text data oversampling algorithm based on a multi-layer selection dual-criteria evaluation mechanism. This method is the first oversampling method based on genetic programming. In this method, an individual in genetic programming is regarded as a synthetic instance, and its leaf nodes are high-quality instances automatically selected from the minority class through the tree. More importantly, the model structure of the synthetic instance is also automatically evolved, which is independent of the manually defined structure and the neighborhood information of the original instance. This can reduce the risk of generating noisy instances when there is noise in the original data. At the same time, in order to effectively evaluate the quality of individuals, we designed a fitness evaluation method in which the distance and angle measurement methods work in parallel without providing weights in advance. Finally, this method overcomes the shortcomings of the existing oversampling methods, generates high-quality data sets, and improves the accuracy of imbalanced data classification.

[0051] like Figure 1 As shown, the embodiment of the present invention is a genetic programming text data oversampling algorithm based on a multi-layer selection dual-criteria evaluation mechanism, and the oversampling processing of the unbalanced data set Ecoli-0-3-4_vs_5 includes the following steps:

[0052] Step (1): Data processing: Divide the population into sub-populations according to the number of samples required to be generated, and assign target minority class samples and target majority class samples to each sub-population;

[0053] (1.1) Divide the data set and the population: Calculate the number of minority classes and majority classes in the data set Ecoli-0-3-4_vs_5. The data set contains 32 minority class samples and 132 majority class samples. It is necessary to generate 132-32=100 samples, which corresponds to dividing the population into 100 sub-populations.

[0054] (1.2) Calculate the center point: Calculate the mean of the minority class and majority class samples in each dimension in the data set respectively, and obtain the center points of the minority class and majority class, denoted as PC and NC. The specific location of the center point is as follows: Figure 2 As shown, the two squares NC and PC in the figure represent the center point of the majority class and the center point of the minority class respectively;

[0055] (1.3) Sorting: Calculate the Euclidean distance from each majority class sample to NC in the Ecoli-0-3-4_vs_5 dataset, and select 100 majority class samples and minority class samples in ascending order of distance to obtain 100 sets of instance information;

[0056] (1.4) Combined target pairs: Assign a set of target instance information from the original minority class and majority class to each divided sub-population, denoted as (Mint,Majt).

[0057] Step (2): Design the evolution process and evolve the instance: Design the evolution process of genetic programming. Each subpopulation uses the designed evolution process to independently evolve the target sample. The fitness function adopts a dual-criteria evaluation function based on distance and angle. The selection operator adopts a selection algorithm suitable for dual-criteria evaluation based on the tournament selection method. The goal of subpopulation evolution is to obtain the best synthetic instance.

[0058] (2.1) Initialization module: Initialize the subpopulation, set the number of subpopulations to 100, and the size to 100×30=3000; set the function set F={+,-,*, / }, the termination set T={Min,random(0,1)}, Min is the minority class sample; set the number of iterations to 100, the individual crossover probability to 0.8, the mutation probability to 0.2, and limit the individual tree height to no more than 10;

[0059] (2.2) Use the ramped half-and-half method to initialize the population individuals and obtain the initial population I;

[0060] (2.3) Fitness function evaluation module: For each individual in the population, a dual-criteria evaluation function based on distance and angle is used to evaluate the individual and calculate the individual's fitness value tuple (L1, L2). The detailed fitness calculation process diagram is shown in Figure 3 The three squares in the figure represent the relative positions of the three generated samples and Mint and Majt in space, where a, b, c and α (i.e. L2) are calculated as follows;

[0061] (2.3.1) For the distance metric L1 in the evaluation criteria: Assume that each individual is I s , L1 is the synthetic instance generated from I s The difference between the distance to Majt and the distance to Mint. The mathematical calculation method is as follows:

[0062]

[0063] (2.3.2) For the angle measurement in the evaluation criteria: According to Majt, Mint and I sThe position in space can naturally form a triangle. In the triangle, b represents I s The distance between I and Mint, c represents I s and Majt, a represents the distance between Mint and Majt. According to the law of cosines, the angle between b and c can be calculated as follows:

[0064]

[0065] (2.4) Selection operator: For each individual in the evaluated population, the improved tournament selection method is used to confirm whether the individual is selected based on the calculated fitness value. First, the distance metric L1 is used as a constraint. Only individuals with L1>0 can be selected. Then, the individual with a larger L2 is selected based on the angle metric L2. The detailed selection process is shown in Figure 4 , Figure 4 The selection mechanism of the selection operator is explained by taking the selection of 5 generated individuals as an example;

[0066] (2.5) Crossover and mutation operator: First, select the best individual to keep for the next generation, and perform crossover / mutation on the other individuals in the population with a probability of 0.8 / 0.2. After all operations are completed, a new population is obtained;

[0067] (2.6) Repeat the operation (2.3);

[0068] (2.7) Loop through (2.4)-(2.6) G times;

[0069] (2.8) After the evolution process is completed, a set of solutions is obtained.

[0070] Step (3): Select individuals: Select the best individual from each evolved sub-population as the synthetic instance;

[0071] (3.1) After step (2), 100 groups of evolved subpopulations are obtained for the dataset Ecoli-0-3-4_vs_5. The subpopulations are screened according to the L1 metric of the individuals in the population. If the L1 metric of a subpopulation is less than 0, an individual is randomly selected from the minority class samples of Ecoli-0-3-4_vs_5 as the final result of the population evolution.

[0072] (3.2) Select the individual with the largest L2 metric in each subpopulation. If the L2 metrics are the same, select the individual with the largest L1. If they are the same, they are considered equal individuals and can be randomly selected.

[0073] (3.3) The selected individual is represented as a tree, which is parsed into an expression. The result of the expression is the generated data. For a specific generation example, see Figure 5 , Figure 5 Each expression corresponding to each tree in the diagram is solved to generate a piece of data. For example, the three trees in the diagram correspond to x 10 / x 5 *x 12 、x 12 *0.9627, x 7 *x 10 / x 6 .

[0074] Step (4): Combine to generate a new dataset: Combine the selected individuals with the original dataset to obtain a complete balanced dataset.

[0075] (4.1) Combine the individuals selected in step (3) as a dataset consisting entirely of minority classes, where the dataset size is 100;

[0076] (4.2) Combine the data set obtained in (4.1) with the data set before processing in (1) to obtain a balanced data set with a data set size of 200. This data set is the final solution. The distribution diagram of the generated data set is shown in 6. Figure 6 The hexagon in the middle represents the majority class in the dataset, and the circle represents the minority class in the dataset. After processing, the number of majority classes in the dataset is the same as that of minority classes.

Claims

1. A genetic programming text data oversampling method based on a multi-layer selection dual-criteria evaluation mechanism, characterized in that: The specific steps are as follows: Step (1): Divide the population into subpopulations according to the number of samples required to be generated, and assign target minority class samples and target majority class samples to each subpopulation; Step (2): Design the evolution process of genetic programming, and each subpopulation uses the designed evolution process to independently evolve the target sample; The fitness function adopts a dual-criteria evaluation function based on distance and angle, and the selection operator adopts a selection algorithm based on the tournament selection method that is suitable for dual-criteria evaluation; the goal of subpopulation evolution is to obtain the best synthetic instance; Step (3): Select an optimal individual from each evolved sub-population as a synthetic instance; Step (4): Combine the selected individuals with the original dataset to obtain a complete balanced dataset.

2. According to claim 1, a genetic programming text data oversampling method based on a multi-layer selection dual-criteria evaluation mechanism is characterized in that: The step (1) is specifically as follows: (1.1) The data set contains N minority class samples and M majority class samples. MN samples need to be generated, and the population is divided into MN sub-populations accordingly; (1.2) Calculate the mean of the minority class and majority class samples in each dimension in the data set respectively, and obtain the center points of the minority class and majority class, denoted as PC and NC; (1.3) Calculate the Euclidean distance from the majority class sample to the NC, and select MN majority class samples and minority class samples in ascending order of distance to obtain the MN group instance information; (1.4) Each subpopulation is assigned a set of target instance information from the original minority class and majority class, denoted as (Mint, Majt).

3. The method for oversampling text data using genetic programming based on a multi-layer selection dual-criteria evaluation mechanism according to claim 1, characterized in that: The step (2) is specifically as follows: (2.1) Initialize the subpopulation, set the number of subpopulations to MN and the size to S; set the function set F = {+, -, *, / }, the termination set T = {Min, random (0, 1)}, Min is the minority class sample; set the number of iterations to G, the individual crossover probability to Cxpb, the mutation probability to Mutpb, and limit the individual tree height to no higher than H; (2.2) Use the ramped half-and-half method to initialize the population individuals and obtain the initial population I; (2.3) For each individual in the population, a dual-criteria evaluation function based on distance and angle is used to evaluate and calculate the individual's fitness value tuple (L1, L2); (2.4) For each individual in the evaluated population, the improved tournament selection method is used to confirm whether the individual is selected based on the calculated fitness value; first, the distance metric L1 is used as a constraint, and only individuals with L1>0 have the possibility of being selected. Then, the individual with a larger L2 is selected based on the angle metric L2; (2.5) First select the best individuals to be retained for the next generation, and perform crossover / mutation on the other individuals in the population with a probability of Cxpb / Mutpb. After all operations are completed, a new population is obtained; (2.6) Repeat the operation (2.3); (2.7) Loop through (2.4)-(2.6) G times (2.8) After the evolution process is completed, a set of solutions is obtained.

4. A genetic programming text data oversampling method based on a multi-layer selection dual-criteria evaluation mechanism according to claim 3, characterized in that: Step (2.3) is as follows: (2.3.1) For the distance metric L1 in the evaluation criteria: Assume that each individual is I s , L1 is the synthetic instance generated from I s The difference between the distance to Majt and the distance to Mint; the mathematical calculation method is as follows: (2.3.2) For the angle measurement in the evaluation criteria: According to Majt, Mint and I s The position in space can naturally form a triangle; in the triangle, b represents I s The distance between I and Mint, c represents I s and Majt, a represents the distance between Mint and Majt; according to the cosine theorem, the angle between b and c can be calculated as follows:

5. The method for oversampling text data using genetic programming based on a multi-layer selection dual-criteria evaluation mechanism according to claim 1, characterized in that: The step (3) is specifically as follows: (3.1) After step (2), the subpopulations evolved by the MN group are obtained and screened according to the L1 metric of the individuals. If the L1 metric of a subpopulation is less than 0, an individual is randomly selected from the original data set; (3.2) Select the individual with the largest L2 metric in each subpopulation. If the L2 metrics are the same, select the individual with the largest L1. If they are the same, they are considered equal individuals and can be randomly selected. (3.3) The representation of a selected individual is a tree, which is parsed into an expression. The result of the expression is the generated data.

6. A genetic programming text data oversampling method based on a multi-layer selection dual-criteria evaluation mechanism according to claim 1, characterized in that: The step (4) is specifically as follows: (4.1) Combining the individuals selected in step (3) as a dataset consisting entirely of the minority class, where the dataset size is MN; (4.2) Combine the dataset obtained in (4.1) with the dataset before processing in (1) to obtain a balanced dataset. The number of majority classes in the dataset before processing is M, the number of minority classes is N, and the size of the newly generated dataset is MN. Therefore, the size of the combined dataset is M+N+(MN)=2M, which is the final solution.

Images