High-dimensional feature selection method based on long and short term memory guided evolution

Abstract

The invention discloses a high-dimensional feature selection method based on long and short-term memory guided evolution, and the method comprises the steps: a long-term memory set and a short-term memory set are used for guiding genetic algorithm evolution, the long-term memory set is responsible for accumulating global excellent feature information in an evolution process, and the short-term memory set is used for recording feature information which is good in recent performance; the long-term memory set and the short-term memory set are continuously updated by using a memory forgetting strategy in the evolution process so as to better guide the evolution direction of the genetic algorithm; the dual-memory updating mechanism gives consideration to the utilization of existing information and the exploration of new information, and the long-term memory set stores effective characteristics verified for a long time and provides a stable evolution direction; and the short-term memory set is sensitive to new information, so that the algorithm is promoted to explore a new feature combination. According to the method, higher classification accuracy is obtained, meanwhile, a smaller feature subset is achieved, and an effective direction is provided for solving the feature selection problem.

Description

Technical Field

[0001] The present invention relates to the field of high-dimensional feature selection in machine learning, and in particular to a high-dimensional feature selection method based on long short-term memory guided evolution, which is used to improve the performance of an algorithm in feature selection of high-dimensional data sets. Background Art

[0002] With the rapid development of big data and artificial intelligence, data in various fields is showing an explosive growth in dimensions and quantity, especially in natural language processing, image retrieval, speech signal processing and gene analysis. The high-dimensional nature of data makes feature selection an important technical means to solve big data problems. Feature selection is widely used as an important data preprocessing technology. By selecting representative, informative and highly relevant features from a large number of features, it can reduce data redundancy and improve model performance. In high-dimensional data scenarios, feature selection can not only reduce the consumption of computing resources, but also improve the generalization and explanation capabilities of learning algorithms. Therefore, how to effectively select features without losing effective information has become a key issue in the field of data science and machine learning.

[0003] High-dimensional data often contains a large number of redundant and irrelevant features. If all of them are added to the model training without selection, the model complexity will increase, overfitting will occur easily, and the generalization ability of the model will be reduced. In addition, too high a feature dimension will increase the time and computational cost of data processing. Therefore, effectively removing redundant and noisy features through feature selection can significantly improve model performance and enhance model interpretability. However, in high-dimensional data scenarios, feature selection faces many challenges. First, the feature space of high-dimensional data is extremely large, and the process of feature selection is equivalent to finding the optimal feature subset in the huge feature space; second, there are complex correlations between different features, and it is not easy to screen out features that are related to the target variable and are not redundant; finally, feature selection also needs to take into account the characteristics of the data and the selection task. Different learning algorithms may have different requirements for features, so a single method is difficult to meet the diverse application needs. In order to meet these challenges, a variety of feature selection methods have been proposed in academia and industry.

[0004] Traditional feature selection methods can be roughly divided into three categories: filtering, wrapping and embedding. These methods have their own advantages and disadvantages when applied. Filtering refers to the selection based on the statistical properties of features, usually using univariate analysis indicators such as variance, mutual information, chi-square test, correlation coefficient, etc. This type of method has a fast calculation speed and does not depend on the specific learning algorithm, so it is suitable for preliminary screening of relevant features. However, the filtering method ignores the interaction between features, which may cause some information loss. The wrapping method combines the feature selection process with the learning algorithm and selects the optimal feature combination by evaluating the performance of different feature subsets under a specific algorithm. Common wrapping methods include forward selection, backward elimination, recursive feature elimination, etc. The wrapping method can discover the combination effect between features, but the computational complexity is high, especially when processing high-dimensional data, the computational cost is huge and it is easy to overfit. The embedding method directly embeds the feature selection process into the model training, such as decision trees and L1 regularized linear regression. This type of method automatically selects important features during the model training process, and does not require additional calculation of the evaluation indicators of feature subsets. The embedding method can achieve efficient feature selection, but its effect depends on the specific model, so its generalization ability is subject to certain limitations.

[0005] Due to the limitations of traditional methods in high-dimensional data processing, intelligent optimization algorithms have gradually become an important direction of feature selection research in recent years. Intelligent optimization algorithms search for the optimal feature subset in the feature space by simulating natural evolution and group intelligence behavior, and have good global search capabilities and strong adaptability. Commonly used intelligent optimization algorithms include genetic algorithms, particle swarm optimization, ant colony algorithms, etc. Genetic algorithms are an optimization algorithm based on the principles of natural selection and genetic variation. They guide the population to evolve through operations such as selection, crossover, and mutation to search for the optimal solution. In the feature selection problem, genetic algorithms encode feature subsets as population individuals and obtain the optimal feature combination through continuous iterative optimization. However, in high-dimensional feature space, genetic algorithms are prone to fall into local optimality and converge slowly. The particle swarm optimization algorithm PSO realizes global search by simulating the foraging behavior of bird flocks. Each particle represents a feature subset and updates its position according to the optimal position of itself and its neighbors. The PSO algorithm is simple and efficient, but it may converge to the local optimum in high-dimensional data and it is difficult to jump out of the local extreme value. The ant colony algorithm ACO simulates the path selection behavior of ants foraging and associates the selection probability of each path with the importance of the features on the path. ACO performs well on small-scale problems, but in high-dimensional data scenarios, the computational complexity is large and the path selection is prone to fall into local optimality.

[0006] However, existing feature selection methods still have some problems in feature space, with high computational complexity and difficulty in efficiently processing ultra-high-dimensional data. In addition, most current methods rely on fixed parameter settings and model structures, lacking adaptability to different types of data and feature distributions, thus affecting the versatility and adaptability of the algorithm. Therefore, how to design a feature selection algorithm with higher global search capabilities and strong adaptability to data characteristics without increasing the complexity of the algorithm, in order to improve the efficiency and selection accuracy of the algorithm, still needs further exploration. Summary of the invention

[0007] In view of the shortcomings and defects of the above-mentioned existing feature selection methods, the purpose of the present invention is to provide a high-dimensional feature selection method based on long short-term memory guided evolution.

[0008] The object of the present invention is achieved through the following technical solution: a high-dimensional feature selection method based on long short-term memory guided evolution, comprising the following steps:

[0009] Constructing a training data set and preprocessing the data in the training data set, wherein the training data set includes multiple types of high-dimensional biomedical data, and the high-dimensional biomedical data has different sample numbers, feature numbers, and category labels to ensure the applicability and effectiveness of the algorithm in diverse high-dimensional data scenarios; the preprocessing includes outlier processing and normalization to improve the consistency of the data and the robustness of the algorithm;

[0010] The preprocessed training data set is used as input to initialize the population and generate a population size that satisfies evolution. The population includes a variety of initial individuals. Each initial individual is regarded as a specific feature selection scheme, which defines a feature subset that needs to be retained in the current evolution process.

[0011] Generate new individuals using a long short-term memory search operator, wherein the long short-term memory search operator includes a crossover operation and a mutation operation guided by long short-term memory; guide the evolution process of the population by combining historical memory information stored in long-term memory and short-term memory, thereby accelerating convergence and improving the accuracy and efficiency of feature selection;

[0012] The fitness of the new individuals is evaluated to evaluate the classification accuracy and feature compression rate of the feature subset of the new individuals, and the top N individuals with the highest fitness are retained through environmental selection to enter the next generation population to ensure that the population evolves in the right direction; in the later stage of each evolutionary iteration, the memory set is updated using the long-term and short-term memory dynamic update mechanism; wherein the long-term memory set is used to record historical features with excellent performance to preserve global excellent feature information, and the short-term memory set is used to record features that frequently appear in the recent period to improve the diversity of features, thereby dynamically adjusting the weights of features;

[0013] When the evolution process meets the iteration termination condition, the individual with the best fitness is selected from the population to obtain the optimal feature subset for model training and testing.

[0014] Furthermore, the training data set is derived from a public high-dimensional biomedical data set, and the training data set includes CNS, arcene, CLLSUB, Prostate, Lung, Ovarian, SMKCAN, CrohnDisease, GLI, BreastCancer1 and BreastCancer2.

[0015] Furthermore, the population initialization includes: assigning different numbers of features to initial individuals by a linear interpolation method to ensure the differences between the individuals, so as to generate a diversified initial population with different feature densities in a high-dimensional search space.

[0016] Furthermore, the long short-term memory search operator includes a crossover operation and a mutation operation; wherein the crossover operation adopts a plurality of crossover methods, including single-point crossover, double-point crossover and uniform crossover, and one of the crossover methods is randomly selected during the evolution process; through the combination of a plurality of crossover methods, it is ensured that the offspring can inherit the excellent characteristics of the parent generation, and at the same time, new feature combinations are introduced to increase the diversity of the population; the mutation operation is guided by the historical feature selection information in the long-term memory set and the short-term memory set, so as to accelerate the convergence speed of the algorithm.

[0017] Furthermore, the use of the long-term and short-term memory dynamic update mechanism to update the memory set is specifically as follows: a logarithmic scale mapping method is used to dynamically adjust the number of features used to update the memory set; a feature set used to update the memory is selected according to the selection frequency of the features; a memory forgetting decay strategy is used to dynamically adjust the memory set by combining old memory information and new feature information; the update of the long-term memory set is used to consolidate excellent features that have been repeatedly verified, and to gradually forget features that perform poorly; the short-term memory set is used to capture new features and explore new feature combinations, and to quickly forget features that perform poorly.

[0018] Furthermore, the fitness evaluation of the new individual includes: using KNN as a classifier, setting the number of neighbors K to 5, and using 5-fold cross validation in each evolutionary iteration to comprehensively evaluate the classification accuracy and feature compression rate of the feature subset.

[0019] The beneficial effects of the present invention are: improving the accuracy of feature selection: accurately measuring the performance of feature subsets in classification tasks through KNN evaluation, so as to effectively screen out feature combinations with high classification performance. Dynamic optimization adaptability: the evolutionary mechanism guided by long-term and short-term memory combined with the dynamic adjustment of long-term and short-term memory makes the feature selection process more adaptable to the distribution changes of features and avoids local optimal traps. High computing efficiency: the memory decay strategy can effectively screen out invalid features, significantly improve computing efficiency, and is suitable for various high-dimensional data scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of the system structure of the high-dimensional feature selection method guided by long short-term memory evolution of the present invention;

[0021] Figure 2 is a flow chart of a long short-term memory guided evolutionary algorithm according to an embodiment of the present invention;

[0022] Figure 3 This is a comparison chart of the average classification accuracy of the present invention and other feature selection methods in 11 high-dimensional medical data sets;

[0023] Figure 4 This is a comparison chart of the average feature subset size of the present invention and other feature selection methods in 11 high-dimensional medical data sets;

[0024] Figure 5 This is a comparison chart of the average running time of the present invention and other feature selection methods in 11 high-dimensional medical data sets. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical methods, creative features and advantages of the present invention clearer, the high-dimensional feature selection method based on long short-term memory guided evolution of the present invention is further elaborated in detail below in combination with the embodiments and drawings. Obviously, the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the embodiments described below are part of the embodiments of the present invention, rather than all the embodiments, wherein the technical features involved in each embodiment can be combined with each other as long as there is no conflict between them.

[0026] The present invention takes the feature selection of biomedical data sets in the field of machine learning as an example, and guides the optimization and evolution of the feature selection process by constructing a long short-term memory mechanism. After the population is initialized, the method guides the search operator based on the long short-term memory to dynamically screen and update the feature subset to improve the adaptability of the population. In order to verify the effectiveness of the method of the present invention in feature selection, the present invention compares the classification accuracy, feature subset size and training time with the traditional machine learning method for the same task scenario.

[0027] The specific embodiment is to identify the disease categories in biomedical data. The specific process is as follows: First, collect and preprocess biomedical data sets, such as data on cancer types, immune diseases and other pathological states, which have been manually labeled with correct classification labels. In this embodiment, the category information in the data is called "label". Next, the data features and the corresponding labels are input into a feature selection model guided by long short-term memory for training. The model improves the recognition of key features of the data by gradually optimizing feature selection, and finally generates a prediction model with an optimal feature subset. In the test phase, new unlabeled data is input into the trained model, and the model outputs the prediction results. Then, the prediction results are compared with the true labels of the data, and the classification accuracy of the model is calculated to evaluate the performance of the model. Through this embodiment, the effectiveness and practicality of the present invention in feature selection optimization are verified.

[0028] like Figure 1 As shown, a high-dimensional feature selection method based on long short-term memory guided evolution provided by an embodiment of the present invention includes the following steps:

[0029] Step 1: Construct a training data set and preprocess and normalize the data.

[0030] Select multiple mainstream biomedical data sets, such as CNS, arcene, CLLSUB, Prostate, Lung, Ovarian, SMKCAN, CrohnDisease, GLI, BreastCancer1 and BreastCancer2; construct a training set based on the above biomedical data sets, and at the same time, divide part of the data as a test set for subsequent model verification. Preprocess the data in the training set, including missing value processing, noise filtering and other operations, and convert the data to the same dimensional range through normalization or standardization methods to ensure consistent effect of feature selection model training.

[0031] Step 2: Initialize the population and generate a population size that satisfies evolution.

[0032] In the present invention, the purpose of the population initialization stage is to generate initial individuals with sufficient diversity to help determine the direction of the evolutionary process. Therefore, the quality of the population directly affects the efficiency of the entire optimization process and the quality of the final solution. However, relying solely on random initialization strategies often does not work well in high-dimensional feature selection problems. Especially in high-dimensional feature spaces, most features are redundant, and random initialization often contains a large number of redundant features, resulting in insufficient population diversity, which affects the convergence speed of the algorithm and increases the calculation time. In order to solve this problem, the present invention uses an improved population initialization strategy for generating diverse individuals with different feature densities in high-dimensional feature spaces. This strategy controls the number of features selected by each individual through linear interpolation to ensure that individuals in the population differ in the diversity of feature selection. The specific process is as follows:

[0033] First, each individual in the initial population is represented as a binary vector with a length equal to the dimension D of the feature space. i , each bit of its binary vector x i,j Indicates whether a feature is selected. i,j = 0, indicating that the jth feature is not selected; if x i,j =1, indicating that the jth feature is selected. In order to ensure that the number of features selected by each individual is controlled, the present invention determines the number of features selected by each individual K by the following formula: i , whose value is at the minimum value D st To the maximum value D changes between:

[0034]

[0035] Among them, K i is the number of features selected by the i-th individual, D st is the minimum number of features selected in the initialization phase, D is the total dimension of the feature space, and N is the total number of individuals in the population. By linear interpolation, K i Will be in D st Dynamically changes between and D, ensuring that the number of features selected by each individual is diverse, and gradually changes from the number of features to D as the population evolves. st The transition is when the number of features approaches D.

[0036] For each individual X i The present invention ensures that the total number of selected features is exactly K by the following constraints: i :

[0037]

[0038] This constraint ensures that each individual chooses exactly K ifeatures are selected and the remaining features remain unselected.

[0039] Through this improved population initialization strategy, the generated initial population can effectively avoid the interference of redundant features, so that each individual has significant differences in the diversity and density of feature selection, which can provide a richer search space for the subsequent genetic algorithm evolution. In this way, not only the exploration ability of the algorithm is improved, but also the convergence speed can be accelerated, and the efficiency and accuracy of feature selection can be improved.

[0040] Step three, long short-term memory guides genetic algorithm evolution.

[0041] In high-dimensional feature selection problems, the feature space usually contains a large number of redundant features, which makes it very difficult to find an effective feature subset in the huge search space. In order to effectively solve this problem, Figure 2 As shown, the present invention proposes a genetic algorithm evolution method based on long short-term memory guidance, which guides the evolution process of the population by utilizing the historical information in the long-term memory set LTM and the short-term memory set STM, thereby accelerating convergence and improving the accuracy and efficiency of feature selection. The evolution process of the long short-term memory guided genetic algorithm mainly includes crossover operations and mutation operations. By combining the historical memory information stored in LTM and STM, these operations can more effectively guide the population to produce offspring with high-quality features.

[0042] The crossover operation generates new offspring individuals by combining the genetic characteristics of two parent individuals. Single-point crossover: select a crossover point, truncate the feature vectors of the two parent individuals at the crossover point, and exchange the two parts of the features to generate two offspring individuals; double-point crossover: select two crossover points, exchange the feature vectors of the two parent individuals between the two crossover points to generate a new feature combination; uniform crossover: randomly generate a mask vector composed of 0 and 1, and each position in the mask is 1, indicating that the feature of parent 1 is selected, and when it is 0, it indicates that the feature of parent 2 is selected.

[0043] In the present invention, the crossover operation includes three strategies: single-point crossover, double-point crossover and uniform crossover. These strategies can ensure that the offspring inherits the excellent features of the parent generation and introduce new feature combinations. The combination of crossover strategies enables the offspring individuals to effectively inherit the excellent features of the parent generation and increase the diversity of features, providing a richer search space for the subsequent evolution of the genetic algorithm.

[0044] The purpose of the mutation operation is to increase the diversity of the population by randomly changing certain feature bits in individual genes to prevent the population from converging prematurely. In classical genetic algorithms, the mutation operation is usually completed by randomly selecting gene sites and modifying them. However, this method lacks guidance and may lead to slow convergence. In order to improve the efficiency of the mutation operation, the present invention proposes a mutation operator guided by long short-term memory. Specifically, the mutation operation is guided by the historical feature selection information in the long-term memory set LTM and the short-term memory set STM. LTM records the features that perform best throughout the evolutionary process, while STM saves the feature information that has performed well in the recent period. Based on this information, the mutation probability of each feature can be calculated.

[0045] (3.1) Mutation probability calculation: First, the memory information in LTM and STM is fused to obtain the comprehensive memory value fmu[i] of each feature. fmu[i] represents the importance of feature i in historical and recent memory. The specific calculation method is as follows:

[0046]

[0047] The higher the value, the better the performance of feature i in history and recent times, and the higher the possibility of mutation.

[0048] (3.2) Mutation operation: For each feature i, according to the calculated mutation probability fmu[i], if the generated random number is less than fmu[i], the feature mutates, that is, changes from 0 to 1 or from 1 to 0. In this way, the mutation operation can be flexibly guided to generate more potential feature combinations while maintaining the diversity of the population.

[0049] Through the mutation operation guided by long short-term memory, it is possible to increase the diversity of feature selection while maintaining the protection of high-quality features, thereby improving the convergence speed and selection performance of the genetic algorithm.

[0050] Step 4: The environment selects the next generation population.

[0051] In the problem of high-dimensional feature selection, the population selection strategy of the genetic algorithm is crucial and can effectively balance the diversity and convergence of the solution. In order to ensure the convergence performance and diversity of the algorithm, the present invention adopts the elite retention strategy in NSGA-II in the environment selection stage, which combines fast non-dominated sorting and crowding distance comparison to obtain a high-quality Pareto solution set. The following is the specific process of environment selection in the present invention.

[0052] (4.1) Population merging. In the evolutionary process of each generation, the parent population P and the offspring population O are first merged to form a new population R of size 2N, where N is the population size. The merged population contains all individuals of the parent and offspring generations.

[0053] (4.2) Fast non-dominated sorting. The purpose of fast non-dominated sorting is to divide individuals into different non-dominated fronts according to the dominance relationship between solutions. Specifically, for each individual p, the following two values ​​are calculated: the dominance number n p and the dominated set S p If individual x i No better than individual x on all objectives j Bad, and better than x in at least one objective j , then x i Dominate x j Based on this dominance relationship, the population R is divided into multiple non-dominated fronts. The division of the non-dominated front follows the following steps: Find all individuals with a dominance number of 0, which are not dominated by any other individuals, and thus constitute the first layer of non-dominated fronts. For each individual p in the first layer of non-dominated fronts, traverse all individuals q in its dominance set and reduce the dominance number n of q q When the dominance number n of an individual q When it decreases to 0, the individual q is added to the second-layer non-dominated frontier, and so on, until all individuals are sorted.

[0054] (4.3) Calculation of crowding distance. In order to solve the problem of different numbers of individuals in different non-dominated fronts, the crowding distance is used to measure the sparseness of individuals in the population. Individuals with larger crowding distances are farther away from other individuals and more evenly distributed, which helps to maintain the diversity of the population. The process of calculating the crowding distance is as follows: For each individual in the non-dominated front, sort them in ascending order according to the value of each objective function. For each objective function, the boundary solution (minimum and maximum) can be found, and the crowding distance value of the boundary solution is set to infinity. For non-boundary solutions, the crowding distance is calculated based on the difference in objective function values ​​between adjacent solutions. Specifically, the crowding distance is the sum of the differences between adjacent solutions on the objective function. By calculating the crowding distance, it can be ensured that individuals with a long distance are selected to enter the next generation of the population, thereby avoiding premature convergence of the population and maintaining diversity.

[0055] (4.4) Select the next generation population. According to the fast non-dominated sort and crowding distance, select individuals to enter the next generation population P′ in turn. The specific process is as follows: Add individuals from each layer to the next generation population P′ in the order of the non-dominated front. After adding individuals from a certain layer, check the number of individuals in the current population P′. If the predetermined population size N has been reached, stop adding. If the number of individuals on a certain front exceeds N, select according to the crowding distance of each individual. Prioritize individuals with large crowding distances to ensure the diversity of the population. The selected individuals will be added to the next generation population until the population size is exactly N.

[0056] Through the above-mentioned environment selection strategy, the present invention can effectively screen out individuals with high fitness, ensure the retention of excellent solutions, and maintain the diversity of the population through the crowding distance, thereby effectively improving the global search ability and convergence speed of the genetic algorithm.

[0057] Step five: Long short-term memory dynamic update mechanism is used to update the memory set.

[0058] In order to improve the algorithm's search capability and avoid falling into local optimality, the present invention introduces a long short-term memory dynamic update mechanism, which combines the long-term memory set LTM and the short-term memory STM, and dynamically updates the memory through the memory forgetting decay strategy. The main goal of this step is to dynamically select appropriate features to update LTM and STM in order to better guide the evolution process of the genetic algorithm. The following is the specific implementation process.

[0059] (5.1) Dynamically select the number of retained features. In order to adapt to the feature selection requirements of different population sizes during the feature selection process, the present invention adopts a logarithmic scale mapping method to dynamically adjust the number of retained features N. R First, define the following variables: N SF : The number of features selected in the current population, F min : The minimum number of features that should be retained and, F max : Maximum number of features that should be retained.

[0060] By logarithmic scaling, the number of features N R It will change smoothly so that the appropriate number of features can be selected for update in different situations. The formula is as follows:

[0061]

[0062] Where D represents the total number of features, μ is a normalized value, which is used to adjust the number of features N retained R , so that it is in F min and F max If N SF Less than F max, then keep all selected features; if N SF Greater than F min , redundant features are removed through logarithmic scaling to ensure a reasonable number of features.

[0063] (5.2) Calculate the feature selection frequency. In order to update LTM and STM according to the selection of features in the population, we first need to calculate the selection frequency of each feature fre j For each feature j in the population, its selection frequency is the number of times the feature appears in all individuals, and the formula is as follows:

[0064] Fre=(fre1,fre2,…,fre D )#(5)

[0065]

[0066] Among them, x i,j Indicates whether the i-th individual has selected the j-th feature. The selection frequency of each feature is obtained by accumulating the number of selections of each feature in the population.

[0067] (5.3) Feature sorting and allocation. According to the calculated selection frequency fre j , sort the features in descending order. Then, the top 0.3*N R The features are added to the long-term memory set LTM to save those features that are frequently selected and perform well. These features are usually features that have been verified to be effective for a long time and can provide a stable evolutionary direction. Features will be randomly selected and added to the short-term memory set STM. Although these features are selected less frequently, they have high potential and can provide new exploration directions for the algorithm.

[0068] (5.4) Update LTM and STM. The update of STM and STM adopts the memory forgetting decay strategy, which dynamically adjusts the features in each memory set by combining old memory information and new feature information. This update strategy helps balance the use of existing information and the exploration of new information, avoiding over-reliance on old features and falling into local optimality.

[0069] (5.4.1) Update the short-term memory set LTM. The long-term memory set LTM is updated as follows:

[0070] LTM=α·LTM+(1-α)·LTV

[0071] Among them, α is the forgetting decay rate, and the present invention sets α = 0.7. This formula indicates that LTM is mainly determined by previous memory information, and new information has little impact on LTM, thereby ensuring that LTM can retain long-term verified and effective features.

[0072] Updated long-term memory value LTV j The calculation is as follows:

[0073]

[0074] Among them, p m is the memory value associated with the feature, F L It is the set of features in long-term memory.

[0075] (5.4.2) Update the short-term memory set STM. The update of the short-term memory set STM emphasizes the weight of new information. The update formula is as follows:

[0076] STM=(1-α)·STM+α·STV#(8)

[0077] Among them, STM is the updated short-term memory value. Since short-term memory mainly depends on new information, the new features have a greater impact on STM during updating. The updated short-term memory value STV j The calculation is as follows:

[0078]

[0079] Among them, F S is the set of features in the short-term memory.

[0080] Through this dual memory update mechanism, LTM is responsible for preserving those effective features that have been verified for a long time and have stable performance, while STM remains sensitive to new information and promotes the algorithm's exploration ability. The stability of long-term memory ensures the orderliness of the evolutionary process, while the flexibility of short-term memory ensures that the algorithm can adapt to changes in the environment and avoid falling into local optimality. The present invention can efficiently identify and retain the most valuable features for the current task, while continuously exploring new potential solutions through the flexibility of short-term memory, thereby improving the global search capability and convergence speed of the feature selection process.

[0081] The present invention has achieved remarkable results in the field of high-dimensional feature selection, and has shown obvious advantages in multiple key indicators such as classification accuracy, feature subset size, and running time. Figure 3 As shown in Figure 2, the present invention achieved the highest average classification accuracy in 5 of the 11 data sets, namely CNS, arcene, Prostate, BreastCancer1 and BreastCancer2. Figure 4 As shown in FIG. 1 , the present invention obtains the smallest average feature subset size on 9 data sets, namely CNS, CLLSUB, Prostate, Lung, Ovarian, SMKCAN, CrohnDisease, GLI and BreastCancer1. Figure 5 As shown, the present invention achieves the shortest average running time on 8 data sets, namely arcene, Lung, Ovarian, SMKCAN, CrohnDisease, GLI, BreastCancer1 and BreastCancer2.

[0082] These advantages indicate that the present invention has broad application prospects and can effectively solve the problem of feature selection in high-dimensional data in practical applications. The present invention can accurately select the most representative and distinguishing feature subsets from a large number of features through an improved feature selection method. Experimental results show that compared with traditional methods, the present invention has significantly improved classification accuracy. This means that the features selected by the method of the present invention are more in line with the requirements of the classification task, thereby improving the predictive ability and robustness of the model. Traditional feature selection methods often improve classification accuracy while causing the selected feature subset to be too large, and cannot effectively reduce the feature dimension. However, in the process of optimizing feature selection, the present invention not only ensures a higher classification accuracy, but also significantly reduces the size of the feature subset. By accurately selecting the most informative features, the present invention can retain key features while reducing redundant information, thereby achieving a more concise and efficient feature representation. The feature selection process in high-dimensional data sets usually requires a long computing time, especially when processing large-scale data. However, the present invention adopts optimized algorithms and strategies to significantly reduce the computational complexity and running time of the feature selection process. By reducing unnecessary calculations, the present invention can complete the feature selection task in a shorter time, thereby improving the practicality and efficiency of the algorithm on large-scale data sets.

[0083] In summary, the present invention has shown significant advantages in high-dimensional feature selection, which can not only improve classification accuracy, but also optimize feature subset size and reduce running time. These advantages make the present invention widely applicable in practical applications, especially in the field of high-dimensional data processing, and can provide a more efficient and accurate feature selection solution. Therefore, the present invention has broad application prospects in high-dimensional feature selection problems and has important theoretical value and practical significance.

[0084] The above embodiments are used to illustrate the present invention rather than to limit the present invention. Any modification and change made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A high-dimensional feature selection method based on long short-term memory guided evolution, characterized in that: The following steps are involved: Constructing a training data set and preprocessing the data in the training data set, wherein the training data set includes multiple types of high-dimensional biomedical data, the high-dimensional biomedical data has different sample numbers, feature numbers and category labels, and the preprocessing includes outlier processing and normalization; The preprocessed training data set is used as input to initialize the population and generate a population size that satisfies evolution. The population includes a variety of initial individuals. Each initial individual is regarded as a specific feature selection scheme, which defines a feature subset that needs to be retained in the current evolution process. Generate new individuals using a long short-term memory search operator, wherein the long short-term memory search operator includes a long short-term memory guided crossover operation and a mutation operation; The fitness of the new individuals is evaluated to evaluate the classification accuracy and feature compression rate of the feature subset of the new individuals, and the top N individuals with the highest fitness are retained through environmental selection to enter the next generation population to ensure that the population evolves in the right direction; in the later stage of each evolutionary iteration, the long-term memory set and the short-term memory set are updated using the long-term and short-term memory dynamic update mechanism; wherein the long-term memory set is used to record historical features with excellent performance to preserve global excellent feature information, and the short-term memory set is used to record features that frequently appear in the recent period to improve the diversity of features, thereby dynamically adjusting the weights of features; When the evolution process meets the iteration termination condition, the individual with the best fitness is selected from the population to obtain the optimal feature subset for model training and testing.

2. The high-dimensional feature selection method based on long short-term memory guided evolution according to claim 1, characterized in that: The training data set is derived from a public high-dimensional biomedical data set, and the training data set includes CNS, arcene, CLLSUB, Prostate, Lung, Ovarian, SMKCAN, CrohnDisease, GLI, BreastCancer1 and BreastCancer2.

3. The high-dimensional feature selection method based on long short-term memory guided evolution according to claim 1, characterized in that: The population initialization includes: assigning different numbers of features to initial individuals by a linear interpolation method to ensure differences between the individuals, so as to generate a diverse population with different feature densities in a high-dimensional search space.

4. The high-dimensional feature selection method based on long short-term memory guided evolution according to claim 1, characterized in that: The long short-term memory search operator includes a crossover operation and a mutation operation; wherein the crossover operation adopts a plurality of crossover methods, including single-point crossover, double-point crossover and uniform crossover, and one of the crossover methods is randomly selected during the evolution process; through the combination of a plurality of crossover methods, it is ensured that the offspring can inherit the excellent characteristics of the parent generation, and at the same time, a new feature combination is introduced, thereby increasing the diversity of the population; the mutation operation is guided by the historical feature selection information in the long-term memory set and the short-term memory set, thereby accelerating the convergence speed of the algorithm.

5. The high-dimensional feature selection method based on long short-term memory guided evolution according to claim 1, characterized in that: The method of updating the memory set using the long-term and short-term memory dynamic update mechanism is specifically as follows: a logarithmic scale mapping method is used to dynamically adjust the number of features used to update the memory set; a feature set used to update the memory is selected according to the selection frequency of the features; a memory forgetting decay strategy is used to dynamically adjust the memory set by combining old memory information and new feature information; the update of the long-term memory set is used to consolidate excellent features that have been repeatedly verified, and gradually forget features with poor performance; the short-term memory set is used to capture new features and explore new feature combinations, and quickly forget features with poor performance.

6. The high-dimensional feature selection method based on long short-term memory guided evolution according to claim 1, characterized in that: The fitness evaluation of the new individual includes: using KNN as a classifier, setting the number of neighbors K to 5, and using 5-fold cross validation in each evolutionary iteration to comprehensively evaluate the classification accuracy and feature compression rate of the feature subset.

Images