A high-dimensional feature selection method based on long short-term memory-guided evolution.

Abstract

This invention discloses a high-dimensional feature selection method based on long short-term memory-guided evolution, comprising: using a long-term memory set and a short-term memory set to guide the evolution of a genetic algorithm; the long-term memory set is responsible for accumulating globally excellent feature information during the evolution process, while the short-term memory set is used to record recently well-performing feature information; during the evolution process, a memory-forgetting strategy is used to continuously update the long-term memory set and the short-term memory set to better guide the evolutionary direction of the genetic algorithm; this dual-memory update mechanism takes into account both the utilization of existing information and the exploration of new information, with the long-term memory set preserving those effective features that have been validated over a long period, providing a stable evolutionary direction; while the short-term memory set remains sensitive to new information, promoting the algorithm to explore new feature combinations. This invention achieves higher classification accuracy while having a smaller feature subset, providing an effective direction for solving the feature selection problem.

Description

Technical Field

[0001] This invention relates to the field of high-dimensional feature selection in machine learning, and in particular to a high-dimensional feature selection method based on long short-term memory-guided evolution, which is used to improve the performance of algorithms on feature selection of high-dimensional datasets. Background Technology

[0002] Against the backdrop of the rapid development of big data and artificial intelligence, data is exhibiting explosive growth in both dimensionality and quantity across various fields, particularly in natural language processing, image retrieval, speech signal processing, and gene analysis. The high-dimensionality of this data makes feature selection a crucial technical means for solving big data problems. Feature selection, as an important data preprocessing technique, is widely used. By filtering representative, informative, and highly relevant features from a large pool, it reduces data redundancy and improves model performance. In high-dimensional data scenarios, feature selection not only reduces computational resource consumption but also enhances the generalization and interpretability of learning algorithms. Therefore, how to effectively perform feature selection without losing valuable information has become a key issue in data science and machine learning.

[0003] High-dimensional data often contains a large number of redundant and irrelevant features. Indiscriminately adding all of these features to a model during training increases model complexity, easily leads to overfitting, and reduces the model's generalization ability. Furthermore, excessively high feature dimensionality increases data processing time and computational costs. Therefore, effectively removing redundant and noisy features through feature selection can significantly improve model performance and enhance interpretability. However, feature selection faces several challenges in high-dimensional data scenarios. First, the feature space of high-dimensional data is extremely large, and the feature selection process is equivalent to finding the optimal subset of features within this vast space. Second, complex relationships exist between different features, making it difficult to select features that are relevant to the target variable and not redundant. Finally, feature selection needs to consider both the characteristics of the data and the task at hand; different learning algorithms may have different requirements for features, making it difficult for a single method to meet diverse application needs. To address these challenges, academia and industry have proposed various feature selection methods.

[0004] Traditional feature selection methods can be broadly categorized into three types: filtering, wrapping, and embedding. Each method has its own advantages and disadvantages. Filtering methods select features based on their statistical properties, typically using univariate analysis metrics such as variance, mutual information, chi-square test, and correlation coefficient. These methods are computationally fast and independent of specific learning algorithms, making them suitable for initial feature screening. However, filtering ignores the interactions between features, potentially leading to information loss. Wrapping methods combine the feature selection process with learning algorithms, selecting the optimal feature combination by evaluating the performance of different feature subsets under a specific algorithm. Common wrapping methods include forward selection, backward elimination, and recursive feature elimination. Wrapping methods can discover the combined effects of features, but they have high computational complexity, especially when dealing with high-dimensional data, where computational costs are enormous and overfitting is a risk. Embedding methods directly embed the feature selection process into model training, such as decision trees and L1-regularized linear regression. These methods automatically select important features during model training without requiring additional computation of feature subset evaluation metrics. Embedding methods can achieve efficient feature selection, but their effectiveness depends on the specific model, thus limiting their generalization ability.

[0005] Due to the limitations of traditional methods in processing high-dimensional data, intelligent optimization algorithms have gradually become an important research direction in feature selection in recent years. Intelligent optimization algorithms search for optimal feature subsets in the feature space by simulating natural evolution and swarm intelligence behavior, exhibiting good global search capabilities and strong adaptability. Commonly used intelligent optimization algorithms include genetic algorithms, particle swarm optimization (PSO), and ant colony optimization (ACO). Genetic algorithms are optimization algorithms based on the principles of natural selection and genetic mutation. They guide the population to evolve through operations such as selection, crossover, and mutation to search for the optimal solution. In feature selection problems, genetic algorithms encode feature subsets as individuals in the population and iteratively optimize to obtain the optimal feature combination. However, in high-dimensional feature spaces, genetic algorithms are prone to getting trapped in local optima and have slow convergence speeds. Particle swarm optimization (PSO) achieves global search by simulating the foraging behavior of bird flocks. Each particle represents a feature subset and updates its position based on its own and its neighbors' optimal positions. PSO is simple and efficient, but in high-dimensional data, it may converge to local optima and struggle to escape local extrema. Ant Colony Algorithm (ACO) simulates the path selection behavior of ants foraging by associating the probability of choosing each path with the importance of features along that path. ACO performs well on small-scale problems, but it is computationally expensive in high-dimensional data scenarios and path selection is prone to getting stuck in local optima.

[0006] However, existing feature selection methods still suffer from several problems in the feature space, exhibiting high computational complexity and difficulty in efficiently handling ultra-high-dimensional data. Furthermore, most current methods rely on fixed parameter settings and model structures, lacking adaptability to different data types and feature distributions, thus affecting the algorithm's versatility and adaptability. Therefore, designing a feature selection algorithm with higher global search capabilities and stronger adaptability to data characteristics without increasing algorithm complexity, thereby improving efficiency and selection accuracy, requires further exploration. Summary of the Invention

[0007] To address the shortcomings and defects of existing feature selection methods, the present invention aims to provide a high-dimensional feature selection method based on long short-term memory-guided evolution.

[0008] The objective of this invention is achieved through the following technical solution: a high-dimensional feature selection method based on long short-term memory-guided evolution, comprising the following steps:

[0009] A training dataset is constructed, and the data in the training dataset is preprocessed. The training dataset includes various types of high-dimensional biomedical data with different sample numbers, feature numbers, and class labels to ensure the applicability and effectiveness of the algorithm in diverse high-dimensional data scenarios. The preprocessing includes outlier handling and normalization to improve data consistency and algorithm robustness.

[0010] The preprocessed training dataset is used as input to initialize the population and generate a population that satisfies the evolutionary requirements. The population includes diverse initial individuals, and each initial individual is regarded as a specific feature selection scheme. The feature selection scheme defines the subset of features that need to be retained in the current evolutionary process.

[0011] New individuals are generated using a long short-term memory search operator, which includes long short-term memory-guided crossover and mutation operations. By combining historical memory information stored in long-term and short-term memory, the evolutionary process of the population is guided, thereby accelerating convergence and improving the accuracy and efficiency of feature selection.

[0012] The new individuals are subjected to fitness evaluation to assess the classification accuracy and feature compression rate of their feature subsets. The top N individuals with the highest fitness are retained in the next generation of the population through environmental selection to ensure that the population evolves in the right direction. In the later stages of each evolutionary iteration, the memory set is updated using a long short-term memory dynamic update mechanism. The long short-term memory set is used to record historically excellent features to preserve global good feature information, while the short short-term memory set is used to record recently frequently occurring features to improve feature diversity and thus dynamically adjust feature weights.

[0013] When the evolutionary process meets the iteration termination condition, the individual with the best fitness is selected from the population to obtain the optimal feature subset for model training and testing.

[0014] Furthermore, the training dataset is derived from publicly available high-dimensional biomedical datasets, including CNS, arcene, CLLSUB, Prostate, Lung, Ovarian, SMKCAN, CrohnDisease, GLI, BreastCancer1, and BreastCancer2.

[0015] Furthermore, the population initialization includes: assigning different numbers of features to initial individuals using a linear interpolation method to ensure the differences between the individuals, so as to generate a diverse initial population with different feature densities in a high-dimensional search space.

[0016] Furthermore, the long short-term memory search operator includes crossover and mutation operations; wherein, the crossover operation employs multiple crossover methods, including single-point crossover, double-point crossover, and uniform crossover, and one of these methods is randomly selected during the evolution process; by combining multiple crossover methods, it is ensured that offspring can inherit the superior characteristics of their parents, while introducing new feature combinations, thereby increasing the diversity of the population; the mutation operation is guided by historical feature selection information from the long-term memory set and the short-term memory set, accelerating the convergence speed of the algorithm.

[0017] Furthermore, the method of using the long short-term memory dynamic update mechanism to update the memory set specifically involves: using a logarithmic scaling method to dynamically adjust the number of features used to update the memory set; selecting the feature set used to update the memory based on the feature selection frequency; using a memory forgetting decay strategy to dynamically adjust the memory set by combining old memory information and new feature information; the update of the long-term memory set is used to consolidate repeatedly verified excellent features and gradually forget poorly performing features; the short-term memory set is used to capture new features and explore new feature combinations, and quickly forget poorly performing features.

[0018] Furthermore, the fitness evaluation of the new individual includes: using KNN as a classifier, setting the number of neighbors K to 5, and using 5-fold cross-validation in each evolutionary iteration to comprehensively evaluate the classification accuracy and feature compression rate of the feature subset.

[0019] The beneficial effects of this invention are as follows: Improved feature selection accuracy: KNN evaluation accurately measures the performance of feature subsets in classification tasks, thereby effectively selecting feature combinations with high classification performance. Dynamic optimization and adaptability: The evolutionary mechanism guided by long short-term memory, combined with the dynamic adjustment of long-term and short-term memory, makes the feature selection process more adaptable to changes in feature distribution, avoiding local optima traps. High computational efficiency: The memory decay strategy can effectively filter out invalid features, significantly improving computational efficiency and making it suitable for various high-dimensional data scenarios. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the system structure of the high-dimensional feature selection method for long short-term memory-guided evolution of the present invention;

[0021] Figure 2 This is a flowchart of the Long Short-Term Memory Guided Evolutionary Algorithm in an embodiment of the present invention;

[0022] Figure 3 This is a comparison chart of the average classification accuracy of this invention and other feature selection methods in 11 high-dimensional medical datasets;

[0023] Figure 4 This is a comparison chart of the average feature subset size of the present invention and other feature selection methods in 11 high-dimensional medical datasets;

[0024] Figure 5 This is a comparison chart of the average running time of the present invention and other feature selection methods on 11 high-dimensional medical datasets. Detailed Implementation

[0025] To make the objectives, technical methods, inventive features, and advantages of this invention clearer, the high-dimensional feature selection method based on long short-term memory-guided evolution of this invention will be further described in detail below with reference to embodiments and accompanying drawings. Obviously, the specific embodiments described herein are only for explaining this invention and are not intended to limit this invention. Furthermore, the embodiments described below are only some embodiments of this invention, not all embodiments, and the technical features involved in each embodiment can be combined with each other as long as there is no conflict between them.

[0026] This invention takes feature selection in biomedical datasets within the field of machine learning as an example. It constructs a long short-term memory (LSTM) mechanism to guide the optimization and evolution of the feature selection process. After population initialization, this method uses LTM to guide the search operator, dynamically filtering and updating the feature subset to improve population adaptability. To verify the effectiveness of this method in feature selection, it compares it with traditional machine learning methods in terms of classification accuracy, feature subset size, and training time for the same task scenario.

[0027] The specific embodiment described herein identifies disease categories in biomedical data. The specific process is as follows: First, biomedical datasets, such as data on cancer types, immune diseases, and other pathological states, are collected and preprocessed. These datasets have been manually labeled with correct classification tags. In this embodiment, the category information in the data is referred to as a "label." Next, the data features and corresponding labels are input together into a feature selection model guided by long short-term memory for training. This model improves the identification of key features of the data by progressively optimizing feature selection, ultimately generating a prediction model with an optimal subset of features. During the testing phase, unlabeled new data is input into the trained model, and the model outputs prediction results. Then, the prediction results are compared with the true labels of the data to calculate the model's classification accuracy, thereby evaluating the model's performance. This embodiment verifies the effectiveness and practicality of the present invention in feature selection optimization.

[0028] like Figure 1 As shown, an embodiment of the present invention provides a high-dimensional feature selection method based on long short-term memory-guided evolution, comprising the following steps:

[0029] Step 1: Construct a training dataset and preprocess and normalize the data.

[0030] Several mainstream biomedical datasets were selected, such as CNS, arcene, CLLSUB, Prostate, Lung, Ovarian, SMKCAN, CrohnDisease, GLI, BreastCancer1, and BreastCancer2. A training set was constructed based on these datasets, and a portion of the data was used as a test set for subsequent model validation. The data in the training set underwent preprocessing, including missing value handling and noise filtering. Normalization or standardization methods were used to convert the data to the same dimensionality to ensure consistent training performance of the feature selection model.

[0031] Step 2: Population initialization, generating a population size that satisfies evolutionary requirements.

[0032] In this invention, the purpose of the population initialization phase is to generate initial individuals with sufficient diversity to help determine the direction of the evolutionary process. Therefore, the quality of the population directly affects the efficiency of the entire optimization process and the quality of the final solution. However, simply relying on random initialization strategies often yields poor results in high-dimensional feature selection problems. Especially in high-dimensional feature spaces, most features are redundant, and random initialization often includes a large number of redundant features, leading to insufficient population diversity, which in turn affects the convergence speed of the algorithm and increases computation time. To address this issue, this invention uses an improved population initialization strategy to generate diverse individuals with different feature densities in a high-dimensional feature space. This strategy controls the number of features selected by each individual through linear interpolation, ensuring that individuals in the population differ in the diversity of feature selection. The specific process is as follows:

[0033] First, each individual in the population is initialized as a binary vector with a length equal to the dimension D of the feature space. For each individual X... i Each bit of its binary vector x i,j Both indicate whether a feature is selected. If x i,j =0, indicating that the j-th feature was not selected; if x i,j =1 indicates that the j-th feature is selected. To ensure that the number of features selected by each individual is controlled, this invention determines the number of features K selected by each individual using the following formula. i Its value is at the minimum value D st Variation between the maximum value D and the maximum value D:

[0034]

[0035] Among them, K i D is the number of features selected by the i-th individual. st K is the minimum number of features selected during the initialization phase, D is the total dimension of the feature space, and N is the total number of individuals in the population. Using linear interpolation, K... i In D st The dynamic change between D and the number of features ensures that the number of features chosen by each individual is diverse, and gradually changes from a feature number of D as the population evolves. st The number of features approaches D.

[0036] For each individual X i The present invention ensures that the total number of selected features is exactly K through the following constraints. i :

[0037]

[0038] This constraint guarantees that each individual chooses exactly K from the binary vector. iOne feature is selected, while the rest remain unselected.

[0039] This improved population initialization strategy effectively avoids interference from redundant features, ensuring that each individual exhibits significant differences in the diversity and density of feature selection. This provides a richer search space for subsequent genetic algorithm evolution. Consequently, it not only enhances the algorithm's exploration capabilities but also accelerates convergence and improves the efficiency and accuracy of feature selection.

[0040] Step 3: Long Short-Term Memory guides the evolution of the genetic algorithm.

[0041] In high-dimensional feature selection problems, the feature space typically contains a large number of redundant features, making it extremely difficult to find an effective subset of features within the vast search space. To effectively address this problem, methods such as... Figure 2 As shown, this invention proposes a genetic algorithm evolution method based on Long Short-Term Memory (LTM). By utilizing historical information from the Long-Term Memory (LTM) and Short-Term Memory (STM) sets, the evolutionary process of the population is guided, thereby accelerating convergence and improving the accuracy and efficiency of feature selection. The evolutionary process of the LTM-guided genetic algorithm mainly includes crossover and mutation operations. By combining the historical memory information stored in the LTM and STM, these operations can more effectively guide the population to produce offspring with high-quality characteristics.

[0042] Crossover generates new offspring by combining the genetic characteristics of two parent individuals. Single-point crossover: A crossover point is selected, the feature vectors of the two parent individuals are truncated at the crossover point, and the two parts of the feature vectors are swapped to generate two offspring individuals. Two-point crossover: Two crossover points are selected, and the feature vectors of the two parent individuals are swapped between the two crossover points to generate a new feature combination. Uniform crossover: A mask vector consisting of 0s and 1s is randomly generated, where each position in the mask is 1 indicating that the features of parent 1 are selected, and 0 indicates that the features of parent 2 are selected.

[0043] In this invention, the crossover operation includes three strategies: single-point crossover, double-point crossover, and uniform crossover. These strategies ensure that offspring inherit the superior characteristics of their parents while introducing new combinations of features. The combination of crossover strategies enables offspring individuals to effectively inherit the superior characteristics of their parents, while increasing feature diversity and providing a richer search space for the subsequent evolution of the genetic algorithm.

[0044] The purpose of mutation is to increase population diversity and prevent premature convergence by randomly altering certain eigenlocutions in an individual's genes. In classical genetic algorithms, mutation is typically performed by randomly selecting and modifying gene loci. However, this method lacks guidance and can lead to slow convergence. To improve the efficiency of mutation, this invention proposes a mutation operator guided by Long Short-Term Memory (LTM). Specifically, the mutation is guided by historical feature selection information from the LTM and STM sets. The LTM records the best-performing features throughout the evolutionary process, while the STM preserves information on recently performing features. Based on this information, the mutation probability of each feature can be calculated.

[0045] (3.1) Calculation of mutation probability: First, the memory information in LTM and STM is fused to obtain the comprehensive memory value fmu[i] for each feature. fmu[i] represents the importance of feature i in historical and recent memory, and the specific calculation method is as follows:

[0046]

[0047] The higher the value, the better feature i has performed historically and recently, and the higher the probability of mutation.

[0048] (3.2) Mutation operation: For each feature i, based on the calculated mutation probability fmu[i], if the generated random number is less than fmu[i], then the feature mutates, i.e., changes from 0 to 1 or from 1 to 0. In this way, mutation operations can be flexibly guided to generate more potential feature combinations while maintaining population diversity.

[0049] By using mutation operations guided by long short-term memory, it is possible to increase the diversity of feature selection while protecting high-quality features, thereby improving the convergence speed and selection performance of genetic algorithms.

[0050] Step four: Environmental selection for the next generation of population.

[0051] In high-dimensional feature selection problems, the population selection strategy of genetic algorithms is crucial, effectively balancing solution diversity and convergence. To ensure both convergence performance and diversity, this invention employs the elite retention strategy from NSGA-II in the environment selection phase. This strategy combines fast non-dominated sorting and crowding distance comparison to obtain a high-quality Pareto solution set. The specific process of environment selection in this invention is as follows.

[0052] (4.1) Merging populations. In the process of each generation of evolution, the parent population P and the offspring population O are first merged to form a new population R of size 2N, where N is the population size. The merged population contains all individuals of the parent and offspring generations.

[0053] (4.2) Fast Non-Dominated Sort. A fast non-dominated sort is performed on the merged population R to divide individuals into different non-dominated fronts based on the dominance relationships between solutions. Specifically, for each individual p, the following two values ​​are calculated: the dominance number n. p and the dominated set S p If individual x i No more than individual x in any goal j It is inferior to x, but superior to x in at least one objective. j Then x is called i Dominate x j Based on this dominance relationship, population R is divided into multiple non-dominated fronts. The division of non-dominated fronts follows these steps: Identify all individuals with a dominance number of 0, as these individuals are not dominated by any other individuals, thus forming the first layer of non-dominated fronts. For each individual p in the first layer of non-dominated fronts, iterate through all individuals q in its dominance set, decreasing the dominance number n of q. q When the number of dominance n of an individual q When the number of individuals decreases to 0, the individual q is added to the second-level non-dominated frontier, and so on, until all individuals are sorted.

[0054] (4.3) Crowding Distance Calculation. To address the issue of varying individual numbers across different non-dominated fronts, crowding distance is used to measure the sparsity of individuals within the population. Individuals with larger crowding distances are more evenly distributed and farther from other individuals, which helps maintain population diversity. The crowding distance is calculated as follows: For each non-dominated front, individuals are sorted in ascending order based on the value of each objective function. For each objective function, boundary solutions (minimum and maximum values) are found, and the crowding distance for these boundary solutions is set to infinity. For non-boundary solutions, the crowding distance is calculated based on the difference in objective function values ​​between adjacent solutions. Specifically, the crowding distance is the sum of the differences between adjacent solutions on the objective function. By calculating the crowding distance, it is ensured that individuals with greater distances are selected for the next generation of the population, thus avoiding premature population convergence and maintaining diversity.

[0055] (4.4) Selecting the Next Generation Population. Individuals are selected sequentially to enter the next generation population P′ based on the fast non-dominated sorting and crowding distance. The specific process is as follows: Individuals at each level are added to the next generation population P′ in the order of the non-dominated front. After adding individuals at a certain level, the number of individuals in the current population P′ is checked. If the predetermined population size N has been reached, addition is stopped. If the number of individuals at a certain front exceeds N, selection is based on the crowding distance of each individual. Individuals with larger crowding distances are prioritized to ensure population diversity. The selected individuals will be added to the next generation population until the population size is exactly N.

[0056] Through the above-mentioned environmental selection strategy, this invention can effectively screen out individuals with high fitness, ensure the retention of excellent solutions, and maintain population diversity through crowding distance, thereby effectively improving the global search capability and convergence speed of the genetic algorithm.

[0057] Step 5: The long short-term memory dynamic update mechanism is used to update the memory set.

[0058] To improve the algorithm's search capability and avoid getting trapped in local optima, this invention introduces a dynamic update mechanism for Long Short-Term Memory (LTM). This mechanism combines Long Memory Set (LTM) and Short Memory Set (STM) and dynamically updates the memory through a memory decay strategy. The main goal of this step is to dynamically select appropriate features to update the LTM and STM to better guide the evolutionary process of the genetic algorithm. The specific implementation process is as follows.

[0059] (5.1) Dynamically selecting the number of features to retain. To adapt to the feature selection needs of different population sizes during the feature selection process, this invention adopts a logarithmic scaling method to dynamically adjust the number of features N to retain. R First, define the following variable: N SF F: The number of features selected in the current population min The minimum number of features to be retained, F max The maximum number of features that should be retained.

[0060] Through logarithmic scaling, the number of features N R The changes will be smoothed out to allow for the selection of an appropriate number of features for updates under different circumstances. The formula is as follows:

[0061]

[0062] Where D represents the total number of features, and μ is a normalization value used to adjust the number of features N retained. R , so that it is in F min and F max The dynamic changes between them. If N SF Less than F maxIf N, then all selected features are retained; if N SF Greater than F min Then, redundant features are removed by adjusting the logarithmic scale to ensure that the number of features is reasonable.

[0063] (5.2) Calculate the feature selection frequency. To update the LTM and STM based on the selection of features in the population, it is first necessary to calculate the selection frequency (fre) of each feature. j For each feature j in the population, its selection frequency is the number of times that feature appears in all individuals, as shown in the following formula:

[0064] Fre = (fre1, fre2, ..., fre D )#(5)

[0065]

[0066] Where, x i,j This indicates whether the i-th individual selected the j-th feature. The selection frequency of each feature is obtained by summing the number of times each feature is selected in the population.

[0067] (5.3) Feature ranking and allocation. Based on the calculated selection frequency (fre) j Sort the features in descending order. Then, select the top 0.3*N features. R Features are added to the Long Memory Set (LTM) to preserve frequently selected and well-performing features. These features are typically validated over a long period and provide a stable evolutionary direction. The remaining features... These features will be randomly selected and added to the Short-Term Memory (STM) set. Although these features are selected infrequently, they have high potential and can provide new directions for the algorithm.

[0068] (5.4) Updating LTM and STM. The STM and STM updates employ a memory forgetting decay strategy, dynamically adjusting the features in each memory set by combining old memory information with new feature information. This update strategy helps balance the utilization of existing information with the exploration of new information, avoiding over-reliance on old features and getting trapped in local optima.

[0069] (5.4.1) Update the Short-Term Memory Set (LTM). The Long-Term Memory Set (LTM) is updated as follows:

[0070] LTM = α·LTM + (1-α)·LTV

[0071] Where α is the forgetting decay rate, and in this invention, α is set to 0.7. This formula indicates that LTM is mainly determined by previous memory information, and new information has a relatively small impact on LTM, thus ensuring that LTM can retain features that have been validated over a long period of time.

[0072] Updated Long-Term Memory Value (LTV) j The calculation is as follows:

[0073]

[0074] Where, p m It is the memory value associated with the feature, F L It is a set of features concentrated in long-term memory.

[0075] (5.4.2) Update the Short-Term Memory Set (STM). Updating the STM places greater emphasis on the weight of new information. The update formula is as follows:

[0076] STM=(1-α)·STM+α·STV#(8)

[0077] Here, STM represents the updated short-term memory value. Since short-term memory relies heavily on new information, new features have a significant impact on STM during updates. The updated short-term memory value is STV. j The calculation is as follows:

[0078]

[0079] Among them, F S It is a set of features concentrated in short-term memory.

[0080] Through this dual-memory update mechanism, LTM is responsible for preserving effective features that have been validated over a long period and exhibit stable performance, while STM remains sensitive to new information, enhancing the algorithm's exploratory capabilities. The stability of long-term memory ensures the orderly nature of the evolutionary process, while the flexibility of short-term memory ensures that the algorithm can adapt to changes in the environment and avoid getting trapped in local optima. This invention can efficiently identify and retain the most valuable features for the current task, while continuously exploring new potential solutions through the flexibility of short-term memory, thereby improving the global search capability and convergence speed of the feature selection process.

[0081] This invention has achieved significant results in the field of high-dimensional feature selection, demonstrating clear advantages in several key indicators such as classification accuracy, feature subset size, and running time. For example... Figure 3 As shown, this invention achieved the highest average classification accuracy on 5 out of 11 datasets: CNS, arcene, Prostate, BreastCancer1, and BreastCancer2. Figure 4 As shown, this invention obtained the smallest average feature subset size on nine datasets: CNS, CLLSUB, Prostate, Lung, Ovarian, SMKCAN, CrohnDisease, GLI, and BreastCancer1. Figure 5 As shown, this invention achieves the shortest average runtime on eight datasets: arcene, Lung, Ovarian, SMKCAN, CrohnDisease, GLI, BreastCancer1, and BreastCancer2.

[0082] These advantages demonstrate that this invention has broad application prospects and can effectively solve the feature selection problem in high-dimensional data in practical applications. Through an improved feature selection method, this invention can accurately select the most representative and discriminative feature subset from massive amounts of features. Experimental results show that compared with traditional methods, this invention significantly improves classification accuracy. This means that the features selected using the method of this invention are more in line with the requirements of the classification task, thereby improving the model's predictive ability and robustness. Traditional feature selection methods often improve classification accuracy but result in an excessively large selected feature subset, failing to effectively reduce feature dimensionality. This invention, however, optimizes the feature selection process while ensuring higher classification accuracy and significantly reducing the size of the feature subset. By accurately selecting the most informative features, this invention can reduce redundant information while retaining key features, thus achieving a more concise and efficient feature representation. Feature selection in high-dimensional datasets typically requires long computation times, especially when processing large-scale data. However, this invention employs optimized algorithms and strategies, significantly reducing the computational complexity and runtime of the feature selection process. By reducing unnecessary computation, this invention can complete the feature selection task in a shorter time, improving the practicality and efficiency of the algorithm on large-scale datasets.

[0083] In summary, this invention demonstrates significant advantages in high-dimensional feature selection, not only improving classification accuracy but also optimizing feature subset size and reducing runtime. These advantages make this invention widely applicable in practical applications, especially in fields requiring the processing of high-dimensional data, providing a more efficient and accurate feature selection solution. Therefore, this invention has broad application prospects in high-dimensional feature selection problems and possesses significant theoretical and practical value.

[0084] The above embodiments are used to explain and illustrate the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A high-dimensional feature selection method based on long short-term memory-guided evolution, characterized in that, Includes the following steps: A training dataset is constructed, and the data in the training dataset is preprocessed. The training dataset includes various types of high-dimensional biomedical data with different sample numbers, feature numbers, and class labels. The preprocessing includes outlier handling and normalization. The preprocessed training dataset is used as input to initialize the population and generate a population that satisfies the evolutionary requirements. The population includes diverse initial individuals, and each initial individual is regarded as a specific feature selection scheme. The feature selection scheme defines the subset of features that need to be retained in the current evolutionary process. New individuals are generated using a long short-term memory search operator, which includes long short-term memory-guided crossover and mutation operations; The new individuals are subjected to fitness evaluation to assess the classification accuracy and feature compression rate of their feature subsets. The top N individuals with the highest fitness are retained in the next generation of the population through environmental selection to ensure that the population evolves in the right direction. In the later stages of each evolutionary iteration, the long short-term memory (LSTM) dynamic update mechanism is used to update the long-term memory set and the short-term memory set. The long-term memory set is used to record historically excellent features to preserve global good feature information, while the short-term memory set is used to record recently frequently occurring features to improve feature diversity and thus dynamically adjust feature weights. When the evolutionary process meets the iteration termination condition, the individual with the best fitness is selected from the population to obtain the optimal feature subset for model training and testing.

2. The high-dimensional feature selection method based on long short-term memory-guided evolution as described in claim 1, characterized in that, The training dataset is derived from publicly available high-dimensional biomedical datasets, including CNS, arcene, CLLSUB, Prostate, Lung, Ovarian, SMKCAN, CrohnDisease, GLI, BreastCancer1, and BreastCancer2.

3. The high-dimensional feature selection method based on long short-term memory-guided evolution as described in claim 1, characterized in that, The population initialization includes: assigning different numbers of features to initial individuals using a linear interpolation method to ensure the differences between the individuals, so as to generate a diverse population with different feature densities in a high-dimensional search space.

4. The high-dimensional feature selection method based on long short-term memory-guided evolution as described in claim 1, characterized in that, The Long Short-Term Memory (LSTM) search operator includes crossover and mutation operations. The crossover operation employs multiple methods, including single-point crossover, double-point crossover, and uniform crossover, with one method randomly selected during the evolutionary process. This combination of methods ensures that offspring inherit the superior characteristics of their parents while introducing new feature combinations, thereby increasing population diversity. The mutation operation is guided by historical feature selection information from the long-term and short-term memory sets, accelerating the algorithm's convergence speed.

5. The high-dimensional feature selection method based on long short-term memory-guided evolution as described in claim 1, characterized in that, The specific steps of updating the memory set using the long short-term memory dynamic update mechanism are as follows: A logarithmic scaling method is used to dynamically adjust the number of features used to update the memory set; the feature set used for updating the memory is selected based on the feature selection frequency; a memory decay strategy is used to dynamically adjust the memory set by combining old memory information and new feature information; the update of the long-term memory set is used to consolidate repeatedly validated excellent features and gradually forget poorly performing features; the short-term memory set is used to capture new features and explore new feature combinations, and quickly forget poorly performing features.

6. The high-dimensional feature selection method based on long short-term memory-guided evolution as described in claim 1, characterized in that, The fitness evaluation of the new individual includes: using KNN as the classifier, setting the number of neighbors K to 5, and using 5-fold cross-validation in each evolutionary iteration to comprehensively evaluate the classification accuracy and feature compression rate of the feature subset.

Images