Feature selection method and system based on highly robust autoencoder, and terminal

Abstract

Disclosed in the present invention are a feature selection method and system based on a highly robust autoencoder, and a terminal. The method comprises: acquiring the parameter weight of a feature selection layer in a feature selection model, and carrying out an indexation operation on the parameter weight and then multiplying the parameter weight with a feature to obtain a target parameter; gradually reducing a local feature selection window in the training process of the feature selection model until the window is reduced to a feature number needing to be selected; and adding a classification module behind an autoencoder in the feature selection model, using a full connection layer to provide a classification loss optimization model, and selecting key features more beneficial to classification. On the basis of feature selection technology in combination with methods, such as mathematical modeling, statistical inference and deep learning, the present invention innovatively develops a novel feature selection method, aiming at efficiently and stably screening for key features for different types of data, filtering out redundant and noise data, and screening the key features on the basis of a downstream task objective.

Description

Feature selection method, system and terminal based on strong robust autoencoder TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a feature selection method, system, terminal and computer readable storage medium based on a strong robust autoencoder. BACKGROUND

[0002] With the advancement of technology and the acceleration of digital transformation, not only has there been an explosive growth in data volume, but there has also been a significant increase in data depth (feature dimension), which has raised new requirements for the complexity of data storage, processing and analysis.

[0003] Feature selection is an important technique in machine learning and data science, which is used to select the most relevant features from a dataset to improve the performance and interpretability of models. This technique helps to simplify the model, reduce overfitting, speed up training, and improve the generalization ability of the model. Currently, feature selection mainly has three forms: selecting features through the correlation between features and variables (filtering method), using existing models as evaluation criteria to select feature subsets (wrapper method), and selecting features while training and evaluating the model (embedding method).

[0004] Feature selection not only improves the performance of the model, but also provides data scientists with a deep understanding of the data, helping them to more effectively solve practical problems. In practical applications, the selection of a suitable feature selection method depends on the nature of the specific task, the characteristics of the data, and the machine learning algorithm used.

[0005] In recent years, due to the popularity of deep learning, many embedded feature selection algorithms utilizing deep learning have emerged, often outperforming traditional algorithms. Therefore, this paper mainly introduces existing feature selection algorithms based on deep learning. For example, feature selection based on the CAE (Concrete Autoencoders) model. The main innovation of this model lies in proposing a concrete selector layer as the encoder of the model. During the training phase, the temperature of the concrete selector layer gradually decreases, and its parameters gradually converge to a one-hot encoded form under the influence of the concrete variables, thereby completing the feature selection task. Another example is the Stochastic Gates method, which uses a relaxed Bernoulli distribution to approximate the L0 regularization of features. During training, the model can learn the parameters of this approximate Bernoulli distribution through gradient descent, so that the model can automatically select a subset of features after training. For example, the FAE (Fractal Autoencoders) feature selection model has been proposed. This model uses a master-sub-network structure with shared parameters. Each parameter in the feature selection layer corresponds to an input feature. The parameters are sorted according to their weights, and the k largest parameters (features) are selected as the inputs of the sub-network. During training, feature information is obtained from both global and local perspectives to optimize the model. After training, the features corresponding to the parameters selected by the sub-network are the feature subsets selected by the model. This method has validated the performance of its model on 15 datasets, and its average performance is the best. These datasets include different types of data such as images and cell transcriptome expression values.

[0006] First, CAE models suffer from feature redundancy, where neurons repeatedly select the same feature. Furthermore, because neurons ultimately only use the feature with the highest probability, ignoring second and third probabilities, and have excessively high threshold requirements, these methods often yield poor results when performing feature selection on high-dimensional data. Random gate models also exhibit the potential for feature redundancy and cannot control the number of selected features. While FAE models allow setting the number of selected features, the number becomes unstable due to the choice of objective function. Additionally, these autoencoder-based unsupervised algorithms fail to effectively utilize the label information contained in the dataset, making it impossible to select features more helpful for specific downstream tasks.

[0007] Therefore, existing technologies still need to be improved and developed.

[0008] Summary of the Invention

[0009] The main objective of this invention is to provide a feature selection method, system, terminal, and computer-readable storage medium based on a robust autoencoder. This invention aims to solve the problems in existing feature selection methods that cannot efficiently and stably filter out key features, and that do not effectively utilize the label information contained in the dataset, thus failing to select features that are more helpful to specific downstream tasks.

[0010] To achieve the above objectives, the present invention provides a feature selection method based on a robust autoencoder, the feature selection method based on a robust autoencoder comprising the following steps:

[0011] Obtain the parameter weights of the feature selection layer in the feature selection model, and multiply the parameter weights by the features after exponentialization to obtain the target parameters after computation.

[0012] During the training process of the feature selection model, the local feature selection window is gradually reduced until the window is reduced to the number of features that need to be selected.

[0013] A classification module is added after the autoencoder in the feature selection model, and a classification loss optimization model is provided using a fully connected layer. The feature selection model is used to select key features that are more conducive to classification.

[0014] Optionally, in the feature selection method based on a strong robust autoencoder, the step of multiplying the exponentially quantified parameter weights by the features to obtain the calculated target parameters specifically involves:

[0015] W = exp(w);

[0016] Where w is the parameter weight of the feature selection layer, exp is the exponential operation, and W is the target parameter after the operation, which is used to select features.

[0017] Optionally, in the feature selection method based on a strong robust autoencoder, the step of gradually narrowing the local feature selection window during the training of the feature selection model until the window is reduced to the number of features to be selected specifically involves:

[0018] Where m is the total number of features, e is the current number of training epochs, s is the set epochs threshold, k is the set number of features to select, and t is the current window size.

[0019] Optionally, in the feature selection method based on a strong robust autoencoder, the loss function of the classification loss optimization model is:

[0020] L op (W, f, g) = -Σ i y ilog(h(f(g(X☉W))) i );

[0021] Where f is the function represented by the decoder, g is the function represented by the encoder, h is the function represented by the classification module, i represents the i-th data in the training set, X represents the feature of the data, and y i Let represent the label of the i-th data, and ⊙ represent the dot product.

[0022] Optionally, in the feature selection method based on a strong robust autoencoder, the loss function is used to backpropagate gradients based on the difference between the predicted and true values ​​to optimize model parameters.

[0023] Optionally, in the feature selection method based on a strong robust autoencoder, the encoder is used to compress the data feature dimension and project the data into a latent space; the decoder is used to restore the variables from the latent space to the original data.

[0024] Optionally, in the feature selection method based on a strong robust autoencoder, the feature selection model is used to perform feature selection for a specific downstream task using the label information present in the dataset.

[0025] Furthermore, to achieve the above objectives, the present invention also provides a feature selection system based on a robust autoencoder, wherein the feature selection system based on a robust autoencoder includes:

[0026] The feature selection weight transformation module is used to obtain the parameter weights of the feature selection layer in the feature selection model, and multiply the parameter weights by the features after performing an exponential operation to obtain the target parameters after computation.

[0027] The feature selection window dynamic adjustment module is used to gradually shrink the local feature selection window during the training process of the feature selection model until the window is shrunk to the number of features that need to be selected.

[0028] The feature selection model optimization module adds a classification module after the autoencoder in the feature selection model. It uses a fully connected layer to provide a classification loss optimization model. The feature selection model is used to select key features that are more conducive to classification.

[0029] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a feature selection program based on a strong robust autoencoder stored in the memory and executable on the processor, wherein when the feature selection program based on the strong robust autoencoder is executed by the processor, it implements the steps of the feature selection method based on the strong robust autoencoder as described above.

[0030] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a feature selection program based on a robust autoencoder, and when the feature selection program based on a robust autoencoder is executed by a processor, it implements the steps of the feature selection method based on a robust autoencoder as described above.

[0031] In this invention, the parameter weights of the feature selection layer in the feature selection model are obtained. These weights are then exponentially multiplied by the features to obtain the calculated target parameters. During the training of the feature selection model, the local feature selection window is gradually reduced until the window is shrunk to the number of features to be selected. A classification module is added after the autoencoder in the feature selection model, using a fully connected layer to provide a classification loss optimization model. The feature selection model is used to select key features that are more conducive to classification. Based on feature selection technology and combined with mathematical modeling, statistical inference, and deep learning methods, this invention innovatively develops a novel feature selection method. It aims to efficiently and stably select key features from different types of data, filter redundant and noisy data, and simultaneously select key features based on downstream task objectives. Attached Figure Description

[0032] Figure 1 is a schematic diagram of the overall model framework in a preferred embodiment of the feature selection method based on a strong robust autoencoder of the present invention;

[0033] Figure 2 is a flowchart of a preferred embodiment of the feature selection method based on a strong robust autoencoder of the present invention;

[0034] Figure 3 is a schematic diagram of dynamic window adjustment in a preferred embodiment of the feature selection method based on a strong robust autoencoder of the present invention;

[0035] Figure 4 is a schematic diagram of a comparative experiment of the exponential weighting strategy in a preferred embodiment of the feature selection method based on a strong robust autoencoder of the present invention.

[0036] Figure 5 is a schematic diagram of a comparative experiment of dynamic window strategies in a preferred embodiment of the feature selection method based on a strong robust autoencoder of the present invention.

[0037] Figure 6 is a structural diagram of a preferred embodiment of the feature selection system based on a strong robust autoencoder of the present invention.

[0038] Figure 7 is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0040] This invention, based on feature selection technology and combined with methods such as mathematical modeling, statistical inference, and deep learning, studies the feature selection task from a computational perspective:

[0041] (1) Research on feature selection methods with strong robustness. There are many feature selection methods based on deep learning, but most of them have the problem of redundant selection of features (such as CAE, random gate, etc.) and unstable training process (such as FAE, etc.). This invention optimizes the training process of feature selection model from the perspective of computation, and obtains a stable and controllable feature selection method.

[0042] (2) Research on feature selection methods for downstream task requirements. Currently, there are many feature selection methods based on unsupervised learning (such as CAE, FAE, random gate, etc.). These methods select key features by only considering the relationship between data and have achieved good results. However, in real life, feature selection is often accompanied by different downstream tasks such as classification. At the same time, the data is often labeled, and unsupervised learning methods cannot make good use of this label information to help the model select features that are more beneficial to downstream tasks.

[0043] This invention develops a feature selection method based on a strong robust autoencoder. This method can solve the problems of feature redundancy, unstable number of selected features, and the inability of unsupervised methods to utilize label information to optimize feature selection for downstream tasks.

[0044] This invention, based on the FAE model, improves the robustness of the feature selection algorithm from three directions: feature selection weight transformation, adding a dynamic adjustment strategy for the feature selection window, and adding an optional custom layer to utilize label information. This results in the RFAE (Robust Fractal Autoencoder), which is used to perform feature selection. The main aspects include the following three:

[0045] (1) Feature selection weight exponential transformation: Obtain the parameter weights of the feature selection layer in the feature selection model, perform an exponential operation on the parameter weights and multiply them with the features to obtain the target parameters after the operation. The exponential weight strategy can effectively enhance the stability of the model in selecting important features.

[0046] (2) Dynamic adjustment of feature selection window: During the training process of the feature selection model, the local feature selection window is gradually reduced until the window is reduced to the number of selected features, making it easier for the model to find key features and increasing the robustness of the model.

[0047] (3) Add a classification module: A classification module is added after the autoencoder in the feature selection model. A fully connected layer is used to provide a classification loss optimization model so that the feature selection model can select features that are more conducive to classification and use the label information that may exist in the dataset to select features for specific downstream tasks.

[0048] As shown in Figure 1, the feature selection process is as follows: the data first enters an autoencoder with shared parameters ① and ② master-sub dual-path feature selection layers to reconstruct the data, and uses global and local information to optimize the feature selection layer weights. Depending on whether label information is needed, it is decided whether to go through the ③ classification layer to optimize the feature selection weights, and finally obtain the selected features.

[0049] Specifically, the feature selection method based on a robust autoencoder according to a preferred embodiment of the present invention, as shown in Figure 2, includes the following steps:

[0050] Step S10: Obtain the parameter weights of the feature selection layer in the feature selection model, and multiply the parameter weights by the features after exponentialization to obtain the target parameters after computation.

[0051] Specifically, when observing the experimental results of FAE, it was found that FAE performed worse than other methods on some datasets. Therefore, this invention carefully investigated and found that the number of weights greater than 0 in the feature selection layer of the model trained on these datasets was less than k, indicating excessive sparsity. This led to a decline in the performance of the selected features on downstream tasks. This phenomenon occurred because the model used L1 regularization to make the features sparse, but the degree of sparsity could not be controlled. Experimental comparisons showed that L1 regularization does indeed help improve feature selection performance. Therefore, this invention decided to perform an exponential operation on the weights before multiplying them with the features, while retaining the L1 regularization loss. This satisfies the constraint of non-negativity of the weights and ensures that the coefficients multiplied with the features will not be 0 even after L1 regularization optimization. The mathematical form is:

[0052] W = exp(w);

[0053] Where w is the parameter weight of the feature selection layer, exp is the exponential operation, and W is the target parameter after the operation. The target parameter is used to select features. The sub-network selects multiple features according to the parameter size (the number of parameters and the number of features are the same, maintaining a one-to-one correspondence).

[0054] Step S20: During the training process of the feature selection model, gradually reduce the local feature selection window until the window is reduced to the number of features that need to be selected.

[0055] Specifically, FAE employs a dual-network structure, utilizing both global features and local feature subset reconstruction losses to optimize feature selection weights. However, since the autoencoder weight training is a black box, it cannot guarantee that the reconstruction loss will correctly adjust the feature selection weights as needed. Furthermore, the local feature subsets used by the sub-networks only constitute a small portion of the global features, meaning that only a small portion of the weights in each training batch can be trained based on the sub-network's reconstruction loss. This is clearly detrimental to finding the optimal subset. To address this, this invention proposes a dynamically adjusted local feature selection window, as shown in Figure 3. This allows the model to have a large receptive field for local feature subsets in the early stages of training, gradually shrinking the local feature selection window as training progresses until the window shrinks to the desired number of features k (the minimum window size is the number of features k selected; this means that the receptive field of the sub-network in this invention gradually shrinks from all features M to k, rather than starting with k in FAE). This better avoids the model getting trapped in local optima and increases the model's robustness. Specifically:

[0056] Where m is the total number of features, e is the current number of training epochs, s is the set epochs threshold, k is the set number of features to select, and t is the current window size.

[0057] Figure 3 illustrates the dynamic window adjustment strategy of this invention. In the early stages of training, the model possesses a large receptive field for a subset of local features. As training progresses, the local feature selection window gradually shrinks until it reaches the set number of selectable features, k.

[0058] Step S30: A classification module is added after the autoencoder in the feature selection model. A fully connected layer is used to provide a classification loss optimization model. The feature selection model is used to select key features that are more conducive to classification.

[0059] Specifically, this invention found that in the comparative experiments, the dataset selected by FAE (Feature Expression Engineering) was only unlabeled for GEO (GENE EXPRESSION OMNIBUS high-throughput gene expression data), while the other 14 datasets were labeled. Since FAE uses unsupervised training, it cannot utilize label information when selecting features, which is undoubtedly a drawback for selecting features for subsequent classification tasks. Therefore, this invention aims to add a classification module after the decoders of the autoencoder (part of the FAE model; the encoder compresses the feature dimension of the data, projecting the data into the latent space, and the decoder restores the variables from the latent space to the original data). Here, this invention uses a fully connected layer (the classification module is a concept, which is a model that performs a classification task based on the input data; here, this invention uses the simplest fully connected layer (linear layer) to demonstrate the effect), providing a classification loss to optimize the model, allowing it to select features more conducive to classification. The corresponding loss function is:

[0060] L op (W, f, g) = -Σ i y i log(h(f(g(X☉W))) i );

[0061] Where f is the function represented by the decoder, g is the function represented by the encoder, h is the function represented by the classification module, i represents the i-th data in the training set, X represents the feature of the data, yi represents the label of the i-th data, and ⊙ represents the dot product.

[0062] The loss function is used to backpropagate gradients based on the difference between predicted and true values, optimizing model parameters. Here, the loss function is a classification loss, which is the difference between the model's predicted classification and the true classification. This invention requires adding a classification loss to train and optimize the model, allowing it to select features more conducive to performing classification tasks.

[0063] This invention addresses the problem of unstable feature selection numbers in existing technologies by employing a weight exponentialization strategy; it also optimizes the training process using a dynamic window strategy, making it easier for the model to find key features; and finally, it adds an optional classification layer, enabling the model to utilize the label information that may exist in the dataset to select features for specific downstream tasks.

[0064] Furthermore, this invention conducted comparative experiments on indexed weighting strategies, dynamic window strategies, optional classification modules, reconstruction capability evaluation, and stability evaluation using (1) the Mice protein expression dataset, (2) the COIL-20 grayscale image database, (3) the Activity human action dataset, (4) the ISOLET speech recognition dataset, (5) the MNIST handwritten digit recognition dataset, (6) the MNIST-Fashion fashion product image dataset, (7) the USPS postal handwritten digit dataset, (8) the GLIOMA brain tumor gene expression classification dataset, (9) the Leukemia gene expression classification dataset, (10) the Pixraw10P facial image dataset, (11) the Prostate_GE prostate tumor grading dataset, (12) the warpAR10P facial expression dataset, (13) the SMK_CAN_187 smoker bronchial epithelial gene expression dataset, (14) the Arcene cancer classification dataset, (15) the GEO gene expression dataset, and (16) the synthetic dataset.

[0065] Experimental results:

[0066] (1) Comparative Experiment of Exponential Weighting Strategy: In the dataset 16 synthetic, since important features are predefined, the model's ability to select features can be evaluated based on the number of important features finally selected. As shown in Figure 4, it is clear that after defining important features using a uniform distribution with smaller parameters, the number of important features selected by the FAE without the exponential weighting strategy began to fluctuate after about 100 training epochs. As the uniform distribution parameter increases, the fluctuation in the number of important features selected becomes more obvious, until it finally stabilizes when a U[0, 14] distribution is used. In contrast, after adding the exponential weighting strategy, the number of important features selected remains constant at 50 (representing the selection of all important features) after each 100 training epochs, without any fluctuations observed. This proves that the exponential weighting strategy can effectively enhance the stability of the model's selection of important features.

[0067] (2) Comparative Experiment of Dynamic Window Strategy: Due to the high cost of measuring gene expression levels, researchers have begun to explore using a small subset of gene expression information to infer the expression information of the remaining genes. Both CAE and FAE introductions mention that their selected gene features improve performance in inferring the expression values ​​of the remaining genes compared to the L1000 landmark genes. To compare and verify the effectiveness of the dynamic window module, this invention conducted experiments on the GEO dataset using the same settings as FAE. This invention uses a linear regression model to calculate the reconstruction error, represented by the mean squared error (MSE), to evaluate the merits of the genes selected by the FAE model and the L1000 landmark genes under the dynamic window strategy.

[0068] As shown in Figure 5, the model clearly achieves better reconstruction performance in gene selection after incorporating the dynamic window module. When selecting 700 genes, it achieves similar reconstruction performance to FAE without the dynamic window module, which selects 800 genes and 943 landmark genes from L1000. Therefore, compared to FAE without the dynamic window module, it reduces the number of genes by approximately 12%. This demonstrates that the dynamic window strategy can select more important features.

[0069] (3) This invention uses extremely randomized trees as classifiers and uses the features selected by the model to evaluate the classification accuracy in order to assess the impact of adding an optional classification layer on the RFAE model. As shown in Table 1, the addition of this layer significantly improves the classification accuracy of RFAE on most datasets. This indicates that adding an optional classification layer enables the model to effectively utilize the additional information provided by the labels and select features that are more beneficial to downstream classification tasks.

[0070] Table 1: Comparison Experiment of Optional Classification Layers

[0071] (4) Reconstruction Capability Evaluation Experiment: Similar to the methods of FAE and CAE, after obtaining the features selected by the model, this invention trains an unregularized linear regression model to reconstruct the input data. The mean squared error between the reconstructed data and the original data is considered as the reconstruction error. Table 2 shows the average reconstruction error and standard deviation of different methods on 14 datasets. As can be seen from Table 2, RFAE has a lower reconstruction loss than other methods, including FAE, on most datasets. This indicates that the features selected by RFAE can better represent the original data.

[0072] Table 2: Reconstruction error performance of each method on 14 datasets

[0073] (5) Stability Evaluation Experiment: This invention records in detail the average overlap of selected features between any two experiments after FAE and RFAE were tested 5 times on 15 datasets. The results are shown in Table 3. In most cases, RFAE with added weighted exponentiation or dynamic window strategies had a higher average feature overlap than FAE alone. The combination of these strategies significantly improved performance, thus highlighting the robustness of the algorithm in this invention.

[0074] Table 3: Overlap in feature selection between RFAE and FAE across 15 datasets

[0075] Furthermore, as shown in Figure 6, based on the above-described feature selection method based on a robust autoencoder, the present invention also provides a feature selection system based on a robust autoencoder, wherein the feature selection system based on a robust autoencoder includes:

[0076] The feature selection weight transformation module 51 is used to obtain the parameter weights of the feature selection layer in the feature selection model, and multiply the parameter weights by the features after performing an exponential operation to obtain the target parameters after the operation.

[0077] The feature selection window dynamic adjustment module 52 is used to gradually shrink the local feature selection window during the training process of the feature selection model until the window is shrunk to the number of features that need to be selected.

[0078] The feature selection model optimization module 53 is used to add a classification module after the autoencoder in the feature selection model. It uses a fully connected layer to provide a classification loss optimization model. The feature selection model is used to select key features that are more conducive to classification.

[0079] Furthermore, as shown in Figure 7, based on the aforementioned feature selection method and system based on a robust autoencoder, the present invention also provides a terminal, which includes a processor 10, a memory 20, and a display 30. Figure 7 only shows some components of the terminal; however, it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0080] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Further, the memory 20 may include both internal and external storage units. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a feature selection program 40 based on a robust autoencoder, which can be executed by the processor 10 to implement the feature selection method based on a robust autoencoder in this application.

[0081] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the feature selection method based on a robust autoencoder.

[0082] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0083] In one embodiment, when the processor 10 executes the feature selection program 40 based on a robust autoencoder in the memory 20, the following steps are performed:

[0084] Obtain the parameter weights of the feature selection layer in the feature selection model, and multiply the parameter weights by the features after exponentialization to obtain the target parameters after computation.

[0085] During the training process of the feature selection model, the local feature selection window is gradually reduced until the window is reduced to the number of features that need to be selected.

[0086] A classification module is added after the autoencoder in the feature selection model, and a classification loss optimization model is provided using a fully connected layer. The feature selection model is used to select key features that are more conducive to classification.

[0087] Specifically, the step of multiplying the exponentialized parameter weights by the features to obtain the calculated target parameters is as follows:

[0088] W = exp(w);

[0089] Where w is the parameter weight of the feature selection layer, exp is the exponential operation, and W is the target parameter after the operation, which is used to select features.

[0090] Specifically, the step of gradually narrowing the local feature selection window during the training process of the feature selection model until the window is reduced to the number of features that need to be selected is as follows:

[0091] Where m is the total number of features, e is the current number of training epochs, s is the set epochs threshold, k is the set number of features to select, and t is the current window size.

[0092] Optionally, in the feature selection method based on a strong robust autoencoder, the loss function of the classification loss optimization model is:

[0093] L op (W, f, g) = -Σ i y i log(h(f(g(X☉W))) i );

[0094] Where f is the function represented by the decoder, g is the function represented by the encoder, h is the function represented by the classification module, i represents the i-th data in the training set, X represents the feature of the data, and y i Let represent the label of the i-th data, and ⊙ represent the dot product.

[0095] The loss function is used to backpropagate gradients based on the difference between the predicted and actual values, thereby optimizing the model parameters.

[0096] The encoder is used to compress the data feature dimension and project the data into the latent space; the decoder is used to restore the variables from the latent space to the original data.

[0097] The feature selection model is used to select features for specific downstream tasks by utilizing the label information present in the dataset.

[0098] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a feature selection program based on a robust autoencoder, and the feature selection program based on a robust autoencoder, when executed by a processor, implements the steps of the feature selection method based on a robust autoencoder as described above.

[0099] In summary, this invention provides a feature selection method, system, terminal, and storage medium based on a robust autoencoder. The method includes: obtaining the parameter weights of the feature selection layer in the feature selection model; multiplying the parameter weights by the features after exponentialization to obtain the calculated target parameters; gradually narrowing the local feature selection window during the training process of the feature selection model until the window is reduced to the number of features to be selected; adding a classification module after the autoencoder in the feature selection model, using a fully connected layer to provide a classification loss optimization model; and using the feature selection model to select key features that are more conducive to classification. This invention, based on feature selection technology and combined with mathematical modeling, statistical inference, and deep learning methods, innovatively develops a novel feature selection method that aims to efficiently and stably filter key features from different types of data, filtering redundant and noisy data, and simultaneously selecting key features based on downstream task objectives.

[0100] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0101] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0102] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A feature selection method based on a robust autoencoder, characterized in that, The feature selection method based on a strong robust autoencoder includes: Obtain the parameter weights of the feature selection layer in the feature selection model, and multiply the parameter weights by the features after exponentialization to obtain the target parameters after computation. During the training process of the feature selection model, the local feature selection window is gradually reduced until the window is reduced to the number of features that need to be selected. A classification module is added after the autoencoder in the feature selection model, and a classification loss optimization model is provided using a fully connected layer. The feature selection model is used to select key features that are more conducive to classification.

2. The feature selection method based on a robust autoencoder according to claim 1, characterized in that, The process of exponentializing the parameter weights and multiplying them by the features to obtain the calculated target parameters is as follows: W = exp(w); Where w is the parameter weight of the feature selection layer, exp is the exponential operation, and W is the target parameter after the operation, which is used to select features.

3. The feature selection method based on a robust autoencoder according to claim 1, characterized in that, The process of gradually narrowing the local feature selection window during the training of the feature selection model, until the window is reduced to the number of features that need to be selected, specifically involves: Where m is the total number of features, e is the current number of training epochs, s is the set epochs threshold, k is the set number of features to select, and t is the current window size.

4. The feature selection method based on a robust autoencoder according to claim 2, characterized in that, The loss function of the classification loss optimization model is: L op (W,f,g)=-Σ i y i log(h(f(g(X☉W))) i ); Where f is the function represented by the decoder, g is the function represented by the encoder, h is the function represented by the classification module, i represents the i-th data in the training set, X represents the feature of the data, and y i Let represent the label of the i-th data, and ⊙ represent the dot product.

5. The feature selection method based on a robust autoencoder according to claim 4, characterized in that, The loss function is used to backpropagate gradients based on the difference between the predicted and actual values, thereby optimizing the model parameters.

6. The feature selection method based on a robust autoencoder according to claim 4, characterized in that, The encoder is used to compress the data feature dimension and project the data into the latent space; the decoder is used to restore the variables from the latent space back to the original data.

7. The feature selection method based on a robust autoencoder according to claim 1, characterized in that, The feature selection model is used to select features for specific downstream tasks by utilizing the label information present in the dataset.

8. A feature selection system based on a robust autoencoder, characterized in that, The feature selection system based on a robust autoencoder includes: The feature selection weight transformation module is used to obtain the parameter weights of the feature selection layer in the feature selection model, and multiply the parameter weights by the features after performing an exponential operation to obtain the target parameters after computation. The feature selection window dynamic adjustment module is used to gradually shrink the local feature selection window during the training process of the feature selection model until the window is shrunk to the number of features that need to be selected. The feature selection model optimization module adds a classification module after the autoencoder in the feature selection model. It uses a fully connected layer to provide a classification loss optimization model. The feature selection model is used to select key features that are more conducive to classification.

9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a feature selection program based on a robust autoencoder stored in the memory and executable on the processor. When the feature selection program based on the robust autoencoder is executed by the processor, it implements the steps of the feature selection method based on a robust autoencoder as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a feature selection program based on a robust autoencoder, which, when executed by a processor, implements the steps of the feature selection method based on a robust autoencoder as described in any one of claims 1-7.

Images