A deep learning feature screening method and system

Abstract

The application relates to the field of artificial intelligence, and discloses a deep learning feature screening method and system, aiming to solve the problems of feature screening staticization, single evaluation standard, screening and training separation and difficulty in capturing nonlinear dependence in the prior art. The method comprises the following steps: performing standardization processing on input features, generating dynamic importance scores through a differentiable feature scoring module, combining a screening threshold value based on loss gradient volatility self-adaptive adjustment, generating a binary selection mask to realize sparse selection; and using a trainable reconstruction module to map the screened features back to the original dimension, and inputting the features into a target model for forward propagation. The application realizes deep fusion of feature screening and model training through end-to-end joint optimization, dynamically adjusts the screening intensity, improves the model convergence speed and prediction accuracy, and enhances the explainability and deployment universality.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence, specifically relating to a deep learning feature selection method and system. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep learning has been widely used in fields such as image recognition, natural language processing, and speech analysis. Its core lies in automatically extracting abstract features from data through multi-layer neural networks to support the modeling and decision-making of complex tasks.

[0003] In the process of building deep learning models, the input data usually contains a large number of high-dimensional features, including key features that significantly contribute to the model's performance, as well as a large number of redundant, irrelevant, or even noisy features. These features not only increase the computational burden of the model, but may also lead to problems such as overfitting and decreased generalization ability, which seriously affect the training efficiency and prediction accuracy of the model.

[0004] Among them, deep learning feature selection, as an important preliminary step to improve model performance, aims to identify and retain the most discriminative feature subset from the original feature space, thereby achieving the dual goals of model lightweighting and performance optimization.

[0005] This process typically relies on correlation analysis between features and target variables, assessment of feature redundancy, and judgment of the degree of influence of features on model output. Its effectiveness is directly related to the convergence speed, interpretability, and actual deployment efficiency of the subsequent model.

[0006] Existing technologies for feature selection in deep learning still face several key challenges: First, most methods employ static selection strategies, fixing the feature subset before model training, which fails to adapt to the dynamic changes in feature importance during training. Second, feature evaluation criteria are singular, often relying solely on local metrics such as gradient magnitude or attention weights, lacking a comprehensive consideration of the propagation path and interaction effects of features within the global network structure. Third, the selection process is separated from model training, failing to achieve end-to-end joint learning of feature selection and parameter optimization, resulting in a disconnect between the selection results and the final task objective. Finally, existing methods have limited ability to model high-dimensional sparse data and nonlinear feature relationships, making it difficult to effectively capture the complex feature dependency structures in deep networks. These problems are particularly prominent in large-scale, highly complex application scenarios, severely restricting the efficient deployment and practical application of deep learning models in resource-constrained environments. Therefore, there is an urgent need to propose a dynamic, adaptive feature selection mechanism that is deeply integrated with model training to address these technical bottlenecks. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of existing technologies by providing a deep learning feature selection method and system that effectively solves the problems in the background technology. Current deep learning models generally suffer from feature redundancy, noise interference, and low efficiency in extracting key features when processing high-dimensional input data, resulting in slow model training convergence, weak generalization ability, and high inference resource consumption. Traditional feature selection methods often rely on manual priors or static selection mechanisms based on statistical indicators, which are difficult to adapt to the selection requirements of nonlinear, dynamically evolving feature spaces in deep neural networks and cannot achieve co-optimization with the model training process. This invention constructs an end-to-end trainable deep learning feature selection framework to achieve dynamic evaluation and adaptive selection of input features, retaining the most discriminative features while suppressing irrelevant or redundant information, significantly improving the model's learning efficiency and prediction performance.

[0008] To achieve the above objectives, the present invention provides the following technical solution: On one hand, a deep learning feature selection system for processing image data, text data, and sensor data. This system comprises the following components: a feature input module for receiving raw high-dimensional feature data and performing standardized preprocessing, outputting an input feature tensor with uniform dimensions; a differentiable feature scoring module for assigning differentiable weights to each feature dimension in the input feature tensor, generating a continuous scoring vector corresponding to the feature importance; and an adaptive threshold generation module for dynamically calculating the feature selection threshold based on the loss change rate and gradient magnitude during the current training phase, outputting an adaptively adjusted threshold as the training progresses. The system includes: a threshold scalar; a sparsity selection module, used to perform differentiable hard or soft selection operations based on the scoring vector and the threshold scalar to generate a sparse feature subset representation; a feature reconstruction module, used to map the selected feature subset back to the original feature space dimension to generate a dense feature representation compatible with downstream models; a model co-training module, used to input the reconstructed feature representation into the target deep learning model for forward propagation, and synchronously update the feature scoring parameters and model parameters through a joint backpropagation algorithm; and a monitoring and feedback module, used to record the feature scoring distribution, selection ratio, and model performance indicators in real time, providing visualization of the training process and anomaly detection capabilities.

[0009] Preferably, the differentiable feature scoring module is implemented using a learnable feature weight vector. The feature weight vector is initialized as a normally distributed random value with a mean of 0.5 and a standard deviation of 0.1. Its dimension is consistent with the dimension of the input features. Each weight value is mapped to the (0,1) interval through the Sigmoid activation function to form an initial scoring vector. This scoring vector is continuously optimized during training using the gradient descent method, so that the weights of features strongly correlated with the model loss function approach 1, and the weights of irrelevant features approach 0.

[0010] Furthermore, the adaptive threshold generation module calculates the threshold based on the dynamic loss gradient within the sliding window. Specifically, it collects the L2 norm of the loss gradient of the current batch every 50 training steps and calculates the standard deviation σ of the most recent 10 sampling points. When σ is greater than the upper limit of the preset dynamic threshold interval [0.02, 0.05], the model is determined to be in a period of drastic adjustment. At this time, the feature selection threshold is lowered to 0.3 to retain more potentially useful features. When σ is lower than the lower limit of the interval, the model is determined to be stable. The threshold is then raised to 0.7 to enhance the selection intensity. The threshold adjustment process uses exponential moving average smoothing with a decay coefficient of 0.9 to avoid threshold oscillation.

[0011] Furthermore, the sparsity selection module supports two operating modes: in soft selection mode, the Gumbel-Softmax function with temperature parameter τ=0.1 is used to reparameterize and sample the score vector, generating an approximately discrete but differentiable selection mask to achieve end-to-end training; in hard selection mode, a combination of a step function and a straight-through estimator is used to perform binarization truncation in forward propagation and retain the original gradient in backpropagation to ensure training stability; the two modes can be configured to use soft selection in the early stages of training and switch to hard selection later to approximate the true sparsity effect.

[0012] Preferably, the feature reconstruction module adopts a gated reconstruction mechanism, introducing a learnable reconstruction weight matrix W_r∈R^{d×k}, where d is the original feature dimension and k is the number of features retained after filtering. This matrix expands the k-dimensional sparse features into a d-dimensional output through matrix multiplication and adds it to a learnable bias vector b_r. The final output is normalized by the LayerNormalization layer to ensure the numerical stability and distribution consistency of the reconstructed features.

[0013] Furthermore, the model co-training module adopts a phased parameter update strategy during each backpropagation: first, the target model parameters are frozen, and the parameters of the feature scoring module and the reconstruction module are updated separately for one optimization step to quickly establish the prior of feature importance; then, all parameters are unfrozen and joint optimization is performed. This strategy is executed cyclically, and the freezing ratio is adjusted every 3 training cycles, gradually decreasing from 100% to 0%, to achieve a smooth transition from feature selection-driven to model learning-driven.

[0014] In addition, the monitoring feedback module periodically outputs feature score heatmaps, cumulative screening frequency statistics tables, and validation set accuracy curves. The sampling interval is set to once every 200 training steps. All monitoring data is stored in structured log format, which supports real-time reading and visualization by external analysis tools, and is used to diagnose feature selection bias or model overfitting.

[0015] On the other hand, a deep learning feature selection method is used to process image data, text data, and sensor data. The specific steps of this method are as follows: Step S110: Receive the original input feature matrix and perform Z-score standardization on each feature dimension column-wise to obtain a normalized feature matrix; Step S120: Generate a feature score vector using a learnable weight vector and a Sigmoid function; Step S130: Calculate an adaptive selection threshold based on the current training step number t and the historical loss gradient sequence, and then dynamically adjust it according to gradient volatility, with the adjustment step size limited to ±0.05; Step S140: [The remaining text appears to be incomplete and requires further context.] The vector is compared with a threshold to generate a binary selection mask, resulting in a filtered feature index set. In step S150, the feature columns corresponding to the indexes are extracted from the normalized feature matrix to form a dimensionality-reduced feature subset. In step S160, the sparse subset is converted into a d-dimensional dense representation through a trainable reconstruction mapping layer, ensuring that its dimension is consistent with the original input. In step S170, the dense features are input into the target deep learning model for forward computation to obtain the predicted output and calculate the loss function value. In step S180, backpropagation is performed to synchronously update the feature score vector w, reconstruction mapping parameters, and target model weights, completing one round of joint training iteration.

[0016] Furthermore, in step S140, the generation process of the binary mask m adopts the pass-through estimator technique during the training phase. The forward propagation uses a step function to achieve hard selection, and the gradient is directly passed to the scoring vector s during the back propagation, avoiding gradient interruption due to non-differentiable operations and improving the reliability of training convergence.

[0017] In addition, the reconstruction mapping layer in step S160 includes a fully connected transformation and a residual connection structure. The weight matrix of the fully connected layer has a dimension of k×d and adopts the Xavier initialization method. The residual connection adds the mean vector of the original X_norm to the reconstruction output to compensate for the distribution shift caused by feature loss and enhance the fidelity of feature reconstruction.

[0018] Preferably, the parameter update in step S180 adopts a phased optimization strategy. In each complete training cycle, the parameters of the feature scoring module and the reconstruction module are updated only in the first 3 iterations, and the parameters of the target model are kept frozen. In subsequent iterations, the frozen parameters are unfrozen and global joint optimization is performed. This strategy continues for 5 training cycles before the phased control stops and conventional end-to-end training is switched to.

[0019] Furthermore, the method continuously records the cumulative frequency of each feature dimension being selected during the training process, and generates a feature importance ranking list based on the frequency after training. This list can be used to explain the model's decision-making basis and serve as a reference for input feature selection in subsequent lightweight model design.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] By introducing a differentiable feature scoring mechanism, end-to-end joint optimization of feature importance assessment and deep learning model training is achieved, overcoming the suboptimal problem caused by the separation of feature selection and model learning in traditional methods. This enables the selection process to dynamically respond to the model learning state, improving the accuracy and adaptability of feature selection.

[0022] An adaptive threshold generation strategy based on the dynamics of loss gradient is adopted to automatically adjust the feature selection intensity as the training process progresses. More features are retained in the early stage of training to ensure exploration ability, and sparsity is enhanced in the later stage to improve model compactness, effectively balancing learning stability and model efficiency.

[0023] A closed-loop structure incorporating sparse selection and gated reconstruction was designed, which not only achieves effective dimensionality reduction of input features but also ensures the compatibility of output feature dimensions with downstream models, avoiding architecture reconstruction problems caused by dimensionality mismatch and enhancing the system's general deployment capability.

[0024] A phased parameter update mechanism is proposed, which prioritizes the optimization of the feature selection component in the early stage of training to establish a reliable feature prior, and then gradually transitions to overall collaborative optimization. This significantly improves the convergence speed and stability of the training process. Experiments show that the model accuracy is improved by an average of 6.8% under the same number of training rounds.

[0025] The integrated monitoring and feedback module provides real-time output of feature scoring evolution trajectory and screening statistics, enhancing the interpretability and debugging convenience of deep learning models. It also helps engineers identify redundant features or abnormal screening behaviors, reducing model tuning costs. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall technical architecture of the deep learning feature selection method and system proposed in this invention. Detailed Implementation

[0027] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0028] Example 1

[0029] Please refer to Figure 1This embodiment uses a medical image classification task based on convolutional neural networks (CNN) as the application scenario, specifically the task of distinguishing between benign and malignant lung nodules in lung CT images. The original input data is a three-dimensional voxel image from the publicly available dataset LIDC-IDRI. After preprocessing, it forms grayscale data with a size of 256×256×64. A 1024-dimensional multi-scale radiomics feature vector is extracted from each sample to form the input feature matrix. ,in =800 represents the total number of training samples. This scenario presents a significant feature redundancy problem: some texture features (such as the Gray-Level Co-occurrence Matrix (GLCM) derived index) are highly correlated in different directions, and some low-order statistical features contribute little to the classification task. Without filtering, this will lead to model overfitting and increased inference latency. Therefore, this embodiment deploys the deep learning feature selection system described in this invention to achieve an end-to-end trainable dynamic feature selection mechanism.

[0030] First, in step S110, the feature input module receives the original feature matrix. Then, Z-score standardization is performed. Specifically, the mean of each feature dimension is independently calculated on the training set. with standard deviation Then, for each sample, the first eigenvalues Perform the transformation:

[0031]

[0032] Generate normalized feature matrix This process ensures all features are on the same scale, preventing high-amplitude features from dominating the scoring module's learning process. Standardized data is loaded into the GPU memory buffer via memory mapping, and asynchronous data pipeline technology is used to parallelize the model's forward computation. The batch size is set to 32, and the data transfer latency per batch is controlled to within 8 milliseconds.

[0033] Proceeding to step S120, the differentiable feature scoring module begins operation. This module contains a learnable weight vector. Its initial value is randomly generated from a normal distribution with a mean of 0.5 and a standard deviation of 0.1, i.e. The weight vector is not directly multiplied by the normalized feature matrix; instead, it participates in subsequent score calculations as an independent parameter. Specifically, it will... Input the Sigmoid activation function:

[0034]

[0035] Output continuous rating vector Each element Indicates the first The importance score for each feature. This score is not dependent on the content of the current input sample, but is a globally shared static importance measure, which is continuously updated through backpropagation during training. (Score vector) It is stored in a dedicated parameter tensor and a gradient accumulation flag is set to support the accumulation of gradients over multiple mini-batches to improve optimization stability.

[0036] In step S130, the adaptive threshold generation module dynamically adjusts the screening threshold based on model training. This module maintains a sliding window queue to store the L2 norm of the loss gradients from the most recent 10 samples. The sampling frequency is set to once every 50 training steps. At each sampling, the system extracts the global L2 norm from the gradient tensor of the loss function with respect to the model parameters for the current batch: The scalar value is then written into the sliding window. Once the window contains 10 data points, its standard deviation is calculated. .like A threshold value >0.05 indicates that the model is in a phase of drastic parameter adjustment (e.g., just crossing a local minimum). In this case, the threshold should be adjusted accordingly. According to the formula Lowering the threshold reduces the screening intensity and retains more potentially useful features; if <0.02 indicates that the model tends to converge and stabilize. Increase the threshold to enhance the sparsity effect; otherwise, keep it unchanged. Unchanged. The threshold update process uses an exponential moving average (EMA) for smoothing:

[0037]

[0038] A decay coefficient of 0.9 effectively suppresses threshold oscillations caused by single gradient fluctuations. The initial threshold is set to 0.5, located at the center of the dynamic range, ensuring moderate initial screening intensity.

[0039] Step S140 performs a sparsity selection operation. The system will then perform a sparsity selection operation on the scoring vector. Compared with the current threshold Element-by-element comparison to generate a binary selection mask Defined as:

[0040]

[0041] This mask determines the set of indices for the preserved features. In the forward propagation phase, a step function is used to achieve hard selection, ensuring the true sparsity of the output features. In the backpropagation phase, a straight-through estimator (STE) technique is applied to directly bypass the non-differentiable step function and pass the gradients propagated from downstream to the scoring vector as is. This maintains the integrity of the gradient path. This mechanism avoids sampling noise introduced by approximation methods such as Gumbel-Softmax, while ensuring training stability. Mask selection. It is then converted into a sparse index tensor for subsequent feature extraction.

[0042] In step S150, the system obtains the normalized feature matrix Extract all The feature columns corresponding to =1 constitute the subset after dimensionality reduction. ,in The number of features retained for the current batch. This extraction process is efficiently completed using a sparse index gather operation on the GPU, with a time cost of less than 1.2 milliseconds. Due to different training strides... This may change; the system employs a dynamic memory allocation strategy, pre-allocating a buffer with the maximum capacity (corresponding to...). =1024), and adjust according to actual conditions at runtime. Value truncation avoids latency caused by frequent memory allocation.

[0043] Step S160 performs feature reconstruction. Sparse subset Since the original CNN model cannot be directly input (its fully connected layers expect 1024-dimensional input), a feature reconstruction module is needed to map it back to the original dimensions. This module employs a gated reconstruction mechanism and includes a learnable reconstruction weight matrix. and bias vector First, perform matrix transformations:

[0044]

[0045] Will The dimensional feature is expanded to 1024 dimensions; then a bias term is added: Finally, normalization is performed using the LayerNormalization layer.

[0046]

[0047] in and They are respectively Mean and variance along the feature dimension , For learnable scaling and translation parameters, This is a numerical stability constant. This normalization operation ensures that the distribution of the reconstructed features is consistent with the standardized statistical properties of the original input, preventing numerical shifts caused by dimension mapping from affecting downstream model training. Reconstructed weight matrix Using the Xavier initialization method: elements are initially uniformly distributed. Mid-sampling ensures the stability of the initial activation value variance. Furthermore, the module introduces a residual connection structure to convert the original... Mean vector along the sample dimension Add to the final output: It is used to compensate for the global distribution shift caused by missing features and improve the reconstruction fidelity.

[0048] Step S170 represents the reconstructed dense feature. The target deep learning model is a 3D CNN containing 4 convolutional blocks and 2 fully connected layers. Forward propagation is performed to obtain classification predictions, and the loss value L for the current batch is calculated based on the cross-entropy loss function. The last layer of the model outputs 2-dimensional logits, corresponding to the probabilities of "benign" and "malignant" classes, respectively.

[0049] In step S180, backpropagation is performed to complete the joint parameter update. This embodiment adopts a phased parameter update strategy: in the first 3 iterations of each complete training cycle (epoch), the system freezes all parameters of the target CNN model and only enables the weights of the feature scoring module. , Refactoring module , , , Gradient calculation and optimization for parameters such as Adam. The optimizer uses Adam, and the learning rate is set to... One gradient descent update is performed on these components. This stage aims to quickly establish a reliable prior for feature importance, allowing the score vector to initially converge to a reasonable distribution. From the fourth iteration onwards, the system unfreezes the target model parameters, initiates global joint optimization mode, and all parameters (including CNN weights) participate in backpropagation and updates, with the learning rate adjusted to... This phased strategy was terminated after 5 training epochs, and subsequent training switched to regular end-to-end joint optimization. Gradient pruning was used to prevent gradient explosion, with a threshold set to 1.0.

[0050] Throughout the training process, the monitoring and feedback module continuously collects and records key metrics. Every 200 training steps, the system outputs a structured log containing the following information: (1) Feature scoring vector (1) Histogram distribution data, with the number of features in each score segment counted in 10 intervals; (2) Cumulative screening frequency statistics table, recording the total number of times each feature was selected in all past training steps; (3) Validation set classification accuracy, AUC value and loss curve; (4) Current screening ratio All data is written to a local log file in JSON format and pushed to a remote monitoring server in real time via the gRPC interface, supporting visualization using tools such as TensorBoard. After training, the system sorts the 1024 features in descending order based on their cumulative screening frequency, generating a feature importance ranking list. Experimental results show that after 200 rounds of training, the average screening ratio stabilizes at 38.7%, and the Top-50 features are mainly concentrated in higher-order wavelet transform coefficients and local binary pattern (LBP) texture features, which are highly consistent with medical prior knowledge, verifying the effectiveness of this method. The final model achieves an accuracy of 92.4% on the independent test set, a 6.8% improvement over the baseline model without feature screening, and an inference speed 2.3 times faster.

[0051] Example 2

[0052] This embodiment focuses on text sentiment classification in natural language processing, specifically the application scenario of identifying the sentiment polarity (positive / negative) of user reviews on an e-commerce platform. The input data consists of sentence vector representations encoded using a BERT-based model. Each review generates a 768-dimensional context-aware embedding vector, forming the input matrix. ,in =10000 represents the training sample size. Unlike image tasks, text embedding features have strong semantic coupling; each dimension is not independently interpretable, and there is a large amount of semantic redundancy (similar to vector projections of synonyms and near-synonyms). Traditional gradient-based importance assessment struggles to capture these high-order semantic relationships. Therefore, this embodiment introduces a modified differentiable feature scoring module into the system architecture, employing an attention mechanism to replace a single weight vector, thereby modeling the interaction relationships between features.

[0053] In step S110, the feature input module receives the original BERT output vector and performs Z-score normalization processing, the same as in Embodiment 1, to obtain... Since the sample length is variable in text tasks, the system adopts a dynamic batch processing strategy, dividing the data into buckets according to sequence length and filling each batch to a uniform length to ensure consistent feature matrix dimensions.

[0054] In step S120, the differentiable feature scoring module of this embodiment uses a self-attention mechanism to generate a scoring vector. Specifically, the structure involves inputting the feature matrix... Each is achieved through three learnable linear projection matrices. Generate query matrix Key matrix Value matrix Then, the attention weights are calculated:

[0055]

[0056] Attention output is That is, each sample receives a scalar score. Since the goal is to generate globally uniform feature importance scores (rather than sample-dependent scores), the system... Take the average across the sample dimensions:

[0057]

[0058] A 768-dimensional vector is obtained. This vector is then mapped to the (0,1) interval using the Sigmoid function, forming the final feature score vector. This mechanism allows the model to consider the correlation between features during the scoring process. For example, when multiple semantically similar dimensions are activated simultaneously, their attention weights reinforce each other, thus more accurately reflecting their joint discriminative ability. The projection matrix is ​​initialized using the Kaiming initialization method to ensure the stability of gradient propagation after nonlinear activation.

[0059] In this embodiment, the adaptive threshold generation mechanism in step S130 has undergone parameter adjustments. Since the loss surface in NLP tasks is typically smoother and gradient fluctuations are smaller, the sliding window size is adjusted from 10 to 15 to improve statistical stability; the dynamic threshold range is adjusted from [0.02, 0.05] to [0.01, 0.03] to accommodate more subtle gradient changes; the threshold adjustment step size remains ±0.05, but the EMA decay coefficient is increased to 0.95 to further suppress oscillations. The initial threshold is still set to 0.5.

[0060] In this embodiment, the sparsity selection module in step S140 switches to soft selection mode. The system uses temperature parameters. The Gumbel-Softmax function with a value of 0.1 generates a differentiable selection mask. The specific process is as follows: first, noise g is sampled from the Gumbel distribution. Gumbel(0,1), then calculate:

[0061]

[0062] Output mask It is a continuous value vector, approximating one-hot form, but mathematically completely differentiable. This mask is used for... Perform weighted selection: ,in This represents bitwise multiplication. This operation preserves all features but assigns near-zero weights to low-scoring features, achieving "soft selection." This mode is used continuously throughout the training cycle without switching to hard selection to avoid information breaks in a highly coupled semantic space.

[0063] In step S150, because soft selection does not truly reduce dimensionality, It remains 768-dimensional. To achieve practical compression, the system introduces a subsequent sparse regularization constraint: an L1 regularization term is added to the loss function. , This causes the score vectors to tend towards sparseness. In the later stages of training, over 80% of the... If the value approaches 0, dimensions below 0.01 can be manually truncated to achieve an equivalent hard selection.

[0064] In this embodiment, the feature reconstruction module of step S160 is simplified to a combination of identity mapping and residual compensation. Because The dimensions remain unchanged; the refactoring layer only executes: ,in for The mean vector, =0.3 represents the learnable residual coefficient. This design reduces the number of parameters and avoids introducing unnecessary nonlinear distortions in the high-dimensional semantic space.

[0065] In step S170, the reconstructed features are input into a two-layer fully connected classifier (512-dimensional hidden layers, ReLU activation), and the output is the sentiment category probability. The loss function is the sum of cross-entropy and L1 regularization.

[0066] In this embodiment, the parameter update strategy in step S180 is adjusted to a dynamic freeze ratio. The phased optimization still executes for 5 cycles, but the freeze ratio starts at 100% and decreases by 20% each cycle. That is, 100% of the model parameters are frozen in the first cycle, 80% in the second cycle, and so on, until 20% is frozen in the fifth cycle, and then completely unfrozen from the sixth cycle onwards. This gradual unfreezing strategy is smoother than fixed-step switching. The optimizer uses AdamW, and the weight decay is set to... This is to suppress overfitting.

[0067] In addition to standard metrics, the monitoring and feedback module records an attention weight heatmap, displaying the distribution of high-scoring features in the text token space to aid in analyzing the keywords the model focuses on. After training, the feature importance ranking shows that high-scoring dimensions are concentrated in the vector components corresponding to sentiment polarity words (such as "good" and "bad") and degree adverbs ("very" and "extremely"), consistent with linguistic intuition. The final model achieved 94.1% accuracy on the test set, a 5.2% improvement over the baseline, and the feature dimensions were effectively compressed to 22% of their original size.

[0068] Example 3

[0069] This embodiment addresses equipment fault prediction in the Industrial Internet of Things (IIoT), with input consisting of multi-source sensor fusion features. A wind turbine is equipped with 12 types of sensors, including vibration, temperature, current, and wind speed sensors, collecting data every 5 minutes. Statistical features (mean, variance, peak value, kurtosis, etc.) for each time period are extracted using sliding window feature engineering, forming a 1536-dimensional feature vector that constitutes the input matrix. , =5000. In this scenario, the features exhibit strong time dependence and modal heterogeneity, and the fault signals are weak and easily masked by environmental noise. Therefore, this embodiment introduces a time-aware scoring mechanism and a multi-granularity threshold control strategy into the system.

[0070] In step S110, the feature input module not only performs Z-score normalization but also sets independent normalization parameter pools for different sensor types. For example, the variance of a vibration sensor is much larger than that of a temperature sensor, therefore, its variance is calculated separately. and To avoid cross-modal interference.

[0071] In step S120, the differentiable feature scoring module adopts a time-sliding window weighting mechanism. The system maintains a historical scoring cache of length 10. At each update step, the current score... It consists of two parts: instant rating (Generated from learnable weight vectors via Sigmoid) and historical rating index moving average The fusion formula is:

[0072]

[0073] in =0.7 is the momentum coefficient. This mechanism enables feature scoring to have short-term memory capabilities, assigning higher stability weights to features that remain important and suppressing score fluctuations caused by transient noise.

[0074] In step S130, the adaptive threshold generation module is expanded to multi-granularity control. The system not only calculates the global loss gradient standard deviation... It also calculates the local gradient fluctuations of various sensor subsets (such as vibration and electrical sensors). If a certain type If the threshold is greater than 0.06, the threshold for that type of feature is lowered by 0.1 to achieve modal adaptive filtering. For example, when the unit encounters a sudden strong wind, the gradient of wind speed-related features is drastic, and the system automatically lowers its filtering threshold to retain more environmental context information.

[0075] In step S140, the sparsity selection module combines hard selection with soft masking. The system generates a binary mask. Then, the soft weights are further calculated. This is used to adjust the gradient magnitude during backpropagation, forming a hybrid gradient routing mechanism.

[0076] Step S160 The reconstruction module introduces the modality-aware reconstruction matrix. It is divided into 12 sub-matrices, each corresponding to a type of sensor, which are initialized and updated separately to ensure that the features of different physical dimensions remain independent during the reconstruction process.

[0077] The remaining steps are similar to those in the aforementioned embodiments, but the monitoring module pays special attention to the activation trajectories of low-frequency, high-importance features for early fault warning. Experiments show that this method can predict bearing failures 72 hours in advance, with an F1-score of 0.89, which is a significant improvement over traditional methods.

[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A deep learning feature selection method for processing image data, text data, and sensor data, characterized in that, Includes the following steps: Receive the original input feature matrix, and perform standardization processing on each feature dimension of the original input feature matrix to obtain a normalized feature matrix; The normalized feature matrix is ​​input into the differentiable feature scoring module, and a continuously valued feature scoring vector is generated by the learnable feature weight vector and the Sigmoid activation function, wherein each element of the feature scoring vector corresponds to the importance score of an input feature dimension. An adaptive screening threshold is calculated based on the current number of training steps and the historical loss gradient sequence. The adaptive screening threshold is dynamically adjusted according to the volatility of the loss gradient, so as to reduce the screening intensity during the stage of drastic changes in model parameters and enhance the sparsity during the stage of stabilization. The feature score vector is compared with the adaptive filtering threshold to generate a binary selection mask, and the set of indices of the features to be retained is determined based on the binary selection mask. Extract the feature columns specified by the index set from the normalized feature matrix to form a dimensionality-reduced feature subset; The feature subset is transformed into a dense feature representation with the same dimension as the original feature through a trainable reconstruction mapping layer. The reconstruction mapping layer contains a learnable weight matrix and a bias vector, and introduces a normalization operation to maintain the distribution consistency of the output features. The dense feature representation is input into the target deep learning model for forward propagation, and the model prediction loss is calculated. The backpropagation algorithm is executed to synchronously update the parameters of the differentiable feature scoring module, the parameters of the reconstructed mapping layer, and the parameters of the target deep learning model, thereby achieving joint optimization of feature selection and model training.

2. The deep learning feature selection method according to claim 1, characterized in that, The learnable feature weight vector has the same dimension as the input feature dimension, is initialized as a normally distributed random value with a mean of 0.5 and a standard deviation of 0.1, and is continuously optimized during training using gradient descent, so that the score of the feature dimension that is strongly correlated with the model loss function approaches 1, and the score of irrelevant features approaches 0.

3. The deep learning feature selection method according to claim 1, characterized in that, The dynamic adjustment of the adaptive screening threshold is based on the standard deviation of the L2 norm of the loss gradient within the sliding window. The loss gradient is collected once every preset number of training steps, and the standard deviation of the most recent multiple sampling points is calculated. When the standard deviation is greater than a preset upper limit, the adaptive screening threshold is lowered; when the standard deviation is less than a preset lower limit, the adaptive screening threshold is raised. An exponential moving average is used to smooth the threshold update process.

4. The deep learning feature selection method according to claim 1, characterized in that, The operation of generating a binary selection mask uses a step function to achieve hard selection in forward propagation and applies a pass-through estimator technique in back propagation to directly pass the gradients returned from downstream to the feature score vector, bypassing the non-differentiable step function, so as to maintain the integrity of the gradient path.

5. The deep learning feature selection method according to claim 1, characterized in that, The weight matrix of the reconstructed mapping layer maps the dimensionality-reduced feature subset to the original feature space dimension through matrix multiplication, and adds it to the learnable bias vector. The result is processed by the Layer Normalization layer to output a normalized dense feature representation. The weight matrix is ​​initialized with parameters using the Xavier initialization method.

6. The deep learning feature selection method according to claim 1, characterized in that, The synchronous parameter update operation adopts a phased optimization strategy. At the initial stage of each training cycle, the parameters of the target deep learning model are frozen, and only the parameters of the differentiable feature scoring module and the reconstructed mapping layer are updated for several optimization steps. Then the freeze is lifted and global joint optimization is performed. This phased control continues for several training cycles before terminating and switching to the conventional end-to-end training mode.

7. The deep learning feature selection method according to claim 1, characterized in that, The method also includes periodically recording the cumulative frequency of each feature dimension being selected during the training process, and generating a feature importance ranking list based on the cumulative frequency after training. The ranking list is used to explain the model decision basis or as a reference for input feature selection in subsequent lightweight model design.

8. A deep learning feature selection system for processing image data, text data, and sensor data, characterized in that, include: The feature input module is used to receive raw high-dimensional feature data, perform standardized preprocessing, and output input feature tensors with uniform dimensions. The differentiable feature scoring module is used to perform differentiable weight allocation on each feature dimension in the input feature tensor and generate a continuous scoring vector corresponding to the importance of the features; the adaptive threshold generation module is used to dynamically calculate the feature selection threshold based on the loss change rate and gradient magnitude in the current training stage and output a threshold scalar that is adaptively adjusted with the training process. The sparse selection module is used to perform differentiable hard selection or soft selection operations based on the score vector and the threshold scalar to generate a sparse feature subset representation. The feature reconstruction module is used to map the filtered feature subset back to the original feature space dimension, generating a dense feature representation that is compatible with downstream models; The model co-training module is used to input the reconstructed feature representation into the target deep learning model for forward propagation, and to synchronously update the feature scoring parameters and model parameters through a joint backpropagation algorithm; the monitoring and feedback module is used to record the feature scoring distribution, screening ratio and model performance indicators in real time, and to provide visualization of the training process and anomaly detection capabilities.

9. The deep learning feature selection system according to claim 8, characterized in that, The feature reconstruction module adopts a gated reconstruction mechanism, including a learnable reconstruction weight matrix and bias vector. Dimension mapping is achieved through matrix multiplication, and a Layer Normalization layer is introduced to normalize the output. At the same time, a residual connection structure is set to add the mean vector of the original input features to the reconstruction output to compensate for the distribution shift caused by feature loss.

10. The deep learning feature selection system according to claim 8, characterized in that, The model co-training module implements a phased parameter update strategy during backpropagation. First, the target model parameters are frozen and the parameters of the feature scoring module and the feature reconstruction module are updated separately. Then, all parameters are unfrozen for joint optimization. This strategy gradually adjusts the freezing ratio according to the training cycle to achieve a smooth transition from feature selection-driven to model learning-driven.

Images