Sensor feature point automatic selection method based on Pearson correlation optimization

Abstract

The invention discloses a sensor feature point automatic selection method based on Pearson correlation optimization. The method comprises the following steps of data preparation and preprocessing, feature evaluation and selection, model construction and training, and online application and dynamic optimization. According to the method, irrelevant and redundant features are systematically eliminated, so that the data dimension is remarkably reduced, the performance is improved, the model calculation cost is greatly reduced, the efficiency is improved, meanwhile, the selected feature points can represent the data more essentially, and the precision and generalization ability of a classifier or a prediction model are remarkably improved. The algorithm adopted by the method can cooperatively process linear and non-linear relationships, so that the method has higher robustness and universality than a traditional single method when facing complex and diversified industrial data.

Description

Technical Field

[0001] This invention relates to the field of data processing and analysis technology, specifically to an automatic sensor feature point selection method based on Pearson correlation optimization. Background Technology

[0002] With the rapid development of intelligent sensing technology, the scale of sensors deployed in aerospace, aviation, weaponry, vehicles, and underwater equipment continues to expand, generating massive amounts of high-dimensional time-series monitoring data. In complex industrial scenarios such as rocket control, aircraft power management, and missile launch systems, how to automatically select highly representative and discriminative feature points from tens of thousands of measurement points has become a key issue in improving modeling efficiency and enhancing state awareness and decision-making capabilities.

[0003] In traditional engineering practice, the selection of feature points often relies on finite element analysis or expert experience, determining the location of the points based on stress extrema or specific physical criteria. While these methods may be effective under certain conditions, they have significant limitations when dealing with high-dimensional, nonlinear, and strongly coupled real-world data: the finite element method depends on ideal assumptions and static models, making it difficult to adapt to dynamic multiphysics scenarios; and human experience lacks scalability and cannot meet the screening requirements of tens of thousands of measurement points.

[0004] In data-driven approaches, the Pearson correlation coefficient is widely used due to its high computational efficiency and strong interpretability. However, it can only capture linear relationships and is difficult to identify the nonlinear dependencies that are common in sensor data. At the same time, this method lacks effective handling of feature redundancy, which can easily lead to insufficient diversity of the selected feature subset and affect the performance of subsequent modeling.

[0005] In summary, the current selection of feature points for high-dimensional sensors still faces challenges such as difficulty in balancing linear and nonlinear relationships, high feature redundancy, and reliance on manual prior knowledge. There is an urgent need to develop a new feature selection method that can integrate multi-dimensional correlation assessment, has a high degree of automation, and is feasible for engineering implementation. Summary of the Invention

[0006] The purpose of this invention is to provide an automatic sensor feature point selection method based on Pearson correlation optimization to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: an automatic sensor feature point selection method based on Pearson correlation optimization, comprising the following steps:

[0008] S1: Sample massive amounts of continuous, high-sampling-rate sensor data from real-world scenarios to obtain raw, high-dimensional sensor data;

[0009] S2: Apply a bandpass filter to the collected high-dimensional sensor data to remove power frequency interference and improve data quality;

[0010] S3: For each feature processed by S2, apply mutual information measurement to capture the evaluation metric of nonlinear relationships, evaluate its correlation with the target variable, and realize nonlinear correlation measurement and initial screening.

[0011] S4: Based on the correlation metric obtained in S3, sort all features in descending order, and form a candidate subset containing all potentially related features according to a preset threshold or by selecting the top N features.

[0012] S5: Apply the Pearson correlation coefficient to evaluate the linear redundancy between any two features in the candidate feature subset obtained in S4;

[0013] S6: Define a comprehensive feature importance metric and construct a fitness function to provide a clear objective function for subsequent optimization search, ensuring that the selected feature subset is both discriminative and independent;

[0014] S7: Using the comprehensive evaluation criteria defined in S6 as the fitness function, a heuristic search algorithm is used to select the optimal, non-redundant feature subset from the feature candidate subset in S3;

[0015] S8: Take the optimal feature subset finally selected in S7 as input data, normalize all features so that their mean is zero and their variance is one, and ensure that the input features are trained on the same scale.

[0016] S9: Introduce a feature representation layer to the normalized input data to capture the non-linear correlation between features;

[0017] S10: After obtaining the initial feature representation, a convolutional neural network structure is introduced to extract and compress local patterns of the input features, thereby effectively extracting the local correlation information of the input features in the spatial dimension.

[0018] S11: Introducing a multi-head self-attention mechanism on the basis of convolutional features to enhance the ability to model global dependencies;

[0019] S12: Based on the global representation, a multilayer perceptron is used to complete the prediction;

[0020] S13: Introducing a loss function into the deep neural network prediction model based on adversarial training loss enhances the model's generalization feature representation and improves the model's robustness;

[0021] S14: Apply the final feature subset obtained in S7 to specific sensor selection points, determine the actual sensor deployment location based on the selected feature points, or activate these points in the virtual grid for real-time data processing.

[0022] S15: The real-time data stream collected in S14 after the sensor points are selected and deployed is processed through the same feature processing and selection process from S2 to S6, and then input into the deep neural network model trained in S7 for online real-time calculation.

[0023] S16: Pair newly generated real-time data with predicted values ​​and automatically add them to the original historical dataset of S1 to form a continuously expanding database;

[0024] S17: Based on the new data accumulated in S16, when a decline in model prediction performance is detected, online fine-tuning technology is used to quickly and lightweightly update the parameters of the deep neural network model in S7 using the new data;

[0025] S18: Reintegrate the newly processed data from S17 into the original dataset of S1. The data includes the original data, prediction results, and actual validation results. Repeat steps S2 to S6 periodically to achieve dynamic optimization and adaptive enhancement of the model.

[0026] Furthermore, in step S3, a mutual information measurement method is applied to each feature to capture the evaluation metric of nonlinear relationships. The formula for mutual information measurement is:

[0027]

[0028] in, For joint probability density, and This represents the marginal probability.

[0029] Furthermore, in step S5, the Pearson correlation coefficient is applied to assess the linear redundancy between any two features. The formula for calculating the Pearson correlation coefficient is as follows:

[0030]

[0031] Here, X and Y represent two distinct features.

[0032] Furthermore, in step S6, a comprehensive feature importance metric is defined as the fitness function, with the following formula:

[0033]

[0034] Where Rel represents the total relevance score, and represents the information gain of the i-th feature, calculated based on mutual information (MI). α represents the redundancy penalty, which is the average Pearson correlation coefficient between any two features in the candidate subset, and α is an adjustable weight. It is the independent reward term, |⋅| is taken as its absolute value to handle negative correlation, and β is the weight parameter.

[0035] Furthermore, in step S7, the heuristic search algorithm employs a genetic algorithm, a greedy algorithm, or a recursive feature elimination algorithm. Specifically, a genetic algorithm is used to select the optimal, non-redundant feature subset from the feature candidate subset; the fitness is the average fitness of the subset S. ;

[0036] Implemented using the DEAP library, the optimal subset is selected after convergence, providing efficient input for model building.

[0037] Furthermore, in step S9, a feature representation layer is introduced to capture the nonlinear correlation between features. The feature representation layer adopts a graph attention mechanism, and its calculation formula is as follows:

[0038]

[0039] in, Indicates attention weights. This is the weight matrix. It is a nonlinear ReLU activation function; This represents the neighborhood of node i.

[0040] Furthermore, in step S10, a convolutional neural network structure is introduced to extract local patterns. The convolution operation formula of the convolutional neural network is:

[0041]

[0042] in, Indicates the convolution kernel weights. For bias terms, This is the activation function.

[0043] Furthermore, in step S11, a multi-head self-attention mechanism is introduced to enhance global dependency. The calculation formula for the multi-head self-attention mechanism is as follows:

[0044]

[0045] in, , , , representing the query, key, and value matrix, respectively. Let be the scaling factor; for multi-head attention mechanisms, we have:

[0046] .

[0047] Furthermore, in step S13, an adversarial training loss function is introduced to enhance the robustness of the model. The adversarial training loss function is defined as follows:

[0048]

[0049] in, This represents a deep neural network prediction model. Represents the basic loss function. For the input sample, Output as the target. For the disturbance term, For the amplitude constraint of the disturbance.

[0050] Furthermore, in step S17, based on the accumulated new data, when a decline in model prediction performance is detected, an online fine-tuning technique is adopted to quickly and lightweightly update the parameters of the deep neural network model using the new data; the online fine-tuning technique adopts a low-rank adaptation algorithm, which only updates the parameters of the low-rank matrix.

[0051] Compared with the prior art, the beneficial effects of the present invention are:

[0052] This invention significantly reduces data dimensionality and improves performance by systematically eliminating irrelevant and redundant features. This substantially reduces model computation costs and increases efficiency while ensuring that the selected feature points more fundamentally represent the data, significantly improving the accuracy and generalization ability of classifiers or prediction models. The algorithm employed in this method can collaboratively handle linear and nonlinear relationships, making it more robust and universal than traditional single methods when dealing with complex and diverse industrial data. The entire process is highly automated, avoiding reliance on human experience in traditional analysis and preserving the physical meaning of the original features, ensuring the interpretability of the results. Furthermore, this invention can quickly locate and acquire sensor feature regions showing key changes. Through real-time data processing and periodic dynamic optimization, it achieves continuous monitoring of the system state and self-updating of the model. This greatly enhances the versatility, convenience, and timeliness of the method in industrial proprietary protocols and complex scenarios. Attached Figure Description

[0053] Figure 1 This is a flowchart of the present invention;

[0054] Figure 2 A detailed flowchart of the present invention. Detailed Implementation

[0055] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0056] Please see Figure 1-2 This invention provides an automatic sensor feature point selection method based on Pearson correlation optimization, comprising the following steps:

[0057] S1: Data preparation and preprocessing;

[0058] S2: Feature evaluation and selection;

[0059] S3: Model building and training;

[0060] S4: Online application and dynamic optimization.

[0061] like Figure 2 As shown, the specific implementation steps are as follows:

[0062] S1: Data preparation and preprocessing, including sampling massive amounts of continuous, high-sampling-rate sensor data from actual field scenarios to obtain raw high-dimensional sensor data (sub-step S1.1), and applying a bandpass filter to the collected high-dimensional sensor data to remove power frequency interference in order to improve data quality (sub-step S1.2).

[0063] S1.1: Sample massive amounts of continuous, high-sampling-rate sensor data from real-world scenarios to obtain raw high-dimensional sensor data;

[0064] In this step, for the massive high-dimensional time-series monitoring data generated by sensor deployments in aerospace, aviation, and weaponry fields, this embodiment uses a rocket control system as an example to perform data sampling. Specifically, a sensor array, including accelerometers, temperature sensors, strain gauges, and vibration sensors, is first deployed in an actual field scenario (such as a launch site test platform), collecting data streams from 10,000 measurement points. A fixed-interval sampling strategy is adopted, with a sampling frequency of 1000Hz and a sampling duration of 10 minutes, forming an original high-dimensional dataset containing 60 million data points. .

[0065] The sampling process is implemented through dedicated DAQ (Data Acquisition) hardware (such as the NIDAQmx system) to ensure continuity and high sampling rate, while recording auxiliary metadata such as time, ambient temperature, and humidity.

[0066] This step aims to acquire raw, unprocessed high-dimensional sensor data, laying the foundation for subsequent preprocessing and avoiding errors caused by data loss or sampling bias. The sampling strategy considers dynamic operating conditions, such as increasing the sampling density to 2000Hz during rocket ignition simulations.

[0067] S1.2: Apply a bandpass filter to the acquired high-dimensional sensor data to remove power frequency interference and improve data quality.

[0068] In this step, a bandpass filter is applied to the raw data to remove power frequency interference (50 / 60Hz noise source) and low-frequency drift, thereby improving data quality.

[0069] Specifically, this embodiment employs a Butterworth bandpass filter design, with a passband frequency range of 5-200Hz, determined based on the sensor signal spectral characteristics, covering the main vibration and strain frequency bands, and an order of 4. The filtering process is implemented using Python's SciPy library, and the filtering formula is as follows:

[0070]

[0071] in, Let be the filter order (4). and These are the normalized low-pass and high-pass cutoff frequencies (5 / 500 and 200 / 500), respectively. The sampling frequency is 1000Hz. This indicates a convolution operation. After filtering, the power frequency peak (50Hz) is effectively suppressed, and the signal-to-noise ratio (SNR) is improved by approximately 15dB.

[0072] S2: Feature evaluation and selection, including applying mutual information measurement to each feature after filtering in S1 to capture nonlinear relationships, evaluating its correlation with the target variable, and realizing nonlinear correlation measurement and initial screening (sub-step S2.1); based on the correlation measurement values ​​obtained in S2.1, sorting all features in descending order, and forming a preliminary candidate subset containing all potentially related features according to a preset threshold or by selecting the top N features (sub-step S2.2); applying the Pearson correlation coefficient to evaluate the linear redundancy between any two features in the candidate feature subset obtained in S2.2 (sub-step S2.3); defining a comprehensive "feature importance" measure, using I as the fitness function (sub-step S2.4); using the comprehensive evaluation criteria defined in S2.4 as the fitness function, and using a heuristic search algorithm (such as genetic algorithm, greedy algorithm, or recursive feature elimination) to select the optimal, non-redundant feature subset from the candidate feature subset in S2.1 (sub-step S2.5).

[0073] S2.1: After filtering each feature processed in S1.2, apply Mutual Information (MI) measurement to each feature to capture the evaluation metric of nonlinear relationships, so as to evaluate its correlation with the target variable, thereby realizing nonlinear correlation measurement and initial screening;

[0074] The MI formula is expressed as:

[0075]

[0076] in, For joint probability density, and This represents the marginal probability.

[0077] The process of calculating mutual information for features using the MI formula is implemented using Python's sklearn.feature_selection.mutual_info_regression, a powerful tool for feature selection specifically designed for regression problems, capable of discovering any form of linear and non-linear statistical relationship between features and target variables.

[0078] This sub-step outputs a vector of correlation scores. This ensures that nonlinear dependencies (such as vibration-temperature coupling) in the sensor data are captured, providing a basis for subsequent sorting.

[0079] S2.2: Based on the correlation metric obtained in sub-step S2.1 All features are sorted in descending order. Based on a preset threshold (MI > 0.15) or by selecting the top N=5000 features, a preliminary candidate subset containing all potentially relevant features is formed.

[0080] S2.3: For the feature candidate subset obtained in step S2.2 The Pearson correlation coefficient is used to assess the linear redundancy between any two features.

[0081] Specifically, for the feature candidate subset Each pair of features Calculate the Pearson coefficient matrix;

[0082] Pearson's formula is expressed as:

[0083]

[0084] X and Y represent two distinct features.

[0085] Highly redundant pairs are marked by computing the entire matrix (5000×5000) in parallel, with a threshold |ρ| > 0.8. The average redundancy score is output. This sub-step quantifies linear redundancy to ensure that subsequent comprehensive metrics avoid multicollinearity.

[0086] S2.4: Define a comprehensive feature importance metric, using I as the fitness function, as shown below:

[0087]

[0088] In the formula, Rel represents the total relevance score. The information gain of the i-th feature is calculated based on mutual information (MI), which encourages the selection of features that are highly correlated with the target variable. The redundancy penalty is represented by the average Pearson correlation coefficient between any two features in the candidate subset. α is an adjustable weight used to balance the penalty strength. The negative sign ensures that high redundancy will reduce I, promoting feature independence. It is an independent reward item, and |⋅| is taken as its absolute value to handle negative correlation. Approaching 1 for low redundancy, β is another weighting parameter used to further emphasize independence, which complements the penalty term. This comprehensive evaluation criterion provides a clear objective function for subsequent optimization searches, ensuring that the selected feature subset is both discriminative and independent.

[0089] The computation is implemented using vectorized NumPy. Output importance vector This sub-step ensures that the selected feature subset is both discriminative (high Rel) and independent (low Red).

[0090] S2.5: Use the comprehensive evaluation criteria defined in S2.4 as the fitness function, and use a heuristic search algorithm to select the optimal, non-redundant feature subset from the feature candidate subset in S2.1;

[0091] Specifically, this embodiment uses a genetic algorithm (GA), with a population size of 100, 50 generations, a crossover rate of 0.8, and a mutation rate of 0.1. The fitness is the average of subset S. Implemented using the DEAP library, the optimal subset is selected after convergence. This provides efficient input for model building.

[0092] S3: Model building and training, including normalizing the optimal feature subset finally selected in S2 as input data (sub-step S3.1); introducing a feature representation layer into the normalized input data to capture the nonlinear correlation between features, using a graph attention mechanism (GAT) (sub-step S3.2); after obtaining the preliminary feature representation, introducing a convolutional neural network (CNN) structure to extract and compress local patterns of the input features (sub-step S3.3); introducing a multi-head self-attention mechanism on the basis of CNN features to enhance the modeling ability of global dependencies (sub-step S3.4); using a multilayer perceptron (MLP) to complete the prediction based on the global representation and output the target physical quantity (sub-step S3.5); introducing a loss function based on adversarial training loss into the deep neural network prediction model to enhance the model's generalization and robustness (sub-step S3.6).

[0093] S3.1: The optimal feature subset finally selected in sub-step S2.5 As input data, the input feature vector is denoted as Each sample The target physical quantity is rocket vibration intensity, etc., denoted as Before input, all Z-scores are normalized to have a mean of zero and a variance of one, ensuring that the input features are trained on the same scale and avoiding scale differences affecting model convergence. The normalization formula is as follows:

[0094]

[0095] in, The mean, Let Variance be the variance.

[0096] S3.2: The normalized input data from sub-step S3.1 A feature representation layer is introduced to capture the nonlinear correlations between features. This embodiment employs a graph attention mechanism (GAT). First, a feature point graph structure is constructed, with 200 features per node and edges based on a spatial neighborhood threshold of 0.5. Then, attention-weighted aggregation is calculated. The GAT calculation formula is as follows:

[0097]

[0098] in, Indicates attention weights. This is the weight matrix. It is a non-linear ReLU activation function. This represents the neighborhood of node i.

[0099] Implemented using PyTorchGeometric, with K=8 heads and 64 hidden dimensions. This sub-step outputs a preliminary nonlinear representation. It effectively captures the spatial dependence between sensor features (such as vibration-strain coupling) and improves nonlinear expression capability by about 25%.

[0100] S3.3: After obtaining the initial feature representation, in order to model the temporal or dynamic dependencies of the input data, a convolutional neural network structure is introduced to extract and compress local patterns of the input features. The convolution operation formula is as follows:

[0101]

[0102] in, Indicates the convolution kernel weights. For bias terms, This is the activation function. This step effectively extracts local correlation information of input features in the spatial dimension.

[0103] This embodiment uses a 1DCNN (adapted to temporal sensors), with a kernel size of 3 and the number of channels increased from 64 to 128. This sub-step outputs compressed features, extracts local patterns (such as short-duration vibration pulses), and reduces the number of parameters by 20%.

[0104] S3.4: A multi-head self-attention mechanism is introduced based on convolutional features to enhance the modeling ability of global dependencies. The calculation process is as follows:

[0105]

[0106] in, , , , representing the query, key, and value matrix, respectively. Let be the scaling factor. For multi-head attention mechanisms, we have:

[0107]

[0108] This mechanism establishes global weighted relationships between features, enabling the model to obtain a more comprehensive feature representation. This sub-step establishes global relationships between features (such as cross-measurement point coupling), improving the model's sensitivity to long-range dependencies by 15%.

[0109] S3.5: Based on the global representation, a Multilayer Perceptron (MLP) is used to complete the prediction. Specifically, the MLP structure consists of two hidden layers (256→128), followed by Dropout (0.2) and a linear output layer. The output is:

[0110]

[0111] in, To fuse feature vectors, These are the network parameters. Output results. This is the predicted value of the target physical quantity.

[0112] S3.6: An adversarial training loss function is introduced into the deep neural network prediction model to enhance its feature representation and thus significantly improve its robustness. Specifically, a perturbation vector is added to the input samples. And limit the amplitude of the disturbance to no more than a preset threshold. This is used to simulate the worst-case input sample, thereby improving the model's generalization ability. Its loss function is defined as follows:

[0113]

[0114] in, This represents a deep neural network prediction model. Represents the basic loss function. For the input sample, Output as the target. For the disturbance term, For the amplitude constraint of the disturbance.

[0115] This sub-step enhances robustness through adversarial approaches, improving validation set accuracy by 10% and resisting sensor noise >20dB.

[0116] S4: Online application and dynamic optimization, including applying the deep neural network model trained in S3 and the optimal feature subset finally selected in S2 to specific sensor locations, determining the actual sensor deployment location based on the selected feature points, or activating these points in a virtual grid for real-time data processing (sub-step S4.1); the real-time data stream collected after sensor location deployment, after preprocessing in S1 and feature processing and selection in S2, is input into the deep neural network model trained in S3 for online real-time calculation (sub-step S4.2); these newly generated real-time data-predicted values ​​are then paired and sampled. This data is automatically added to the original historical dataset of S1, forming a continuously expanding database (sub-step S4.3). Based on the new data accumulated in S4.3, when a decline in model prediction performance is detected, online fine-tuning technology is used to quickly and lightweightly update the parameters of the deep neural network model in S3 using the new data (sub-step S4.4). The newly processed data in S4.4 (including original data, prediction results, and actual validation results) is reintegrated into the original historical dataset of S1, and the steps from S1 to S2 are repeated periodically to achieve dynamic optimization and adaptive enhancement of the model (sub-step S4.5).

[0117] S4.1: Combine the deep neural network model trained in sub-step S3.6 with the optimal feature subset finally selected in sub-step S2.5. (Approximately 200 dimensions) are applied to specific sensor point selection, determining the actual sensor deployment location based on the selected feature points, or activating these points in a virtual grid for real-time data processing.

[0118] Specifically, this embodiment will Map the physical sensor grid (10000×10000 virtual grid, unit m) to the rocket control system, highlighting points with importance I_i > 0.7 (approximately 50 physical locations, such as key nodes of the thruster support).

[0119] The deployment strategy prioritizes deploying high-I_i points (cost optimization <20% of total sensors) or virtual activation (simulation testing). Real-time processing is achieved via an edge device (Raspberry Pi), with latency <10ms. The output activation point set is then used. The coordinates are based on the finite element model. This sub-step bridges offline selection and online deployment, ensuring representativeness of the selected points and reducing redundant deployment by 30%.

[0120] S4.2: The real-time data stream collected in S4.1 after sensor point selection and deployment, after preprocessing in S1 and feature processing and selection in S2, is input into the trained deep neural network model in S3 for online real-time calculation.

[0121] Specifically, real-time data stream (Sampling rate 1000Hz, flow rate 10k / s) First, the sample is bandpass filtered by S1.2, then rapidly evaluated by S2.1-S2.5 (incremental MI / Pearson, <1s / batch) to extract a subset. After normalization, the input model f predicts the vibration intensity. The accuracy rate is over 95%.

[0122] S4.3: Pair these newly generated real-time data-predicted values ​​from sub-step S4.2 with samples and automatically add them to the original historical dataset in S1 to form a continuously expanding database;

[0123] Specifically, each batch of real-time samples (n=1000) After tag verification (manual / sensor feedback), the data is added to the historical database. (Initial 60 million points, growth rate 5% / day).

[0124] Using SQLite / HDF5 storage and incremental indexes accelerate queries. This sub-step supports data self-growth, supporting databases up to hundreds of millions in size and improving model adaptability.

[0125] S4.4: Based on the new data accumulated in sub-step S4.3, when a decline in model prediction performance is detected, such as when MSE > 0.1, online fine-tuning technology is used to quickly and lightweightly update the parameters of the deep neural network model in S3 using the new data.

[0126] Fine-tuning is performed using LoRA (Low-Rank Adaptation), updating only the low-rank matrix (rank=8, <1% of parameters), lr=1e-5, epochs=5. The detection threshold is monitored based on a sliding window MAE. Loss: The update time can be less than 30 seconds. This sub-step maintains performance and can achieve a drift recovery rate of over 90%.

[0127] S4.5: Reintegrate the newly processed data (including original data, prediction results and actual validation results) from sub-step S4.4 into the original historical dataset of S1, and repeat steps S1 to S2 periodically (every 24 hours) to achieve dynamic optimization and adaptive enhancement of the model.

[0128] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An automatic sensor feature point selection method based on Pearson correlation optimization, characterized in that: Includes the following steps: S1: Sample massive amounts of continuous, high-sampling-rate sensor data from real-world scenarios to obtain raw, high-dimensional sensor data; S2: Apply a bandpass filter to the collected high-dimensional sensor data to remove power frequency interference and improve data quality; S3: For each feature processed by S2, apply mutual information measurement to capture the evaluation metric of nonlinear relationships, evaluate its correlation with the target variable, and realize nonlinear correlation measurement and initial screening. S4: Based on the correlation metric obtained in S3, sort all features in descending order, and form a candidate subset containing all potentially related features according to a preset threshold or by selecting the top N features. S5: Apply the Pearson correlation coefficient to evaluate the linear redundancy between any two features in the candidate feature subset obtained in S4; S6: Define a comprehensive feature importance metric and construct a fitness function to provide a clear objective function for subsequent optimization search, ensuring that the selected feature subset is both discriminative and independent; S7: Using the comprehensive evaluation criteria defined in S6 as the fitness function, a heuristic search algorithm is used to select the optimal, non-redundant feature subset from the feature candidate subset in S3; S8: Take the optimal feature subset finally selected in S7 as input data, normalize all features so that their mean is zero and their variance is one, and ensure that the input features are trained on the same scale. S9: Introduce a feature representation layer to the normalized input data to capture the non-linear correlation between features; S10: After obtaining the initial feature representation, a convolutional neural network structure is introduced to extract and compress local patterns of the input features, thereby effectively extracting the local correlation information of the input features in the spatial dimension. S11: Introducing a multi-head self-attention mechanism on the basis of convolutional features to enhance the ability to model global dependencies; S12: Based on the global representation, a multilayer perceptron is used to complete the prediction; S13: Introducing a loss function into the deep neural network prediction model based on adversarial training loss enhances the model's generalization feature representation and improves the model's robustness; S14: Apply the final feature subset obtained in S7 to specific sensor selection points, determine the actual sensor deployment location based on the selected feature points, or activate these points in the virtual grid for real-time data processing. S15: The real-time data stream collected in S14 after the sensor points are selected and deployed is processed through the same feature processing and selection process from S2 to S6, and then input into the deep neural network model trained in S7 for online real-time calculation. S16: Pair newly generated real-time data with predicted values ​​and automatically add them to the original historical dataset of S1 to form a continuously expanding database; S17: Based on the new data accumulated in S16, when a decline in model prediction performance is detected, online fine-tuning technology is used to quickly and lightweightly update the parameters of the deep neural network model in S7 using the new data; S18: Reintegrate the newly processed data from S17 into the original dataset of S1. The data includes the original data, prediction results, and actual validation results. Repeat steps S2 to S6 periodically to achieve dynamic optimization and adaptive enhancement of the model.

2. The automatic sensor feature point selection method based on Pearson correlation optimization according to claim 1, characterized in that: In step S3, a mutual information measurement method is applied to each feature to capture the evaluation metric of nonlinear relationships. The formula for mutual information measurement is: ， in, For joint probability density, and This represents the marginal probability.

3. The automatic sensor feature point selection method based on Pearson correlation optimization according to claim 1, characterized in that: In step S5, the Pearson correlation coefficient is applied to assess the linear redundancy between any two features. The formula for calculating the Pearson correlation coefficient is as follows: ， Here, X and Y represent two distinct features.

4. The automatic sensor feature point selection method based on Pearson correlation optimization according to claim 1, characterized in that: In step S6, a comprehensive feature importance metric is defined as the fitness function, with the following formula: ， Where Rel represents the total relevance score, and represents the information gain of the i-th feature, calculated based on mutual information (MI); α represents the redundancy penalty, which is the average Pearson correlation coefficient between any two features in the candidate subset, and α is an adjustable weight. It is an independent reward term, |⋅| is taken as its absolute value to handle negative correlation, and β is a weighting parameter.

5. The automatic sensor feature point selection method based on Pearson correlation optimization according to claim 1, characterized in that: In step S7, the heuristic search algorithm employs a genetic algorithm, a greedy algorithm, or a recursive feature elimination algorithm. Specifically, a genetic algorithm is used to select the optimal, non-redundant feature subset from the feature candidate subset; the fitness is the average fitness of the subset S. ; Implemented using the DEAP library, the optimal subset is selected after convergence, providing efficient input for model building.

6. The automatic sensor feature point selection method based on Pearson correlation optimization according to claim 1, characterized in that: In step S9, a feature representation layer is introduced to capture the nonlinear correlation between features. The feature representation layer adopts a graph attention mechanism, and its calculation formula is as follows: ， in, Indicates attention weights. This is the weight matrix. It is a nonlinear ReLU activation function; This represents the neighborhood of node i.

7. The automatic sensor feature point selection method based on Pearson correlation optimization according to claim 1, characterized in that: In step S10, a convolutional neural network structure is introduced to extract local patterns. The convolution operation formula of the convolutional neural network is as follows: ， in, Indicates the convolution kernel weights. For bias terms, This is the activation function.

8. The automatic sensor feature point selection method based on Pearson correlation optimization according to claim 1, characterized in that: In step S11, a multi-head self-attention mechanism is introduced to enhance global dependency. The calculation formula for the multi-head self-attention mechanism is as follows: ， in, , , , representing the query, key, and value matrix, respectively. Let be the scaling factor; for multi-head attention mechanisms, we have: 。 9. The automatic sensor feature point selection method based on Pearson correlation optimization according to claim 1, characterized in that: In step S13, an adversarial training loss function is introduced to enhance the robustness of the model. The adversarial training loss function is defined as follows: ， in, This represents a deep neural network prediction model. Represents the basic loss function. For the input sample, Output as the target. For the disturbance term, For the amplitude constraint of the disturbance.

10. The automatic sensor feature point selection method based on Pearson correlation optimization according to claim 1, characterized in that: In step S17, based on the accumulated new data, when a decline in model prediction performance is detected, an online fine-tuning technique is used to quickly and lightweightly update the parameters of the deep neural network model using the new data. The online fine-tuning technique uses a low-rank adaptation algorithm to update only the parameters of the low-rank matrix.

Images