A multi-sensor data correction method based on correlation analysis and integrated model

Through the multi-sensor data correction method based on correlation analysis and integrated model, the nonlinearity and complexity of sensor data correction in marine ecological environment monitoring is solved, efficient, real-time and low-cost data correction is achieved, and the correction accuracy and robustness.

CN118820690BActive Publication Date: 2024-12-03QILU UNIVERSITY OF TECHNOLOGY (SHANDONG ACADEMY OF SCIENCES) +2
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411302755.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-19
Publication Date
2024-12-03
Estimated Expiration
2044-09-19

AI Technical Summary

Technical Problem

The existing marine ecological environment monitoring sensor data correction method has limitations when facing complex nonlinear data, and manual calibration is time-consuming and costly. The data fusion method is not suitable for highly nonlinear or complex correlation problems and lacks real-time performance. and flexibility.

Method used

Using multi-sensor data correction method based on correlation analysis and integrated model, the sensor data is acquired for preprocessing, data drift phenomenon is detected, mutual information and Pearson correlation coefficient are used to construct a joint correlation function, and the correlation is extracted in combination with Bayesian optimization Highly efficient monitoring elements, build ridge regression and Lasso regression models, use the Stacking method to integrate the model, generate the final correction model, and realize the correction of drift data.

Benefits of technology

Capture nonlinear relationships in complex marine environments, improve the robustness and real-time nature of data correction, reduce costs, significantly improve correction accuracy, solve problems caused by data drift, and provide more real-time and accurate correction methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118820690B_ABST
    Figure CN118820690B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of electronic digital data processing, and discloses a multi-sensor data correction method based on correlation analysis and integrated model, comprising the following steps: obtaining monitoring element data of a sensor array in a monitoring sea area, detecting sensors with drift phenomenon, and constructing a training set and a correction set; performing correlation analysis on the data in the training set, constructing a joint correlation function, optimizing the joint correlation function using a Bayesian optimization method, and constructing a monitoring element set C; integrating ridge regression and Lasso regression using a Stacking method, training a meta-learner, and generating a final correction model; extracting data of the same type as the monitoring element set C in the correction set and inputting them into the correction model, and outputting predicted correction data. The method disclosed by the present invention can solve the shortcomings of highly nonlinear or complex correlation data and realize the correction of sensor data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of electrical digital data processing, and in particular to a multi-sensor data correction method based on correlation analysis and integrated model. Background Art

[0002] Monitoring various elements in the marine ecological environment is of great significance for real-time grasp of marine information and research on marine changes. The use of various in-situ sensors to achieve continuous and long-term data collection of various ecological and environmental elements in seawater is an important technical means to achieve marine ecological and environmental monitoring. However, due to the long-term operation of various in-situ sensors in the marine environment, they may be affected by many factors, such as hardware aging, biological attachment, environmental mutations and mechanical damage, which may cause drift in the collected data, causing serious interference to the accuracy of the monitoring data, and also have many impacts on subsequent data analysis and decision-making. Therefore, it is very important and indispensable to take necessary corrections to the sensor data involved in the above situations.

[0003] The existing various correction methods can achieve correction to a certain extent with different technologies and means, but there are still certain limitations. For example, the manual calibration method can ensure the accuracy of measurement by manually calibrating the sensor regularly, but this method is time-consuming and labor-intensive, and lacks the real-time nature of the correction data. Model-based correction methods usually assume that sensor data has a specific pattern and use historical data or physical models to correct sensor data, such as linear regression or Kalman filtering, but for the complex nonlinear data of marine ecological environment monitoring, such methods still have limitations in the analysis of data. Data fusion methods are currently more commonly used methods, such as Bayesian inference, weighted average, etc., which realize the correction function of data by fusing multiple data. Such methods are more effective in processing multi-source linear data, but are not suitable for highly nonlinear or complex correlation problems.

[0004] Based on the above analysis, a data correction method based on the existing multi-sensor monitoring conditions of the marine ecological environment is needed, so that it can realize the function of correcting the sensor data without incurring additional costs, thereby achieving the purpose of saving costs and overcoming the limitations of existing methods. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a multi-sensor data correction method based on correlation analysis and integrated model, so as to achieve the purpose of correcting the sensor data without incurring additional costs.

[0006] To achieve the above object, the technical solution of the present invention is as follows:

[0007] A multi-sensor data correction method based on correlation analysis and integrated model comprises the following steps:

[0008] Step 1: obtain various monitoring element data of the sensor array in the monitoring sea area, perform data preprocessing and classified storage, detect the monitoring element data of sensor A with drift phenomenon, mark the drift starting node T, and use the node T as the dividing boundary, build a training set with the data collected before node T, and use the data collected after node T as the calibration set;

[0009] Step 2: Perform correlation analysis on the data of sensor A in the training set and the data of other sensors to obtain mutual information and Pearson correlation coefficient, and construct a joint correlation function by assigning different weights to the mutual information and Pearson correlation coefficient respectively. Use the Bayesian optimization method to optimize the weights in the joint correlation function, extract the monitoring elements whose correlation with the monitoring elements of sensor A is higher than the set value, and construct a new monitoring element set C;

[0010] Step 3, perform ridge regression modeling and Lasso regression modeling on the data of sensor A for each monitoring element data in monitoring element set C, obtain the prediction values ​​of ridge regression model and Lasso regression model, and perform R² test on the goodness of fit of ridge regression model and Lasso regression model respectively;

[0011] Step 4: Use the Stacking method to integrate the ridge regression and the Lasso regression, combine the predicted values ​​of the ridge regression model and the Lasso regression model to construct the input matrix Z, and select the meta-learner based on the comparison results of the goodness of fit R² test, and use the input matrix Z to train the meta-learner to generate the final correction model;

[0012] Step 5, extract the data of the same type as the monitoring element set C in the calibration set, input them into the calibration model, output the final predicted data, and realize the correction of the data detected to have drift phenomenon.

[0013] In the above scheme, in step 1, data preprocessing includes using the triple standard deviation method to detect and eliminate outliers, using the nearest interpolation method to fill the missing parts after eliminating outliers and the missing parts in the original data, and using the moving average method to smooth and denoise the data; then, the downsampling method is used to unify the sampling frequency of each data.

[0014] In the above scheme, in step 1, after data preprocessing, the data of sensor A and the data of other sensors are respectively constructed into data sets and , represents the jth data collected by sensor A; It represents the j-th data collected by the remaining i-th sensor, that is, the j-th data of the i-th monitoring element.

[0015] In the above scheme, in step 2, the mutual information is calculated as follows:

[0016] ;

[0017] in, express and The mutual information of , i represents the i-th monitoring element, express and The joint probability distribution of express The marginal probability distribution of express The marginal probability distribution of

[0018] The Pearson correlation coefficient is calculated as follows:

[0019] ;

[0020] in, express and The Pearson correlation coefficient, represents the data mean of the i-th monitoring element, Represents the mean value of the data collected by sensor A.

[0021] In the above scheme, in step 2, the joint correlation function is defined as:

[0022] ;

[0023] in, represents the joint correlation function, Expressed as The weight in the joint correlation function, α ∈[0,1] ; express The weight in the joint correlation function, β ∈[0,1] , ;

[0024] The joint correlation function is normalized and percentileized. The normalization formula is as follows:

[0025] ;

[0026] ;

[0027] The percentile formula is as follows:

[0028] ;

[0029] Among them, Total JC represents the sum of the weights of the joint correlations between all monitoring elements and the data of sensor A. represents the normalized result of the joint correlation function value of the i-th monitoring element, Represents the percentile result of the joint correlation function value of the i-th monitoring element.

[0030] In the above scheme, in step 2, the Bayesian optimization objective function formula is:

[0031] ;

[0032] Using Bayesian optimization, initialize weights and , automatically update the weights according to the initial results and repeat the iterative process, further updating the weights according to the current results in each iteration until the best weight combination is found and ; and is the objective function The maximum solution is expressed as:

[0033] ;

[0034] in, and Represents the best Bayesian optimization output and , Indicates When the maximum value is reached, and The value of

[0035] What you get and Substitution , and substitute each monitoring element into the calculation to obtain the final , and sort them by size to determine the importance of the correlation of various monitoring elements to the data of sensor A. According to the sorting size, several monitoring elements with correlation above 70% are selected to construct a new set C of monitoring elements.

[0036] In the above scheme, in step 3, the ridge regression modeling formula is as follows:

[0037] ;

[0038] The predicted values ​​of the ridge regression model are calculated as follows:

[0039] ;

[0040] in, represents the optimal weight vector of the ridge regression model, Indicated as When the minimum value is reached, The value of w =[ w 1 , w 2 ,…, w t ] T represents the weight vector, each Indicates the tth monitoring element The weight vector coefficients in the model, , , …, X t ] T Represents a set of monitoring elements, where X t =[ x t 1 , x t 2 ,…, x tn ] represents the n sample data of the tth monitoring element, Y represents All data of represents the L2 regularization parameter, represents the L2 norm, represents the predicted value of the ridge regression model;

[0041] The Lasso regression modeling formula is as follows:

[0042] ;

[0043] The predicted value of the Lasso regression model is calculated as follows:

[0044] ;

[0045] in, represents the optimal weight vector of the Lasso regression model, represents the L1 regularization parameter, represents the L1 norm, Represents the predicted value of the Lasso regression model.

[0046] In the above scheme, in step 3, the calculation formula of the R² test of the goodness of fit of the ridge regression model is as follows:

[0047] ;

[0048] ;

[0049] ;

[0050] in, represents the residual sum of squares of ridge regression, represents the total sum of squares of ridge regression, represents the jth data collected by sensor A, express The average value of all data, Represents the goodness-of-fit R² test result of the ridge regression model. represents the predicted value of the ridge regression model;

[0051] The calculation formula for the goodness of fit R² test of the Lasso regression model is as follows:

[0052] ;

[0053] ;

[0054] ;

[0055] in, represents the residual sum of squares of Lasso regression, represents the total sum of squares of Lasso regression, It represents the goodness of fit R² test result of Lasso regression model. Represents the predicted value of the Lasso regression model.

[0056] In the above scheme, in step 4, the constructed input matrix Z is as follows:

[0057] ;

[0058] in, represents the predicted value of the ridge regression model, Represents the predicted value of the Lasso regression model;

[0059] According to the goodness of fit R² test results of the ridge regression model The goodness of fit R² test results of the Lasso regression model Choose a meta-learner: If Higher than , select support vector regression as the meta-learner; if Lower than , choose gradient boosting regression as the meta-learner;

[0060] Input the input matrix Z into the meta-learner for training. After training, the final prediction result is obtained. ,calculate and The goodness-of-fit R² test results , according to all Compare the results and select the largest value As the result of the meta-learner training reaching the best state, the final correction model is generated , the test results of goodness of fit The calculation formula is as follows:

[0061] ;

[0062] ;

[0063] ;

[0064] in, express and The residual sum of squares, express The total sum of squares, express and The goodness-of-fit R² test results are: The final prediction result is represents the jth data collected by sensor A, express The average value of all data.

[0065] In the above scheme, in step 5, extract the data of the same type as the monitoring element set C in the calibration set, input it into the calibration model, and output the final prediction data , using the prediction results The data after the marked node is replaced, and finally the drift phenomenon in the sensor A data is corrected.

[0066] Through the above technical solution, the multi-sensor data correction method based on correlation analysis and integrated model provided by the present invention has the following beneficial effects:

[0067] 1. When faced with the complex and dynamically changing marine environment being monitored, the method of the present invention can capture the complex nonlinear and linear relationships in the monitoring data, process the nonlinear and complex correlation patterns in the data, so as to better extract the correlation existing in the data, and has stronger robustness and reliability;

[0068] 2. The present invention adopts a correction model based on the Stacking method to integrate ridge regression and Lasso regression, which can realize the function of quickly correcting the sensor with drift phenomenon. Compared with traditional manual calibration, this method is more flexible and has strong real-time performance;

[0069] 3. The present invention can achieve high-precision correction of data only through low-cost computers, without adding redundant sensors, reducing the overall complexity and operating costs in the field of data correction. At the same time, by integrating relevant information from multiple sensors, the correction accuracy of the target sensor can be significantly improved, solving a series of problems caused by data drift;

[0070] In summary, the multi-sensor data correction method based on joint correlation analysis provided by the present invention can solve the deficiencies of the existing methods to a certain extent and provide a more real-time and accurate correction means. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0072] Figure 1 A schematic flow chart of a multi-sensor data correction method based on correlation analysis and integrated model disclosed in an embodiment of the present invention;

[0073] Figure 2 A flowchart of a joint correlation analysis method based on mutual information and Pearson correlation coefficient disclosed in step 2 of the present invention;

[0074] Figure 3 The present invention discloses a flow chart of a correction model that integrates ridge regression and lasso regression based on the stacking method. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0076] The present invention provides a multi-sensor data correction method based on correlation analysis and integrated model, such as Figure 1 As shown, the following steps are included:

[0077] Step 1: obtain various monitoring element data of the sensor array in the monitored sea area, and perform data preprocessing and classified storage, detect the monitoring element data of sensor A with drift phenomenon, mark the node T where the drift phenomenon starts, and use the node T as the dividing boundary. Construct a training set with the data collected before node T, and use the data collected after node T as the correction set.

[0078] Each sensor in a set of marine ecological environment monitoring sensor arrays deployed in the monitored sea area is an in-situ marine sensor device. The monitored elements include seawater dissolved oxygen, seawater temperature, water depth, seawater salinity, pH value, ammonia nitrogen, silicate, phosphate, nitrate, nitrite, nutrients, chlorophyll-a, etc.

[0079] Data preprocessing includes using the triple standard deviation method to detect and eliminate outliers, using the nearest interpolation method to fill in the missing parts after eliminating outliers and the missing parts in the original data, and using the moving average method to smooth and denoise the data; then, the downsampling method is used to unify the sampling frequency of each data.

[0080] After preprocessing, by visualizing various types of monitoring data and combining expert experience for manual detection, a certain monitoring data with data drift phenomenon is identified, the node T where the data drift phenomenon occurs is marked, and the sensor where the data appears is recorded as sensor A.

[0081] After data preprocessing, the data of sensor A and the data of other sensors are used to construct data sets and , represents the jth data collected by sensor A; It represents the j-th data collected by the remaining i-th sensor, that is, the j-th data of the i-th monitoring element.

[0082] Step 2, such as Figure 2 As shown, a correlation analysis is performed on the data of sensor A in the training set and the data of other sensors to obtain the mutual information and the Pearson correlation coefficient, and a joint correlation function is constructed by assigning different weights to the mutual information and the Pearson correlation coefficient. The Bayesian optimization method is used to optimize the weights in the joint correlation function, and the monitoring elements whose correlation with the monitoring elements of sensor A is higher than the set value are extracted to construct a new monitoring element set C.

[0083] 1. Mutual Information

[0084] Mutual Information (MI) is a concept in probability theory and information theory. It is used to measure the degree of mutual dependence between two random variables and is more suitable for mining features in nonlinear relationships. The calculation of mutual information is as follows:

[0085] ;

[0086] in, express and The mutual information of , i represents the i-th monitoring element, express and The joint probability distribution of express The marginal probability distribution of express The marginal probability distribution of .

[0087] 2. Pearson correlation coefficient

[0088] The Pearson correlation coefficient (PCC) is a statistic used to measure the strength of the linear relationship between two variables. The Pearson correlation coefficient is calculated as follows:

[0089] ;

[0090] in, express and The Pearson correlation coefficient, represents the data mean of the i-th monitoring element, Represents the mean value of the data collected by sensor A.

[0091] 3. Joint correlation function

[0092] The Pearson correlation coefficient focuses on capturing the linear relationship between variables, can quickly evaluate the strength of linear correlation, and performs well in detecting linear relationships with less noise influence; mutual information can detect a wider range of relationship patterns, including linear and nonlinear associations, and has advantages when the data distribution is complex or there are nonlinear patterns. Combining the two can avoid the limitations of a single method and provide a more comprehensive correlation analysis. The complementarity of the two provides a more robust correlation measure that is suitable for different data characteristics and application scenarios. For the complexity of marine ecological environment monitoring data, the combined use of mutual information and Pearson correlation coefficient can reasonably obtain the correlation ratio between the remaining sensor data in the same group of sensor arrays and the sensor data to be corrected. This method can measure the comprehensive impact of different variables on the target variable while capturing linear and nonlinear relationships; the joint correlation (JC) function is defined as:

[0093] ;

[0094] in, represents the joint correlation function, Expressed as The weight in the joint correlation function, α ∈[0,1] ; express The weight in the joint correlation function, β ∈[0,1] , ;

[0095] The joint correlation function is normalized and percentileized. The normalization formula is as follows:

[0096] ;

[0097] ;

[0098] The percentile formula is as follows:

[0099] ;

[0100] Among them, Total JC represents the sum of the weights of the joint correlations between all monitoring elements and the data of sensor A. represents the normalized result of the joint correlation function value of the i-th monitoring element, Represents the percentile result of the joint correlation function value of the i-th monitoring element.

[0101] 4. Bayesian Optimization

[0102] Bayesian optimization is a global optimization method based on Bayesian inference. When looking for the best weight combination, the global search capability of Bayesian optimization can effectively avoid falling into the local optimum and finally find the global optimal solution. Bayesian optimization does not require pre-setting of complex parameters, and can automatically adapt and optimize the weight combination. In the goal of maximizing the correlation, it can find the optimal and Combination. Combined with the above analysis, the Bayesian optimization objective function formula is:

[0103] ;

[0104] The goal of Bayesian optimization is to maximize the above objective function , find the best weight and . Using Bayesian optimization, initialize the weights and , automatically update the weights according to the initial results and repeat the iterative process, further updating the weights according to the current results in each iteration until the best weight combination is found and .

[0105] The formula for Bayesian optimization and iteration can be expressed as:

[0106] ;

[0107] in, Indicates that the weights are automatically updated and iterated through Bayesian optimization. and The latest result of Next Weights indicates the result of Bayesian optimization after iteration. It indicates the process of updating weights through current data using the Bayesian optimization algorithm, and Optimize indicates the optimization process.

[0108] Repeat the above process until convergence or the termination condition is met, that is, the iterative process is completed, and the final and is the objective function The maximum solution is expressed as:

[0109] ;

[0110] in, and Represents the best Bayesian optimization output and , Indicates When the maximum value is reached, and The value of .

[0111] What you get and Substitution , and substitute each monitoring element into the calculation to obtain the final , and sort them by size to determine the importance of the correlation of various monitoring elements to the data of sensor A. The sorting size is selected, and several monitoring elements with correlation above 70% are selected to construct a new set of monitoring elements C. The selected monitoring elements with high correlation with the sensor A data are , , …, .

[0112] Step 3, such as Figure 3As shown, the data of each monitoring element in the monitoring element set C are subjected to ridge regression modeling and lasso regression modeling for the data of sensor A, the predicted values ​​of the ridge regression model and the lasso regression model are obtained, and the goodness of fit R² test of the ridge regression model and the lasso regression model is performed respectively.

[0113] 1. Ridge Regression

[0114] Ridge regression is a linear regression model with L2 regularization, which is used to deal with multicollinearity problems. It constrains the size of the model coefficients by adding a regularization term to the loss function, thereby preventing the model from overfitting. Select the monitoring element set C to be used for the correction of sensor data Perform ridge regression modeling and obtain the predicted value of the ridge regression model.

[0115] The ridge regression modeling formula is as follows:

[0116] ;

[0117] The predicted values ​​of the ridge regression model are calculated as follows:

[0118] ;

[0119] in, represents the optimal weight vector of the ridge regression model, To express When the minimum value is reached, The value of w =[ w 1 , w 2 ,…, w t ] T represents the weight vector, each Indicates the tth monitoring element The weight vector coefficients in the model, , , …, X t ] T Represents a set of monitoring elements, where X t =[ x t 1 , x t 2 ,…, x tn ] represents the n sample data of the tth monitoring element, Y represents All data of represents the L2 regularization parameter, represents the L2 norm, Represents the predicted value of the ridge regression model.

[0120] 2. Lasso regression

[0121] Lasso regression is a linear regression model with L1 regularization. Unlike ridge regression, Lasso regression can compress some weights to zero, thus playing a role in feature selection. Select the monitoring element set C to correct the sensor data Perform Lasso regression modeling to obtain the predicted value of the Lasso regression model. The Lasso regression modeling formula is as follows:

[0122] ;

[0123] The predicted value of the Lasso regression model is calculated as follows:

[0124] ;

[0125] in, represents the optimal weight vector of the Lasso regression model, represents the L1 regularization parameter, represents the L1 norm, Represents the predicted value of the Lasso regression model.

[0126] 3. Goodness of fit R² test

[0127] Goodness of fit R² is a measure of the degree of match between the model prediction results and the actual data, reflecting the model's explanation ratio of the target variable. The value range of goodness of fit R² is [0, 1]. The closer the value is to 1, the stronger the model's explanatory power for the data. The purpose of performing goodness of fit R² test on the results of ridge regression and Lasso regression models is to evaluate the explanatory power of these two models for the data.

[0128] The calculation formula for the R² test of goodness of fit of the ridge regression model is as follows:

[0129] ;

[0130] ;

[0131] ;

[0132] in, represents the residual sum of squares of ridge regression, represents the total sum of squares of ridge regression, represents the jth data collected by sensor A, express The average value of all data, Represents the goodness-of-fit R² test result of the ridge regression model. represents the predicted value of the ridge regression model;

[0133] The calculation formula for the goodness of fit R² test of the Lasso regression model is as follows:

[0134] ;

[0135] ;

[0136] ;

[0137] in, represents the residual sum of squares of Lasso regression, represents the total sum of squares of Lasso regression, It represents the goodness of fit R² test result of Lasso regression model. Represents the predicted value of the Lasso regression model.

[0138] Step 4: Use the Stacking method to integrate the ridge regression and the lasso regression. Combine the predicted values ​​of the ridge regression model and the lasso regression model to construct the input matrix Z. Select the meta-learner based on the comparison results of the goodness-of-fit R² test, and use the input matrix Z to train the meta-learner to generate the final correction model.

[0139] 1. Input matrix construction

[0140] The Stacking method is an integrated learning method that uses the prediction results of different types of base models as input to a higher-level model to train a new model (meta-learner) to learn more complex patterns to improve the overall prediction performance. The Stacking method allows the combination of different types of models. By integrating multiple base models, it can fully utilize the advantages of each model and better capture the complexity of the data, thereby improving the overall prediction effect. In addition, the Stacking method can better reduce the risk of overfitting when the amount of data is limited or the features are complex.

[0141] The constructed input matrix Z is as follows:

[0142] ;

[0143] in, represents the predicted value of the ridge regression model, Represents the predicted value of the Lasso regression model;

[0144] 2. Meta-learner selection

[0145] According to the goodness of fit R² test results of the ridge regression model The goodness of fit R² test results of the Lasso regression model Select a meta-learner:

[0146] if Higher than , select support vector regression (SVR) as the meta-learner; where, Higher than Indicates that there is a strong linear relationship in the data, that is, ridge regression can better explain the relationship between the input features and the target variable. SVR is a regression model of support vector machine (SVM), which can help capture some complex patterns in the data that linear regression cannot capture, and is good at processing complex nonlinear patterns. Based on the stronger performance of ridge regression, SVR can be used as a meta-learner to further integrate the prediction results from multiple models and optimize the overall prediction performance;

[0147] if Lower than , select gradient boosting regression (GBM) as the meta-learner; where, Lower than It indicates that there is a strong nonlinear relationship in the data, that is, Lasso regression can better explain the relationship between the input features and the target variable. GBM is an integrated learning method and belongs to the Gradient Boosting (GB) paradigm in machine learning. It gradually optimizes the model performance by combining multiple weak learners (usually decision trees) and can handle complex nonlinear relationships. Based on the stronger performance of Lasso regression, GBM can be used as a meta-learner to further optimize the model and capture the complex nonlinear relationships in the data, thereby improving the prediction accuracy.

[0148] 3. Meta-learner training:

[0149] Input the input matrix Z into the meta-learner for training. After training, the final prediction result is obtained. ,calculate and The goodness-of-fit R² test results , according to all Compare the results and select the largest value As the result of the meta-learner training reaching the best state, the final correction model is generated , the test results of goodness of fit The calculation formula is as follows:

[0150] ;

[0151] ;

[0152] ;

[0153] in, express and The residual sum of squares, express The total sum of squares, express and The goodness-of-fit R² test results are: Represents the final prediction result. represents the jth data collected by sensor A, express The average value of all data.

[0154] Step 5, extract the data of the same type as the monitoring element set C in the calibration set, input them into the calibration model, output the final predicted data, and realize the correction of the data detected to have drift phenomenon.

[0155] Extract the same type of data as the monitoring element set C in the calibration set and input it into the calibration model , output the final prediction data , using the prediction results The data after the marked node is replaced, and finally the drift phenomenon in the sensor A data is corrected.

[0156] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-sensor data correction method based on correlation analysis and integrated model, It is characterized in that The steps include: Step 1: obtain various monitoring element data of the sensor array in the monitoring sea area, perform data preprocessing and classified storage, detect the monitoring element data of sensor A with drift phenomenon, mark the drift starting node T, and use the node T as the dividing boundary, build a training set with the data collected before node T, and use the data collected after node T as the calibration set; Step 2: Perform correlation analysis on the data of sensor A in the training set and the data of other sensors to obtain mutual information and Pearson correlation coefficient, and construct a joint correlation function by assigning different weights to the mutual information and Pearson correlation coefficient respectively. Use the Bayesian optimization method to optimize the weights in the joint correlation function, extract the monitoring elements whose correlation with the monitoring elements of sensor A is higher than the set value, and construct a new monitoring element set C; Step 3, perform ridge regression modeling and Lasso regression modeling on the data of sensor A for each monitoring element data in monitoring element set C, obtain the prediction values ​​of ridge regression model and Lasso regression model, and perform R² test on the goodness of fit of ridge regression model and Lasso regression model respectively; Step 4: Use the Stacking method to integrate the ridge regression and the Lasso regression, combine the predicted values ​​of the ridge regression model and the Lasso regression model to construct the input matrix Z, and select the meta-learner based on the comparison results of the goodness of fit R² test, and use the input matrix Z to train the meta-learner to generate the final correction model; Step 5: extract the data of the same type as the monitoring element set C in the calibration set, input them into the calibration model, output the final prediction data, and realize the calibration of the data detected to have drift phenomenon; In step 4, the constructed input matrix Z is as follows: ; in, represents the predicted value of the ridge regression model, Represents the predicted value of the Lasso regression model; According to the goodness of fit R² test results of the ridge regression model The goodness of fit R² test results of the Lasso regression model Choose a meta-learner: If Higher than , select support vector regression as the meta-learner; if Lower than , choose gradient boosting regression as the meta-learner; Input the input matrix Z into the meta-learner for training. After training, the final prediction result is obtained. ,calculate and The goodness-of-fit R² test results , according to all Compare the results and select the largest value As the result of the meta-learner training reaching the best state, the final correction model is generated , the test results of goodness of fit The calculation formula is as follows: ; ; ; in, express and The residual sum of squares, express The total sum of squares, express and The goodness-of-fit R² test results are: Represents the final prediction result. represents the jth data collected by sensor A, express The average value of all data.

2. A multi-sensor data correction method based on correlation analysis and integrated model according to claim 1, It is characterized in that In step 1, data preprocessing includes using the triple standard deviation method to detect and eliminate outliers, using the nearest interpolation method to fill in the missing parts after eliminating outliers and the missing parts in the original data, and using the moving average method to smooth and denoise the data; then, the downsampling method is used to unify the sampling frequency of each data.

3. The multi-sensor data correction method based on correlation analysis and integrated model according to claim 1, It is characterized in that In step 1, after data preprocessing, the data of sensor A and the data of other sensors are used to construct data sets and , represents the jth data collected by sensor A; It represents the j-th data collected by the remaining i-th sensor, that is, the j-th data of the i-th monitoring element.

4. The multi-sensor data correction method based on correlation analysis and integrated model according to claim 3, It is characterized in that In step 2, the mutual information is calculated as follows: ; in, express and The mutual information of , i represents the i-th monitoring element, express and The joint probability distribution of express The marginal probability distribution of express The marginal probability distribution of The Pearson correlation coefficient is calculated as follows: ; in, express and The Pearson correlation coefficient, represents the data mean of the i-th monitoring element, Represents the mean value of the data collected by sensor A.

5. The multi-sensor data correction method based on correlation analysis and integrated model according to claim 3, It is characterized in that In step 2, the joint correlation function is defined as: ; in, represents the joint correlation function, Expressed as The weight in the joint correlation function, ; express The weight in the joint correlation function, , ; The joint correlation function is normalized and percentileized. The normalization formula is as follows: ; ; The percentile formula is as follows: ; Among them, Total JC represents the sum of the weights of the joint correlations between all monitoring elements and the data of sensor A. represents the normalized result of the joint correlation function value of the i-th monitoring element, Represents the percentile result of the joint correlation function value of the i-th monitoring element.

6. The multi-sensor data correction method based on correlation analysis and integrated model according to claim 5, It is characterized in that In step 2, the Bayesian optimization objective function formula is: ; Using Bayesian optimization, initialize weights and , automatically update the weights according to the initial results and repeat the iterative process, further updating the weights according to the current results in each iteration until the best weight combination is found and ; and is the objective function The maximum solution is expressed as: ; in, and Represents the best Bayesian optimization output and , Indicates When the maximum value is reached, and The value of What you get and Substitution , and substitute each monitoring element into the calculation to obtain the final , and sort them by size to determine the importance of the correlation of various monitoring elements to the data of sensor A. According to the sorting size, several monitoring elements with correlation above 70% are selected to construct a new set C of monitoring elements.

7. The multi-sensor data correction method based on correlation analysis and integrated model according to claim 1, It is characterized in that In step 3, the ridge regression modeling formula is as follows: ; The predicted values ​​of the ridge regression model are calculated as follows: ; in, represents the optimal weight vector of the ridge regression model, Indicated as When the minimum value is reached, The value of represents the weight vector, each Indicates the tth monitoring element The weight vector coefficients in the model, , , …, Represents a set of monitoring elements, where represents the n sample data of the tth monitoring element, Y represents All data of represents the L2 regularization parameter, represents the L2 norm, represents the predicted value of the ridge regression model; The Lasso regression modeling formula is as follows: ; The predicted value of the Lasso regression model is calculated as follows: ; in, represents the optimal weight vector of the Lasso regression model, represents the L1 regularization parameter, represents the L1 norm, Represents the predicted value of the Lasso regression model.

8. The multi-sensor data correction method based on correlation analysis and integrated model according to claim 1, It is characterized in that In step 3, the calculation formula for the R² test of the goodness of fit of the ridge regression model is as follows: ; ; ; in, represents the residual sum of squares of ridge regression, represents the total sum of squares of ridge regression, represents the jth data collected by sensor A, express The average value of all data, Represents the goodness-of-fit R² test result of the ridge regression model. represents the predicted value of the ridge regression model; The calculation formula for the goodness of fit R² test of the Lasso regression model is as follows: ; ; ; in, represents the residual sum of squares of Lasso regression, represents the total sum of squares of Lasso regression, It represents the goodness of fit R² test result of Lasso regression model. Represents the predicted value of the Lasso regression model.

9. The multi-sensor data correction method based on correlation analysis and integrated model according to claim 1, It is characterized in that In step 5, extract the data of the same type as the monitoring element set C in the calibration set, input it into the calibration model, and output the final prediction data , using the prediction results The data after the marked node is replaced, and finally the drift phenomenon in the sensor A data is corrected.

Citation Information

Patent Citations

  • Dynamic soft measurement method for NOx emission concentration at outlet of SCR (selective catalytic reduction) system

    CN115575580A

  • Air separation system oxygen extraction rate prediction method and system based on integrated learning

    CN117131970A

  • Hematoma expansion probability prediction method based on NGBoost algorithm

    CN118016308A

  • Travel comfort prediction method based on artificial intelligence and sensing technology

    CN118113995A

  • Electric grid analytics learning machine

    US20200334577A1