A nonlinear signal robust filtering processing method based on echo state network

Abstract

The application discloses a kind of nonlinear signal robust filtering processing methods based on echo state network, it is related to digital signal processing technical field.The application is aimed at the problem that traditional echo state network is susceptible to overfitting and abnormal value interference in nonlinear time series signal processing, a kind of robust filtering method combining L2 regularization and weighted least square is proposed, the objective function containing structural risk term and weighted empirical risk term is constructed, based on residual distribution dynamic adjustment sample weight is given to the sample of standardization residual exceeding preset threshold with extremely low weight to inhibit the distortion of abnormal value to output weight, while the model complexity is controlled by L2 regularization term.The experimental results on Mackey-Glass chaotic signal and sunspot actual signal show that the mean square error of the application compared with traditional method is reduced by about 50%, and it is not sensitive to hyperparameter.The application can be used for industrial sensor signal denoising and nonlinear system identification scenes.

Description

Technical Field

[0001] This invention relates to the fields of digital signal processing and nonlinear time series signal processing, specifically to a robust filtering method based on echo state networks, which is particularly suitable for industrial sensor signal denoising, communication channel equalization, and nonlinear dynamic system identification. Background Technology

[0002] Echo State Networks (ESNs) are a variant of recurrent neural networks specifically designed for time series modeling. The core structure of an ESN consists of three parts: an input layer, a reservoir, and an output layer. The input layer receives external signal input; the reservoir is a dynamic system composed of randomly sparsely connected neurons, with the input weight matrix and recurrent weight matrix fixed after initialization; the output layer maps the reservoir state to the target output through linear regression. Compared to traditional recurrent neural networks, ESNs randomly initialize and fix the reservoir weights, requiring only linear regression training on the output layer. This makes their training speed thousands of times faster than gradient descent, while effectively capturing the dynamic characteristics of nonlinear systems. These characteristics make ESNs promising for applications in chaotic time series signal processing, power load analysis, industrial sensor signal processing, and medical signal monitoring.

[0003] However, traditional ESNs use ordinary least squares to train output weights by default, which has the following six obvious drawbacks in signal processing: First, there is the overfitting problem. Traditional ESN trains output weights based on the principle of minimizing empirical risk. When the pool size is large or the training samples are small, the model is prone to over-reliance on noise or random details in the training data. It performs well on the training set but its generalization ability in the test set or real-world scenarios drops significantly. Second, it has poor numerical stability. When the reserve pool is large, due to data redundancy or noise interference, the condition number of the state matrix may be too large, which will lead to numerical instability in the matrix inversion operation, causing the calculation results of the output weights to fluctuate drastically or the calculation to crash, affecting the reliability of actual deployment. Third, the model complexity lacks constraints. Traditional least squares does not impose any restrictions on the size of the output weights, which may lead to the output layer learning meaningless complex patterns, i.e., the absolute value of the weights is too large, thus masking the true patterns in the data; Fourth, it is sensitive to noise. Traditional ESN treats all training samples equally. When there are high-noise segments or outliers in the data, the model will also learn these outliers, leading to an increase in the overall signal processing bias. Fifth, it lacks dynamic adaptability. Traditional ESN is based on the homoscedasticity assumption (constant error variance), but real signals often exhibit heteroscedasticity (error variance changes with time or input), and traditional methods cannot adapt to the dynamic changes in this statistical characteristic; Sixth, outliers are highly destructive. Real-world sensor signals often contain outliers caused by occasional sensor malfunctions, strong electromagnetic interference, or data transmission errors. A single outlier can severely distort the least squares solution, skewing the output weights and causing the model to completely fail in processing subsequent normal data. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a robust nonlinear signal filtering method based on echo state networks, which can simultaneously control model complexity, suppress outlier interference, and maintain the advantages of efficient training.

[0005] The technical solution adopted in this invention is as follows: A robust filtering method for nonlinear signals based on echo state networks includes the following steps: S1. Obtain the original discrete digital signal sequence output by the sensor or analog-to-digital converter and the corresponding target output signal sequence, and divide the original discrete digital signal sequence into training set and test set; S2. Construct an echo state network, randomly initialize and fix the input weight matrix and the reservoir cyclic weight matrix, and use the training set to drive the echo state network to generate the reservoir state matrix. S3. Initialize the sample weight diagonal matrix for the training set; S4. Construct a regularized weighted least squares objective function based on the target output signal sequence, the reservoir state matrix, and the sample weight diagonal matrix, and introduce a Lagrange multiplier matrix to construct an augmented function, obtaining the closed-form solution formula for the output weight matrix: ; in, To output the weight matrix The transpose of the matrix, , Represents the set of real numbers. , For the number of neurons in the reserve pool, The regularization coefficient is . for 3D identity matrix The effective state matrix of the reserve pool, The length of the signal sequence. The length of the signal sequence during the preheating period. The sample weight diagonal matrix, Output the target signal sequence; Solve the output weight matrix using the closed-form solution formula; S5. Calculate the residual matrix of each sample in the training set based on the current output weight matrix, and update the weight of each sample based on the residual distribution. S6. Construct a new sample weight diagonal matrix using the updated sample weights, and substitute it into the closed-form solution formula in step S4 to resolve the output weight matrix, which is the final output weight matrix. S7. The digital signals from the sampling points in the test set are fed into the echo state network point by point to update the reservoir state vector. Based on the reservoir state vector and the final output weight matrix, the filtered and reconstructed signal is obtained, and the filtered and reconstructed signal is used as the final experimental data.

[0006] In the application of sensor signals, the filtered and reconstructed signals can be used for data fusion analysis, fault detection, and closed-loop control. For example, the filtered and reconstructed signals can be used as experimental data to transmit to a display terminal, trigger an alarm, or serve as input control signals for industrial control actuators.

[0007] Further, step S1 includes: acquiring the original discrete digital signal sequence output by the sensor or analog-to-digital converter. and the corresponding target output signal sequence ,in for The input signal vector at time t, for The target output signal vector at time t. , The length of the signal sequence. The input feature dimension, To determine the output dimension, a linear transformation is applied to the original discrete digital signal sequence and the target output signal sequence. This normalizes all signal sequences to The interval, where, The normalized signal vector, The original signal vector, and These are the minimum and maximum values ​​of the original signal vector, respectively.

[0008] Furthermore, the sensor is an industrial vibration sensor, temperature sensor, or pressure sensor, and the analog-to-digital converter is an analog-to-digital converter in a communication receiver; the original discrete digital signal sequence is a noisy nonlinear time-series signal.

[0009] Further, step S2 includes: S2.1 Randomly initialize and fix the input weight matrix and the cyclic weight matrix of the reserve pool , From uniform distribution Generated by random sampling in the middle, From uniform distribution Generated by random sampling and adjusted spectral radius To satisfy the echo state properties, ensure the reservoir has short-term memory capability, and target spectral radius. The activation function of the reservoir is determined during the offline modeling phase using Bayesian optimization; Let tanh be the hyperbolic tangent function, and let the bias term be... ; S2.2, Feed the samples in the training set into the echo state network point by point, according to the recursive formula. Calculate the state vector of the reservoir at each time step. Set the initial state of the reservoir as a zero vector. Collect the state vectors at all times to construct the complete state matrix of the reservoir. ,in Determined during the offline modeling phase through Bayesian optimization; and They are respectively and The state vector of the reservoir at time t; The dimension is In actual implementation, the bias term This can be incorporated into the input weight matrix by augmenting the input vector with a constant term of 1.0, i.e. The augmented input vector At this point, the input weight matrix... The dimensions are expanded accordingly to The first column corresponds to the weights of the bias term. The two implementations are mathematically equivalent; discarding the first few columns of the signal sequence... The initial state of the reservoir is used as a preheating period to obtain the effective state matrix of the reservoir, which is then used to solve for the output weight matrix to eliminate initial transient effects. It is 10% of the total number of training samples.

[0010] Further, step S3 includes: Initialize the sample weight diagonal matrix for the training set. ,in for An identity matrix, meaning that the initial weights of all training samples are 1.

[0011] Furthermore, the regularized weighted least squares objective function constructed in step S4 is as follows: ; in, Determined during the offline modeling phase through Bayesian optimization; It is the Frobenius norm; For structural risk items ( Regularization (or regularization) prevents excessively large weight values ​​by limiting the Frobenius norm of the output weight matrix, thereby controlling model complexity, suppressing overfitting, improving the condition number of the state matrix, and enhancing numerical stability. This is the weighted empirical risk term, where the sample weights are in a diagonal matrix. Control the contribution of each training sample to the objective function: normal samples contribute a large amount, while outlier samples contribute very little. To find the closed-form solution to this objective function, we first introduce the Lagrange multiplier matrix. Constructing augmented functions: ; in, Represents the augmented function. Represents the matrix trace operation. It is a residual matrix and satisfies the equality constraints. Taking the derivatives of the output weight matrix, residual matrix, and Lagrange multiplier matrix and setting them to zero, we obtain the Karush-Kuhn-Tucker optimality conditions: ; Solving the Karush-Kuhn-Tucker optimality conditions simultaneously yields the closed-form solution of the output weight matrix: ; This closed-ended solution only requires... Matrix inversion has a computational complexity of O(n log n). ;when When the number of samples is much smaller than the number of samples in the signal sequence, the computational efficiency is significantly better than that of traditional recurrent neural networks.

[0012] Further, step S5 includes: S5.1 Calculate the residual matrix of each sample in the training set based on the current output weight matrix. And calculate the residual norm for each sample. ,in for The residual row vector of the sample at time step; S5.2 Calculate the residual norm sequence 25th percentile and the 75th percentile ,according to and Calculate robust standard deviation : ; The denominator 0.6745 represents the theoretical ratio of the interquartile range to the standard deviation under a standard normal distribution. The advantage of this method is that the interquartile range only depends on the middle 50% of the sorted data; regardless of how far outliers deviate, it has a significant impact on... and The impact is extremely limited, therefore It can accurately reflect the fluctuation level of normal samples and will not be inflated by individual outliers; S5.3 Update the weights of each sample according to the following piecewise function. : ; in, and This is a preset threshold constant. Based on the properties of the standard normal distribution, the probability of the absolute value of the standardized residual being greater than 2.5 is approximately 1.24%, a low-probability event; while the probability of it being greater than 3 is approximately 0.27%, even rarer. Therefore, the threshold constant can be set to... and These serve as statistical boundaries to distinguish between "normal samples" and "suspicious samples," and between "suspicious samples" and "outlier samples," respectively. This setting has a clear probabilistic statistical basis and does not rely on subjective experience. The piecewise function means that if the standardized residual of a sample is less than or equal to 2.5, it is considered a normal sample and assigned a full weight of 1; if the standardized residual is between 2.5 and 3, it is considered a suspicious sample and assigned a weight from 1 to 10. -4 The weights decrease linearly; if the standardized residual is greater than 3, it is considered an outlier and assigned a very low weight of 10. -4 This makes it almost uninvolved in subsequent model training.

[0013] Furthermore, in step S6, the updated sample weights are used. The newly constructed sample weight diagonal matrix is ​​as follows: .

[0014] Further, step S7 includes: The digital signal from each sampling point in the test set is sent point by point into the echo state network, according to... Update the reserve pool state vector, then substitute the reserve pool state vector and the final output weight matrix into... Calculate the signal after filtering and reconstruction. .

[0015] Compared with the prior art, the advantages of the present invention are as follows: (1) Effectively suppress overfitting. By constructing a regularized weighted least squares objective function based on the target output signal sequence, the reservoir state matrix, and the sample weight diagonal matrix, and introducing into the objective function... L2. The regularization term (structural risk term) penalizes the Frobenius norm of the output weight matrix, limiting the magnitude of the weights and preventing the model from overly catering to the noisy details in the training data, thereby significantly improving the generalization ability. (2) Effectively resist outlier interference. The residual matrix of each sample is calculated by output weight matrix, and the weight of each sample is updated based on the residual distribution, so that outlier samples hardly participate in subsequent model training, thus the solution of output weight is not affected by outliers; (3) Strong numerical stability. L 2. The introduction of regularization terms improves the condition number of the state matrix, which significantly enhances the numerical stability of matrix inversion. (4) Insensitive to hyperparameters and easy to deploy in engineering. This invention reduces the sensitivity to the selection of hyperparameters such as the spectral radius of the reservoir through an adaptive weighting mechanism; (5) Computationally efficient and suitable for real-time applications. This invention can complete the identification and suppression of outliers with a single weighted update, without the need for iterative optimization, greatly reducing the computational complexity and maintaining the original high-efficiency training advantages of ESN, making it suitable for real-time signal processing scenarios. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention. Figure 2 This refers to the original sunspot sequence data provided in Embodiment 2 of the present invention; Figure 3 This is the signal processing result of a conventional ESN provided in Embodiment 2 of the present invention; Figure 4 Provided for Embodiment 2 of the present invention L 2. Signal processing results of ESN; Figure 5 The signal processing results are provided by the nonlinear signal robust filtering method based on echo state network according to Embodiment 2 of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0020] Example 1: Verification of Nonlinear Filtering Based on Mackey-Glass Chaotic Test Signal (a) Purpose of this embodiment In modeling nonlinear dynamic systems, signals acquired by sensors are often subject to two types of interference: small-amplitude but persistent Gaussian white noise, and large-amplitude but occasional pulse outliers. Traditional echo state networks use ordinary least squares to solve for output weights, assigning the same weights to all samples (including outlier samples). This leads to the output weights being skewed by outliers, severely degrading the model's signal processing accuracy on the test set. Therefore, this invention verifies whether the method can accurately reconstruct the true dynamics of the nonlinear system under the simultaneous presence of these two types of interference, and quantitatively evaluates its resistance to outliers and its effectiveness in suppressing overfitting.

[0021] (II) Generation and Preprocessing of Experimental Data 1. Generate a pure nonlinear test signal The chaotic test signal is generated using the Mackey-Glass time-delay differential equation, and its expression is as follows: ; in, express Chaotic test signal at time, This is the gain coefficient. The attenuation rate, It is a time delay. This puts the system in a chaotic state (the Lyapunov exponent is positive). and The nonlinear excitation intensity and damping coefficient are controlled separately. The fourth-order Runge-Kutta numerical integration method is used to solve the differential equation, with an integration step size of [value missing]. A total of 16,000 data points were generated. To eliminate the influence of initial transient effects on the experimental results, the first 1,000 points were discarded, and the last 15,000 points were used for the experiment. All data were linearly transformed to... The interval transformation formula is as follows: ,in, The normalized signal vector, The original signal vector, and These are the minimum and maximum values ​​of the original signal vector, respectively.

[0022] 2. Divide the dataset and add interference signals. The 15,000 data points were divided as follows: the first 10,500 points (70%) were used as the training set, and the last 4,500 points (30%) were used as the test set. The training set was further divided: the first 8,400 points (80%) were used for model parameter identification (i.e., single-time weighted update calculation of the output weight matrix), and the last 2,100 points (20%) were used as the validation set for hyperparameter optimization.

[0023] Adding mixed normally distributed noise to the training set to simulate outlier interference is done as follows: For each training sample... First, generate a Bernoulli random variable. .like Therefore, the noise in this sample comes from a normal distribution with a mean of 0 and a standard deviation of 0.1. ;like Therefore, the noise in this sample comes from a normal distribution with a mean of 1.0 and a standard deviation of 0.1. (i.e., large outliers with a mean deviation of 10 times the standard deviation). This setting makes approximately 90% of the samples normal Gaussian noise and approximately 10% of the samples large outliers, simulating data distortion caused by occasional sensor failures in real industrial scenarios.

[0024] The test set only includes ordinary Gaussian noise with a mean of 0 and a standard deviation of 0.1, and does not contain outliers. This design conforms to the evaluation conventions of robust regression methods: the model learns to resist outliers during the training phase, and only faces normal noise during the testing phase, thereby verifying whether the model has truly learned the dynamic laws of the signal, rather than merely memorizing the abnormal patterns in the training set.

[0025] (III) Construction and Hyperparameter Setting of Echo State Network 1. Network structure initialization Set the number of neurons in the reserve pool. (This value is determined through Bayesian optimization), input weight matrix From uniform distribution Generated by random sampling in the middle, Represents the set of real numbers, cyclic weight matrix From uniform distribution Generated by random sampling, the spectral radius of the current matrix is ​​calculated. Then press Scale to a target spectral radius of 0.65 (this value was determined through Bayesian optimization), i.e., scaled down. This is to ensure that the echo state property (i.e., the reservoir's sensitivity to initial conditions decays over time) is satisfied. The initial state of the reservoir is set to the zero vector. Reserve pool activation function Choose the hyperbolic tangent function tanh, with the bias term... This is achieved by extending the input vector with a constant term of 1.0, i.e., augmenting the input vector. The corresponding bias weights are contained in the input weight matrix. middle.

[0026] 2. Generate the state matrix of the reserve pool input vector Expand to The first element, 1.0, is the bias term constant. The state vector of the reservoir is calculated using the following recursive formula: , and They are respectively and The state vector of the reservoir at time t; substitute it into but: .

[0027] After calculating all time points, the state vectors at each time point are stacked row by row to form a complete state matrix. , To eliminate initial transient effects, discard the first few seconds of the signal sequence. The initial state of the reserve pool, where 10% of the total number of training samples, i.e. Only retain The effective states are used to solve for the output weights, and the effective state matrix is ​​denoted as . Simultaneously, the corresponding target output signal vectors are stacked row by row to form a target output signal sequence. .

[0028] 3. Determination of hyperparameters Number of neurons in the reserve pool Target spectral radius and regularization parameters The determination is made through Bayesian optimization. The specific process is as follows: In the validation set (the number of samples in the validation set...) Using the mean squared error of signal processing as the optimization objective, Bayesian optimization tools (such as Gaussian process-based optimizers) were used to stochastically explore the hyperparameter space until the objective function converged. Table 1 lists the optimal values ​​for each experimental group: Table 1. Optimal hyperparameters for different datasets

[0029] (iv) Single weighted update optimization process 1. Initialize the sample weight diagonal matrix Assume that the initial weights of all training samples are 1, i.e., the sample weight diagonal matrix... , It is a 7560-dimensional identity matrix.

[0030] 2. Perform the following sub-steps 2.1 Constructing the regularized weighted least squares objective function: ; in, Let be the output weight matrix to be solved. for The transpose of the matrix, regularization coefficient The determination was made during the offline modeling phase through Bayesian optimization. It is the Frobenius norm; the objective function contains two key parts: the first part for L 2. The regularization term limits the sum of squares of the elements in the output weight matrix, preventing excessively large absolute values ​​of the weights and thus suppressing overfitting. Part Two This is the weighted empirical risk term, where the sample weights are in a diagonal matrix. Control the contribution of each sample. The regularization coefficient balances the weights of both.

[0031] To find the closed-form solution to the objective function, we first introduce the Lagrange multiplier matrix. Constructing augmented functions: ; in, Represents the augmented function. Represents the matrix trace operation. It is a residual matrix and satisfies the equality constraints. Taking the derivatives of the output weight matrix, residual matrix, and Lagrange multiplier matrix and setting them to zero, we obtain the Karush-Kuhn-Tucker optimality conditions: ; After solving the simultaneous equations, the closed-form solution of the output weight matrix is ​​obtained: ; It should also be noted that, among them for The identity matrix. The computational complexity of this closed-form solution is O(n). In engineering, Cholesky decomposition can be used to solve problems efficiently.

[0032] 2.2 Calculate the residuals Calculate the residual matrix for each training sample based on the current output weight matrix. : ; in The residual row vector of the sample at time step is , for The target output signal vector at time step [time]. The physical meaning of the residual is the signal processing error of the current model for that sample; the larger its absolute value, the more likely the sample is an outlier. Based on this, the residual norm for each sample is calculated. This serves as the basis for subsequent weight calculations.

[0033] 2.3 Calculate the robust standard deviation of the residuals Traditional methods directly calculate the sample standard deviation of the residuals, but outliers themselves can significantly increase the standard deviation, leading to an excessively high discrimination threshold and the inability to identify outliers. This invention uses a method based on interquartile range: firstly, it calculates the norm sequence of all residuals. 25th percentile and the 75th percentile Then estimate the robust standard deviation. : ; The denominator 0.6745 is the theoretical ratio of the interquartile range to the standard deviation under a standard normal distribution. The advantage of this method is that the interquartile range only depends on the middle 50% of the data after sorting; outliers, regardless of their deviation, do not affect the result. and ,therefore It can accurately reflect the fluctuation level of normal samples.

[0034] 2.4 Update sample weights Based on the standardized residuals of each sample Assign sample weights according to the following formula : ; in, and A preset threshold constant is used. Based on the properties of the standard normal distribution, the probability of the absolute value of the standardized residual being greater than 2.5 is approximately 1.24%, a low-probability event; while the probability of it being greater than 3 is approximately 0.27%, even rarer. Therefore, in this embodiment, the threshold constant is set to... and These serve as statistical boundaries to distinguish between "normal samples" and "suspicious samples," and between "suspicious samples" and "outlier samples," respectively. This setting has a clear probabilistic statistical basis and does not rely on subjective experience. The meaning of this rule is: if If it is determined to be a normal sample, it is assigned a full weight of 1; if If it is determined to be a suspicious sample, it is assigned a linearly decreasing weight; if It was identified as an outlier and assigned a very low weight of 10. -4 This means that it hardly participates in the model training during subsequent single-weighted updates.

[0035] 3. Recalculate the output weight matrix Construct a new diagonal matrix using the updated sample weights. ,use In the closed-form solution Recalculate The recalculated output weight matrix is ​​used as the final output weight matrix.

[0036] 4. Applied to online signal processing The final output weight matrix is ​​applied to online signal processing: the digital signal of each sampling point in the test set is fed point-by-point into the echo state network, expanding the input vector to... According to the recursive formula Update the reserve pool state vector, then press Calculate the signal after filtering and reconstruction At this point, offline training is complete, and the resulting output weight matrix can be used for real-time signal processing in practical engineering applications.

[0037] (v) Configuration of the comparative experiment Set up two sets of comparison models: Compared to Model 1 (traditional ESN): without using regularization or weighting, it directly uses... Solve for the output weights, where This represents the Moore-Penrose pseudo-inverse. In actual numerical calculations, due to the effective state matrix of the reservoir... There may be column rank deficiency or pathological conditions, leading to Singular or near-singular, pseudo-inverse can be used to... The minimum norm least squares solution is obtained in singular cases, thus ensuring the numerical stability and reproducibility of the comparative experiments. This model corresponds to the objective function of this invention. and The degenerate state. It is worth emphasizing that traditional ESNs can only rely on pseudo-inverses for approximate solutions when faced with ill-conditioned matrices, while this invention... The regularization term ensures that the matrix is ​​always strictly positive definite, fundamentally avoiding the ill-conditioned inversion problem, eliminating the need for pseudo-inverse approximation, and resulting in superior numerical accuracy and solution efficiency. Comparison Model 2 ( L 2ESN): Use only L 2. Regularization, without using weighted mechanisms, using Solve for the output weight matrix. This model corresponds to the objective function of this invention. Special circumstances.

[0038] All three models were modeled on the same training set and evaluated on the same test set. All experiments were run independently and repeated 20 times, and the mean and variance of each of the 20 trials were reported. The reported performance metrics include mean squared error (MSE), mean absolute error (MAE), and root mean square error (RMSE), calculated using the following formulas: ; ; ; in, and The first The true value and signal processing value of each sample This represents the number of samples in the test set.

[0039] (vi) Experimental Results and Quantitative Analysis The experimental results on the Mackey-Glass test set are shown in Table 2: Table 2. Comparison of test performance of different models on the Mackey-Glass sequence.

[0040] The above experimental data show that, regarding the problems of overfitting and outlier interference, the MSE of this invention is 0.0657, which is 50.2% lower than that of the traditional ESN. L The 2ESN decreased by 11.0%. This indicates that the introduction of the weighting mechanism is more effective than regularization alone in suppressing the misleading influence of outliers on the model, thereby improving generalization ability. Regarding the numerical stability issue: the MSE variance of this invention is 2.97 × 10⁻⁶. -4 It reduces costs by about an order of magnitude compared to traditional ESNs. L The 2ESN is reduced by approximately one order of magnitude. This demonstrates that the present invention can consistently achieve excellent performance under different random initializations. Regarding computational efficiency: the training time of the present invention is approximately 1.96 seconds, and the testing time is approximately 0.24 seconds. Although the training time is slightly increased compared to the traditional ESN, it still meets the engineering requirements of real-time signal processing.

[0041] Example 2: Non-stationary signal processing based on actual sunspot observation signals (a) Purpose of this embodiment Mackey-Glass is a stationary chaotic test signal whose statistical properties do not change over time. However, sensor signals in real-world engineering often exhibit non-stationary characteristics (mean or variance drifts over time), such as vibration signals during the aging process of industrial equipment and the time-varying characteristics of communication channels. This embodiment uses real monthly average observation data of sunspots, totaling 2820 observations from 1749 to 1983. This data exhibits typical non-stationary characteristics: there is an activity cycle of approximately 11 years, but the cycle amplitude is irregular, the mean drifts over time, and it contains actual observation noise. The purpose of this embodiment is to verify the effectiveness of the method of the present invention on non-stationary real signals and its robustness to hyperparameter selection.

[0042] (II) Data Preprocessing and Network Construction After feature construction from 2820 observation data points, 2688 valid samples were obtained. During feature construction, the first 132 points were discarded due to a lack of complete periodicity information. The data was then divided into a 70% training subset (1881 points) and a 30% testing subset (807 points). The training set was further divided into an 80% training subset (1504 points) and a 20% validation subset (377 points). All data were linearly normalized to... Interval. During ESN training, the first 10% (i.e., 188) of the initial pool states are discarded as a warm-up period to eliminate initial transient effects.

[0043] The optimal hyperparameters determined by Bayesian optimization are shown in Table 1. The initialization of the reservoir, the generation of the state matrix (including the augmented input vector processing method with bias terms), the single weighted update of the weights, the calculation of residuals and weight allocation, the final re-solution of weights, and the reservoir state update and the calculation of the signal after filtering and reconstruction in online signal processing are all exactly the same as in Example 1.

[0044] (III) Ablation Experiment Setup To verify the robustness of this invention to hyperparameter selection, the following ablation experiment was designed: a fixed reservoir size of 500 nodes was established, and three different spectral radii were set ( Under each spectral radius setting, run the traditional ESN, L 2ESN and the method of the present invention. For L Both 2ESN and the method of this invention redetermine the optimal solution under a fixed architecture through Bayesian optimization. The value ensures that each model is compared in its optimal state.

[0045] (iv) Experimental Results and Quantitative Analysis The experimental results on the sunspot test set are shown in Table 3: Table 3. Comparison of signal processing performance of different models on sunspot sequences.

[0046] The above experimental data show that, for processing non-stationary signals: the tested MSE of this invention is 0.0195, which is 47.0% lower than the traditional ESN. L The 2ESN was reduced by 27.2%. This demonstrates that even on non-stationary, long-period real signals, the method of this invention can still effectively suppress abnormal noise interference and accurately track the dynamic changes of the signal.

[0047] The results of the ablation experiment are shown in Table 4: Table 4 Ablation Experiment Results of Different Models

[0048] The above ablation experimental data show that: (1) Regarding the robustness of hyperparameters: When the spectral radius changes from 0.28 to 0.70, the RMSE of this invention remains stable between 0.1383 and 0.1385 (fluctuation less than 0.15%), while the RMSE of the traditional ESN fluctuates wildly between 0.4579 and 0.5429. L The 2ESN fluctuates between 0.1604 and 0.1639. This indicates that the present invention is extremely insensitive to the choice of spectral radius, significantly reducing the difficulty of parameter tuning in practical deployment; (2) Regarding numerical stability: Under the three sets of spectral radii, the variance of the MSE tested by the method of this invention remained at 10. -8 The magnitude is larger, while the MSE variance of traditional ESN is as high as 10. -2 Magnitude L The MSE variance of 2ESN is 10. -6 The magnitude indicates that the present invention can consistently achieve excellent performance under different random initializations, solving the problem of traditional ESNs being sensitive to random initialization and exhibiting large fluctuations in results.

[0049] Raw sunspot sequence data such as Figure 2 As shown, the data represents the monthly average sunspot observations from 1749 to 1983, totaling 2820 data points. It exhibits an activity cycle of approximately 11 years, but the cycle amplitude is irregular, and the mean value drifts over time, exhibiting typical non-stationary characteristics. This data is used to verify the ability of this invention to process real non-stationary signals.

[0050] Figure 3This is a comparison of the signal reconstruction results of the traditional Echo State Network (ESN) on sunspot sequences. The training set output is displayed starting from the end of the warm-up period. It can be seen that the traditional ESN has a decent fitting effect on the training set, but on the test set, especially at the peaks, there is a large deviation, and it is sensitive to abnormal noise, showing obvious overfitting.

[0051] Figure 4 For only L 2. Regularized echo state network ( L A comparison of signal reconstruction results from 2ESN on sunspot sequences. The training set output is displayed starting from the end of the warm-up period. Compared to traditional ESN, L 2ESN suppresses overfitting to some extent, but significant fluctuations still exist near some peaks.

[0052] Figure 5 This is a comparison chart showing the signal reconstruction results of the method of this invention on sunspot sequences. This invention introduces an adaptive weighting mechanism based on residual distribution on top of ridge regression. The training set output in the chart is displayed starting from the end of the preheating period. Compared to... Figure 3 and Figure 4 The signal after filtering and reconstruction by the present invention is closer to the actual observed value on both the training set and the test set. In particular, the signal processing deviation near the peak is significantly reduced, which verifies the effectiveness of the method of the present invention in suppressing overfitting and outlier interference.

[0053] Example 3: Actual Engineering Deployment Scheme (a) Deployment environment and hardware configuration The signal processing method of the present invention can be deployed in the following typical engineering environments: (1) Embedded Deployment: Runs on ARM Cortex-M4 / M7 series microcontrollers, using the CMSIS-DSP library to accelerate matrix operations. When the number of neurons in the reserve pool... At that time, memory usage is approximately RAM, which meets the on-chip storage capacity limitations of most industrial-grade microcontrollers; (2) Edge computing deployment: Run on edge devices such as NVIDIA Jetson Nano or Raspberry Pi, use Python / NumPy to implement floating-point operations, and the end-to-end latency of a single signal processing is less than 50ms; (3) Cloud server deployment: It runs on an industrial cloud platform or local server and can process parallel signal streams from multiple channels and multiple sensors at the same time.

[0054] (ii) Input signal source and output control object Input signal sources include, but are not limited to: industrial vibration sensors (collecting acceleration signals from rotating machinery such as motors, fans, and compressors, with sampling rates typically from 1 kHz to 10 kHz); thermocouples or resistance temperature sensors (collecting temperature signals from industrial furnaces and reactors, with sampling intervals typically from 1 to 10 seconds); current or voltage transformers (collecting current and voltage signals from power systems, with sampling rates from 50 Hz to 1 kHz); and analog-to-digital converters in communication receivers (collecting intermediate frequency or baseband digital signals). The controlled objects of the output signals include: a PLC (Programmable Logic Controller), which uses the filtered and reconstructed signal as an analog input (4-20mA or 0-10V) to trigger alarms (such as vibration over-limit shutdown) or PID closed-loop control (such as precise temperature regulation); a host computer human-machine interface (HMI), which displays the filtering results on the monitoring screen for operators' reference; and an actuator drive module, which directly drives valves, frequency converters, and other actuators for real-time adjustment.

[0055] (III) Complete Operation Process for Online Deployment 1. Offline initialization phase In an offline environment, a single weighted update training of the output weight matrix is ​​completed using historical data according to the method in Example 1 or 2. The trained weight matrix and network parameters are then combined. , , It is embedded in the non-volatile memory area (such as Flash or EEPROM) of the target device.

[0056] 2. Online data collection and status update phase After the device is powered on, it will press [the command] whenever a new sampling point digital signal arrives. Update the reserve pool state vector; then press Calculate the signal after filtering and reconstruction.

[0057] 3. Output and Application Stage The filtered and reconstructed signal is converted into an analog voltage / current signal (via a digital-to-analog converter) or packaged and sent to a PLC or HMI according to industrial communication protocols (Modbus RTU / TCP, CANopen, PROFINET, etc.) to achieve real-time monitoring or closed-loop control of field equipment.

[0058] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A robust filtering method for nonlinear signals based on echo state networks, characterized in that, Includes the following steps: S1. Obtain the original discrete digital signal sequence output by the sensor or analog-to-digital converter and the corresponding target output signal sequence, and divide the original discrete digital signal sequence into a training set and a test set; the sensor is an industrial vibration sensor, temperature sensor or pressure sensor, and the analog-to-digital converter is an analog-to-digital converter in a communication receiver; the original discrete digital signal sequence is a noisy nonlinear time-series signal. S2. Construct an echo state network, randomly initialize and fix the input weight matrix and the reservoir cyclic weight matrix, and use the training set to drive the echo state network to generate the reservoir state matrix. S3. Initialize the sample weight diagonal matrix for the training set; S4. Construct a regularized weighted least squares objective function based on the target output signal sequence, the reservoir state matrix, and the sample weight diagonal matrix. After introducing the Lagrange multiplier matrix to construct an augmented function, solve for the closed-form solution to obtain the closed-form solution formula for the output weight matrix: ； in, To output the weight matrix The transpose of the matrix, , Represents the set of real numbers. , For the number of neurons in the reserve pool, The regularization coefficient is . for 3D identity matrix The effective state matrix of the reserve pool, The length of the signal sequence. The length of the signal sequence during the preheating period. The sample weight diagonal matrix, Output the target signal sequence; Solve the output weight matrix using the closed-form solution formula; S5. Calculate the residual matrix of each sample in the training set based on the current output weight matrix, and update the weight of each sample based on the residual distribution. S6. Construct a new sample weight diagonal matrix using the updated sample weights, and substitute it into the closed-form solution formula in step S4 to resolve the output weight matrix, which is the final output weight matrix. S7. The digital signals from the sampling points in the test set are fed into the echo state network point by point to update the reservoir state vector. Based on the reservoir state vector and the final output weight matrix, the filtered and reconstructed signal is obtained, and the filtered and reconstructed signal is used as the final experimental data.

2. The robust filtering method for nonlinear signals based on echo state networks according to claim 1, characterized in that, Step S1 includes: Acquire the raw discrete digital signal sequence output from the sensor or analog-to-digital converter. and the corresponding target output signal sequence ,in for The input signal vector at time t, for The target output signal vector at time t. , The input feature dimension, To determine the output dimension, a linear transformation is applied to the original discrete digital signal sequence and the target output signal sequence. This normalizes all signal sequences to The interval, where, The normalized signal vector, The original signal vector, and These are the minimum and maximum values ​​of the original signal vector, respectively.

3. The robust filtering method for nonlinear signals based on echo state networks according to claim 2, characterized in that, Step S2 includes: S2.1 Randomly initialize and fix the input weight matrix and the cyclic weight matrix of the reserve pool Adjust the spectral radius of the cyclic weight matrix of the reserve pool. Target spectral radius Determined during the offline modeling phase through Bayesian optimization; S2.2, Feed the samples in the training set into the echo state network point by point, according to the recursive formula. Calculate the state vector of the reservoir at each time point. and They are respectively and The state vector of the reservoir at time t. It is the hyperbolic tangent function. The bias term is used; the initial state of the reservoir is set as the zero vector, and the state vectors at all times are collected to form the complete state matrix of the reservoir. ; Discard the first part of the signal sequence The initial state of the reservoir is used as the preheating period to obtain the effective state matrix of the reservoir, where... It is 10% of the total number of training samples.

4. The robust filtering method for nonlinear signals based on echo state networks according to claim 2, characterized in that, Step S3 includes: Initialize the sample weight diagonal matrix for the training set. ,in for 3D identity matrix.

5. The robust filtering method for nonlinear signals based on echo state networks according to claim 1, characterized in that, In step S4, the constructed regularized weighted least squares objective function is: ； in, It is the Frobenius norm; Introducing Lagrange multiplier matrix Constructing augmented functions: ； in, Represents the augmented function. Represents the matrix trace operation. It is a residual matrix and satisfies the equality constraints. Taking the derivatives of the output weight matrix, residual matrix, and Lagrange multiplier matrix and setting them to zero, we obtain the Karush-Kuhn-Tucker optimality conditions: ； After solving the Karush-Kuhn-Tucker optimality conditions simultaneously, a closed-form solution formula for the output weight matrix is ​​obtained.

6. The robust filtering method for nonlinear signals based on echo state networks according to claim 5, characterized in that, Step S5 includes: S5.1 Calculate the residual matrix of each sample in the training set based on the current output weight matrix. And calculate the residual norm for each sample. ,in for The residual row vector of the sample at time step; S5.2 Calculate the residual norm sequence 25th percentile and the 75th percentile ,according to and Calculate robust standard deviation ; S5.3 Update the weights of each sample according to the following piecewise function. : ； in, and The preset threshold constant is taken as follows: , .

7. The robust filtering method for nonlinear signals based on echo state networks according to claim 6, characterized in that, In step S6, the updated sample weights are used. The constructed sample weight diagonal matrix is ​​as follows: .

8. The robust filtering method for nonlinear signals based on echo state networks according to claim 3, characterized in that, Step S7 includes: The digital signal from each sampling point in the test set is sent point by point into the echo state network, according to... Update the reserve pool state vector, then substitute the reserve pool state vector and the final output weight matrix into... The filtered and reconstructed signal is calculated. .

Images