A dynamic modeling method and system based on regularized recursive random configuration network

Abstract

The present invention belongs to the field of dynamic modeling technology, and discloses a dynamic modeling method and system based on a regularized recursive random configuration network, including: S1, data preprocessing; S2, regularized recursive random configuration network initialization; S3, random configuration weights and biases, and a new method for constructing candidate neurons in a reserve pool; S4, supervised selection of candidate neurons according to regularized inequality constraints; S5, determination of the optimal candidate neuron, and addition to the reserve pool; S6, calculation of network output weights in combination with target output and reserve pool output; S7, based on a projection algorithm, online updating of trained output weights according to real-time data, and calculation of real-time output. The present invention adopts a projection algorithm to update the network output weights online, responds to the dynamic changes of the system, can enhance the stability of model performance while ensuring prediction accuracy, and reduce the overall calculation amount of the algorithm, and is suitable for application scenarios with high real-time requirements.

Description

Technical Field

[0001] The present invention belongs to the technical field of dynamic modeling, and in particular relates to a dynamic modeling method and system based on a regularized recursive random configuration network. Background Art

[0002] The nonlinear dynamic system in the industrial process is a complex system in which the data distribution changes with time. It is usually impossible to obtain complete and deterministic information, and it is difficult to establish an accurate analytical model. For such a complex dynamic system, analysis is often performed through the collected time series. However, in the actual process, it is difficult for us to capture all the order delay information of the variables involved, and the input order at different times is also constantly changing. There are huge challenges in achieving accurate modeling of such nonlinear dynamic systems with uncertain orders.

[0003] Through the above analysis, the problems and defects of the prior art are as follows:

[0004] (1) When the system order is unknown, the traditional method first needs to perform order identification to obtain meaningful order information and then establish an appropriate prediction model. However, this process increases the complexity of the algorithm and reduces the training speed of the model, which is not suitable for industrial processes with high real-time requirements.

[0005] (2) Recurrent neural networks can store some historical information because of the feedback connection between their neurons. They do not require order identification and can be used to solve the modeling problem of nonlinear dynamic systems with uncertain orders. However, during the training process, traditional recurrent neural networks need to adjust the network connection weights and biases according to the error gradient descent method. Therefore, the training process is computationally intensive and time-consuming. In addition, it is easy to fall into local minima, produce gradient vanishing or explosion problems, and achieve low efficiency.

[0006] (3) There are also feedback connections between neurons in the echo state network. As a random model, the input weights and reservoir connection weights of the echo state network are randomly assigned and will not change once given. The output weights are calculated by the least squares method, avoiding the shortcomings of gradient algorithms. However, the echo state network is extremely sensitive to the choice of weight parameters and cannot adaptively adjust the range of weights and biases according to input data. Moreover, there is no theoretical basis for the determination of its network structure, and the global approximation properties of the model cannot be guaranteed, which in turn affects the prediction accuracy of the model.

[0007] (4) The recursive random configuration network also uses random connections, introduces inequality constraints, and adds reserve pool neurons in a supervised manner, which can effectively avoid the above problems. However, the recursive random configuration network is prone to overfitting, that is, the degree of fit to the training data is very high and the generalization ability is very poor. In addition, real-time process data is often accompanied by noise, and the overfitting model has a low prediction accuracy for such data.

[0008] In view of the problems and defects of the above-mentioned prior art, the industry faces the following technical problems:

[0009] 1. Reliability and stability issues: Traditional recursive neural networks are prone to falling into local minima or experiencing gradient vanishing or explosion, which will affect the stability and reliability of the model and is unacceptable for industrial systems with strict requirements for continuous operation and safety.

[0010] 2. Low model accuracy: Although the echo state network solves some problems of recursive neural networks, the model accuracy is insufficient due to the sensitive selection of weight parameters and poor adaptability, which is a major problem in quality control and resource optimization.

[0011] 3. Poor generalization ability: Although the recursive random configuration network has advantages in some aspects, it is prone to overfitting, especially in the presence of data noise, its prediction accuracy and generalization ability will be greatly reduced.

[0012] 4. Noise sensitivity: In industrial environments, data is often noisy. Existing technologies are often sensitive to noise, which reduces the robustness of the model, leading to inaccurate predictions and wrong decisions.

[0013] 5. Difficulty in implementing theoretical guarantees: It is difficult for echo state networks and recursive random configuration networks to provide theoretical guarantees of the global approximation or convergence of the model, which increases the risk in industrial applications because it is difficult to predict how the model will perform under unknown data or sudden situations.

[0014] In summary, these problems limit the wide application of existing technologies in the fields of industrial automation, intelligent manufacturing, resource optimization, etc. Solving these problems requires the development of more efficient, stable, accurate, and generalizable dynamic modeling methods. Summary of the invention

[0015] In view of the problems existing in the prior art, the present invention provides a dynamic modeling method and system based on a regularized recursive random configuration network. The purpose is to solve the modeling problem of dynamic systems with uncertain orders and alleviate the influence of model overfitting. The regularized recursive random configuration network does not need to perform order identification, only inputs the data at the current moment, uses the characteristics of the reserve pool that can store historical information for modeling, and adds regularization constraints to the output weights to alleviate the influence of overfitting. In the process of building the reserve pool, suitable neurons are found as candidate neurons through the inequality constraints of the regularized random configuration algorithm. Different from the original random connection of the reserve pool, the regularized recursive random configuration network will redistribute the connection weights of the newly added neurons, only retaining the network connection weights of the newly added neurons to themselves and the original neurons, and the network connection weights of the original neurons to the newly added neurons are set to zero, that is, no connection. After determining the newly added neurons, the regularization constraints on the output weights are added to the loss function, and the output weights are calculated using the least squares method. Finally, the maximum allowable number of neurons in the reserve pool and the maximum allowable output error are used to determine whether the network structure is completed.

[0016] The present invention is implemented as follows: a dynamic modeling method based on a regularized recursive random configuration network improves the initialization and training process of the neural network through regularization technology to achieve efficient simulation and prediction of dynamic systems. The method first performs data preprocessing, then initializes a random configuration network that has been regularized, and then uses a novel method to randomly configure weights and biases, and constructs candidate neurons in the reserve pool. Next, the candidate neurons are supervisedly selected using regularized inequality constraints, and the optimal neurons are added to the reserve pool. Then, the output weights of the network are obtained by calculation by combining the target output and the reserve pool output. Finally, a projection algorithm is used to update the trained output weights online according to real-time data, and the real-time output is calculated, thereby improving the accuracy and generalization ability of the model for predicting time series data.

[0017] Further, including:

[0018] S1, data preprocessing;

[0019] S2, regularized recursive random configuration network initialization;

[0020] S3, randomly configures weights and biases, and uses a new method to construct candidate neurons in the reserve pool;

[0021] S4, supervised selection of candidate neurons based on regularized inequality constraints;

[0022] S5, determine the best candidate neurons and add them to the reserve pool;

[0023] S6, calculate the network output weight by combining the target output and the reserve pool output;

[0024] S7, based on the projection algorithm, updates the trained output weights online according to the real-time data and calculates the real-time output.

[0025] Furthermore, S1 specifically includes: given a set of time series data, input samples: {u(1),u(2),…,u(n max )}={(y(1),u(1)),(y(2),u(2)),…,(y(n max ),u(n max ))}, target output: T = {t(2), t(3), …, t(n max +1)}, where y and u are the system output and the controlled input respectively, and the output y(n+1) at the next moment is predicted only by the current moment input (y(n), u(n)), n max is the number of samples; set the maximum allowable number of neurons in the reserve pool N max , the maximum expected output error allowable value ε, the maximum number of candidate neurons generated G max , input weight distribution parameter γ={λ min ,λ min +Δλ,...,λ max}, regularization coefficient C.

[0026] Furthermore, S2 specifically includes: initializing the output error vector e0=T, the model output error scaling factor 0<r<1, assuming that there is already a reserve pool containing N neurons, and the network model at this time is:

[0027] x(n)=g(W in,N u(n)+W r,N x(n-1)+b N )

[0028]

[0029]

[0030] Where W in,N , W r,N and W out is the input weight, the reservoir connection weight and the output weight matrix, b N is the bias matrix, x(n) is the reservoir state vector at time n, X = [x(1), x(2), ... x(n)] is the reservoir state matrix, and g is the activation function.

[0031] Further, S3 specifically includes: from γ={λ min ,λ min +Δλ,...,λ max}, randomly select the input weight Win,N+1 , the connection weight of the reserve pool is W r,N+1 , bias b N+1 , substitute the activation function x(n)=g(W in,N+1 u(n)+W r,N+1 x(n-1)+b N+1 ) to get G max Group candidate neurons

[0032] Further, S4 specifically includes: max Substitute the candidate neurons into the inequality constraints of the regularized random configuration algorithm:

[0033]

[0034] where m is the output dimension (a sequence of non-negative real numbers {μ N+1}satisfy And μ N+1 ≤(1-r)), from which candidate neurons that satisfy the inequality constraints are selected.

[0035] Further, determining the candidate neurons in S5 specifically includes: defining a set of variables: Select the equations that satisfy the inequality constraints and make ξ N+1 The largest candidate neuron is used as the optimal reserve pool candidate neuron.

[0036] Furthermore, the addition to the reserve pool in S5 specifically includes: adding the best candidate neuron in the reserve pool to the network model, and directly calculating the output weight W of the neural network model according to the target output out , and calculate the output root mean square error of the neural network model, use the root mean square error as the loss function, and update the output error of the model e0: = e N+1 And the number of neurons in the reserve pool N=N+1.

[0037] Further, S6 specifically includes: if the output root mean square error ||e0||2 of the current network model is greater than the maximum expected output error allowable value ε, and the number of neurons in the reserve pool N is less than the maximum allowable number of neurons in the reserve pool N max , then repeat S3 to S5; if the output root mean square error of the current network model ||e0||2 is less than the maximum expected output error tolerance ε, or the number of neurons in the reserve pool N is greater than the maximum allowable number of neurons in the reserve pool N max , the training ends and a network model that meets the constraints of random configuration theory is obtained.

[0038] Further, S7 specifically includes: based on the projection algorithm, updating the trained output weights online according to the real-time data, and calculating the real-time output:

[0039]

[0040] y(n)=W out (n)g(n-1)

[0041] Where 0<a<1, c>0 are two constants, g(n) is the output of the reserve pool at time n, and e(n) is the output error before correction.

[0042] Another object of the present invention is to provide a dynamic modeling system based on a regularized recursive random configuration network using the dynamic modeling method based on a regularized recursive random configuration network, comprising:

[0043] A preprocessing module, used for data preprocessing;

[0044] Initialization module, used to initialize the regularized recursive random configuration network;

[0045] Random configuration module, used to randomly configure weights and biases, and adopt a new method to construct candidate neurons in the reserve pool;

[0046] Neuron selection module, used to select candidate neurons in a supervised manner according to regularized inequality constraints;

[0047] The optimal candidate determination module is used to determine the optimal candidate neuron and add it to the reserve pool;

[0048] Output weight calculation module, used to calculate the network output weight by combining the target output and the reserve pool output;

[0049] The online update module is used to update the trained output weights online based on the projection algorithm and real-time data, and calculate the real-time output.

[0050] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0051] First, the present invention provides a dynamic modeling method and system based on a regularized recursive random configuration network, which does not require order identification and uses historical information stored in a reserve pool for modeling; compared with recursive neural networks, a supervision mechanism is introduced in the incremental construction process to randomly configure the input weights and biases of the reserve pool neurons, and there is no need for iterative updating through back propagation, thereby avoiding the problems of falling into local optimality, gradient disappearance or explosion caused by the gradient descent method for updating network parameters; compared with the echo state network, the weight parameters of this configuration method are variable, and the weights and biases can be adaptively adjusted according to the input data, and the network structure is determined during the configuration process, which can theoretically guarantee the global approximation properties of the model and achieve zero-error approximation of the training data; compared with the recursive random configuration network, the regularized version can effectively alleviate the overfitting phenomenon and has advantages in model prediction accuracy and performance stability.

[0052] Second, the present invention uses a projection algorithm to update the network output weights online to respond to the dynamic changes of the system. Combined with the network framework designed by the present invention, it can enhance the stability of model performance while ensuring the prediction accuracy, reduce the overall calculation amount of the algorithm, and is suitable for application scenarios with high real-time requirements. It has good application prospects in the fields of industrial artificial intelligence, smart medical care, smart transportation, and unmanned driving.

[0053] Third, for dynamic data modeling problems, most people stay at proving the existence of the global approximation ability of the model, but do not start from the perspective of constructivity. The present invention provides a dynamic modeling method and system for a regularized recursive random configuration network, which incrementally constructs a dynamic model with global approximation ability. Moreover, the model does not require order identification, does not require gradient descent to update weights and biases, can adaptively adjust the range of weights and biases, introduces regularization constraints, effectively alleviates the overfitting phenomenon of the model, and is suitable for dynamic modeling problems with uncertain orders.

[0054] Fourth, the dynamic modeling method based on regularized recursive random configuration network has the following significant technical advances in industry:

[0055] 1. Enhance model stability and reliability: Regularized inequality constraints are used for supervised selection of candidate neurons. This step helps prevent the model from falling into local minima or causing gradient vanishing and gradient exploding problems.

[0056] 2. Improve model accuracy: The data-driven weight and bias selection range can adaptively determine the corresponding model parameters according to the input data, so that the prediction accuracy of the model is significantly improved.

[0057] 3. Enhance generalization ability and robustness: The use of regularization technology and online updated projection algorithm can not only reduce the risk of overfitting, but also increase the robustness of the model to real-time data and noise.

[0058] 4. Theoretical Guarantee: Through regularization and constraints, this method provides more theoretical guarantees, such as the global approximation and stability of the network, which are very important in industrial applications.

[0059] 5. Adaptive capability: By updating the output weights online, the model can better adapt to time-varying and non-steady-state processes, which is very important in industrial control systems.

[0060] 6. Reduce algorithm complexity: It eliminates the order identification process in traditional methods, simplifies model construction, and reduces computational complexity.

[0061] 7. Improve system efficiency and resource optimization: Since the real-time, accuracy and robustness of the model have been improved, this helps to utilize resources more effectively, improve production efficiency and reduce costs.

[0062] The method provided by the invention comprehensively considers speed, accuracy, stability and robustness, and has high industrial application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0064] Figure 1 It is a flow chart of a dynamic modeling method based on a regularized recursive random configuration network provided by an embodiment of the present invention;

[0065] Figure 2 It is a schematic diagram of the structure of a dynamic modeling system based on a regularized recursive random configuration network provided by an embodiment of the present invention;

[0066] Figure 3 This is the fitting effect diagram of RRSCN to MG dynamic data;

[0067] Figure 4 It is the fitting effect diagram of each method on the nonlinear system identification data;

[0068] Figure 5 This is a diagram showing the fitting effects of various methods on the butane concentration data at the bottom of the debutanizer tower. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0070] A specific industrial application embodiment provided by the present invention is:

[0071] First, data preprocessing is performed to collect historical operating data of the debutanizer, including seven auxiliary variables: tower top temperature, tower top pressure, tower top reflux rate, tower top product outflow rate, sixth-layer tower plate temperature, tower bottom temperature 1 and tower bottom temperature 2, as well as the corresponding tower bottom butane concentration data.

[0072] Next, the collected data is cleaned, standardized or normalized as necessary to eliminate dimensional effects and potential data biases, ensuring that the impact of different variables on the model is at the same level, thereby improving the model training effect.

[0073] Then, a dynamic modeling system based on a regularized recursive random configuration network (RRSCN) is initialized, and appropriate initial parameters are selected, such as the size of the reservoir, the distribution of input weights and connection weights, and the expected output error of the model.

[0074] The network model is trained using the collected historical data. In each iteration, the weights and biases are randomly configured, the appropriate candidate neurons are selected in a supervised manner according to the regularized inequality constraints, and the network output weights are calculated. By continuously adjusting and optimizing these parameters, the prediction error of the model is reduced.

[0075] Once the model training is completed, it can be used to predict real-time data. By inputting seven variables such as real-time tower top temperature and tower top pressure, the model will output a predicted butane concentration at the bottom of the tower.

[0076] Finally, the projection algorithm is applied to update the output weights of the model online to ensure that the model can adapt to dynamic changes in the process and ensure the accuracy and stability of the prediction.

[0077] Through such an embodiment, the RRSCN model can effectively perform soft measurement of the butane concentration at the bottom of the debutanizer, helping operators to better control and optimize the production process.

[0078] The present invention mainly improves the following problems and defects of the prior art and achieves significant technical progress:

[0079] Insufficient generalization ability of the model: Traditional dynamic system modeling methods may perform poorly in terms of generalization ability, especially when dealing with complex, nonlinear dynamic data. The present invention improves the generalization ability of the model for complex data by introducing a regularized recursive random configuration network.

[0080] Complexity and inefficiency of model training: The existing technology may face the problems of high complexity and low efficiency in the model training process. The present invention reduces the complexity of model training and improves efficiency by adopting a novel method to randomly configure weights and biases and select neurons by regularizing inequality constraints.

[0081] Insufficient adaptability to real-time data: In a dynamic environment, existing models may not be able to effectively adapt to real-time data changes. The present invention uses a projection algorithm to update the output weights online, thereby improving the adaptability of the model to real-time data.

[0082] The technical effects and significant technical progress brought about by the present invention in solving the problems of the prior art are as follows:

[0083] Improve the generalization ability of the model: Regularization processing and optimal selection of the reserve pool enable the model to better adapt and perform in complex dynamic data environments, thereby improving the generalization ability.

[0084] Simplify model training and improve efficiency: By simplifying the configuration process of weights and biases and selecting the optimal neurons using a supervised method, the present invention simplifies the model training process and improves training efficiency.

[0085] Improve real-time data processing capabilities: Use projection algorithms to update the model online, so that it can quickly adapt to changes in real-time data, improving the real-time processing capabilities of the model.

[0086] Optimizing model performance evaluation and adjustment: By combining the target output and the reserve pool output to calculate the network output weights, and using the root mean square error for performance evaluation, the present invention provides a more accurate model performance evaluation and adjustment method.

[0087] Applicable to a variety of dynamic system scenarios: Due to its high generalization ability and real-time data processing capabilities, the present invention is applicable to a variety of complex and dynamic system modeling scenarios, improving its practicality and universality.

[0088] In view of the problems existing in the prior art, the present invention provides a dynamic modeling method and system based on a regularized recursive random configuration network. The present invention is described in detail below in conjunction with the accompanying drawings.

[0089] like Figure 1 As shown, the specific steps of the dynamic modeling method based on the regularized recursive random configuration network provided by the embodiment of the present invention are as follows:

[0090] Step 1: Data preprocessing.

[0091] Step 2: Initialization of the regularized recursive random configuration network.

[0092] Set the maximum number of neurons in the reserve pool N max , the initial number of neurons N = 0, the maximum expected output error allowable value ε, the distribution of input weights γ = {λ min ,λ min +Δλ,...,λ max}, the maximum number of candidate neurons generated G max , regularization coefficient C, model output error initialization: e0=T.

[0093] Step 3: From γ = {λ min ,λ min +Δλ,...,λ max}, randomly select the input weight W in,N+1 , the connection weight of the reserve pool is W r,N+1 and bias b N+1 , substitute the activation function x(n)=g(W in,N+1 u(n)+W r,N+1 x(n-1)+b N+1 ) to get G max Group candidate neurons

[0094] Step 4. Settings G max Substitute the candidate neurons into the inequality constraints of the regularized random configuration algorithm:

[0095]

[0096] Select candidate neurons that satisfy the inequality constraints.

[0097] Step 5. Define a set of variables: Substitute the candidate neurons that satisfy the inequality constraints into Select ξ N+1 The largest candidate neuron, retain W in,N+1 , W r,N+1 and b N+1 , proceed to the next step; otherwise, λ=λ+Δλ, when λ<λ max , return to step 3 to continue. If the above two conditions are not met, let τ∈(0,1-r), r=r+τ, λ=λ min , and then return to step 3 to continue. Add the best candidate neuron to the reserve pool to obtain the latest connection weights and biases and

[0098] Step 6: Update the neural network model, and use the least squares method to obtain the output weight of the model through the reserve pool output and the target output. Finally update the model output error e N =e N+1 And the number of neurons in the reserve pool N=N+1.

[0099] Step 7: When the root mean square error of the model output ||e0||2>ε, and N<N max , then return to step 3 to execute; otherwise, the neural network model training is completed.

[0100] Step 8: Based on the projection algorithm, the trained output weights are calculated according to the real-time data. Update online and calculate real-time output to analyze the generalization performance of the model.

[0101] like Figure 2 As shown, the dynamic modeling system based on the regularized recursive random configuration network provided by the embodiment of the present invention includes:

[0102] A preprocessing module, used for data preprocessing;

[0103] Initialization module, used to initialize the regularized recursive random configuration network;

[0104] Random configuration module, used to randomly configure weights and biases, and adopt a new method to construct candidate neurons in the reserve pool;

[0105] Neuron selection module, used to select candidate neurons in a supervised manner according to regularized inequality constraints;

[0106] The optimal candidate determination module is used to determine the optimal candidate neuron and add it to the reserve pool;

[0107] Output weight calculation module, used to calculate the network output weight by combining the target output and the reserve pool output;

[0108] The online update module is used to update the trained output weights online based on the projection algorithm and real-time data, and calculate the real-time output.

[0109] Preferably, the dynamic modeling method based on the regularized recursive random configuration network provided in the embodiment of the present invention includes:

[0110] S1, data preprocessing;

[0111] S2, regularized recursive random configuration network initialization;

[0112] S3, randomly configures weights and biases, and uses a new method to construct candidate neurons in the reserve pool;

[0113] S4, supervised selection of candidate neurons based on regularized inequality constraints;

[0114] S5, determine the best candidate neurons and add them to the reserve pool;

[0115] S6, calculate the network output weight by combining the target output and the reserve pool output;

[0116] S7, based on the projection algorithm, updates the trained output weights online according to the real-time data and calculates the real-time output.

[0117] S1 specifically includes: given a set of time series data, input samples: {u(1),u(2),...,u(n max )}={(y(1),u(1)),(y(2),u(2)),...,(y(n max ),u(n max ))}, target output: T = {t(2), t(3), ..., t(n max +1)}, where y and u are the system output and the controlled input respectively. No order identification is required. The output y(n+1) at the next moment is predicted only by the current moment input (y(n), u(n)). max is the number of samples; set the maximum allowable number of neurons in the reserve pool N max , the maximum expected output error allowable value ε, the maximum number of candidate neurons generated G max , input weight distribution parameter γ={λ min ,λ min +Δλ,...,λ max}, regularization coefficient C.

[0118] S2 specifically includes: initializing the output error vector e0=T, the model output error scaling factor 0<r<1, assuming that there is already a reserve pool containing N neurons, the network model at this time is:

[0119] x(n)=g(W in,N u(n)+W r,N x(n-1)+b N )

[0120]

[0121]

[0122] Where W in,N , W r,N and W out is the input weight, the pool connection weight and the output weight, b N is the bias, x(n) is the state of the reservoir at time n, X = [x(1), x(2), ... x(n)] is the reservoir state matrix, and g is the activation function.

[0123] S3 specifically includes: from γ={λ min ,λ min +Δλ,...,λ max}, randomly select the input weight W in,N+1 , the connection weight of the reserve pool is W r,N+1 , bias b N+1 , substitute the activation function x(n)=g(W in,N+1 u(n)+W r,N+1 x(n-1)+b N+1 ) to get G max Group candidate neurons

[0124] Further, S4 specifically includes: max Substitute the candidate neurons into the inequality constraints of the regularized random configuration algorithm:

[0125]

[0126] where m is the output dimension (a sequence of non-negative real numbers {μ N+1}satisfy And μ N+1 ≤(1-r)), from which candidate neurons that satisfy the inequality constraints are selected.

[0127] Determining candidate neurons in S5 specifically includes: defining a set of variables: Select the equations that satisfy the inequality constraints and make ξ N+1 The largest candidate neuron is used as the optimal reserve pool candidate neuron.

[0128] In S5, adding the optimal candidate neuron in the reserve pool to the reserve pool specifically includes: adding the optimal candidate neuron in the reserve pool to the network model, and directly calculating the output weight W of the neural network model according to the target output out , and calculate the output root mean square error of the neural network model, use the root mean square error as the loss function, and update the output error of the model e0: = e N+1 And the number of neurons in the reserve pool N=N+1.

[0129] S6 specifically includes: if the output root mean square error ||e0||2 of the current network model is greater than the maximum expected output error tolerance ε, and the number of neurons in the reserve pool N is less than the maximum allowable number of neurons in the reserve pool N max , then repeat S3 to S5; if the output root mean square error of the current network model ||e0||2 is less than the maximum expected output error tolerance ε, or the number of neurons in the reserve pool N is greater than the maximum allowable number of neurons in the reserve pool N max , the training ends and a network model that meets the constraints of random configuration theory is obtained.

[0130] S7 specifically includes: based on the projection algorithm, online updating of the trained output weights according to real-time data, and calculation of real-time output:

[0131]

[0132] y(n)=W out (n)g(n-1)

[0133] Where 0<a<1, c>0 are two constants, g(n) is the output of the reserve pool at time n, and e(n) is the output error before correction.

[0134] S3 to S5 are the key points of the present invention. S3 links the input data with the weight parameters and can adaptively adjust the weight distribution to accelerate error convergence. This data-dependent parameter selection method can effectively improve the prediction accuracy of the model.

[0135] S3 adopts a new method to construct the connection weight of the reserve pool. For the N+1th newly added neuron, only the network connection weight of the newly added neuron to the original neuron is retained. and its own network connection weight The connection weights of the original neurons to the newly added neurons are set to zero That is, no connection:

[0136]

[0137] in represents the connection weight of the ith neuron to the jth neuron, This construction method can ensure that when a neuron is added to the reserve pool, the output states of the first N reserve pool neurons do not change. It is only necessary to configure the new neurons that reduce the model training error to ensure the global approximation property of the network model.

[0138] The principle behind the regularized inequality constraints of the random configuration algorithm mentioned in S4 is paraphrased as follows:

[0139] Assume that the vector space Γ on L2 space is dense and exist Given 0<r<1 and a non-negative real number sequence {μ N+1},in μ N+1 ≤(1-r). Given the following formula:

[0140]

[0141] If the random basis function g N+1 The constructed output weight satisfy:

[0142]

[0143] And the following inequality constraints are satisfied:

[0144]

[0145] but Where T is the target output value, F N+1 is the predicted output value of the model with N+1 reserve pool neurons. That is, the constructed neural network model has a global approximation property.

[0146] S5 is to find the candidate neurons that can reduce the model training error the fastest and speed up the convergence.

[0147] The calculated output weights mentioned in S6 The specific principle is reproduced as follows:

[0148] Assume that the vector space Γ on L2 space is dense and exist Given 0<r<1 and a non-negative real number sequence μ N+1 ≤(1-r), and For the reserve pool neurons N = 1, 2, ..., ..., the following formula is given:

[0149]

[0150] If the random basis function g N+1 The constructed output weight W out,N+1 satisfy:

[0151]

[0152] And the following inequality constraints are satisfied:

[0153]

[0154] but

[0155]

[0156] but Right now The constructed neural network model has global approximation properties.

[0157] Therefore, the suboptimal solution It can be obtained by the least squares method:

[0158]

[0159] Among them G N+1is the matrix of random basis functions g, and I is the identity matrix.

[0160] Taking the Mackey-Glass (MG) chaotic system and a nonlinear dynamic system as examples,

[0161] Example 1: The Mackey-Glass (MG) chaotic system is a very typical chaotic time series, which is generated by the following differential equation with time delay:

[0162]

[0163] Among them, when τ>16.8, the entire sequence is chaotic, has no period, and neither converges nor diverges.

[0164] Example 2: System identification is an important part of modern control theory and signal processing. The dynamic characteristics of the system are considered to be reflected in the changing input and output data. However, most actual systems are nonlinear, so nonlinear system identification becomes an important and complex problem. Given the following nonlinear system, in which u(n) = 1.05 × sin(n / 45) in the training stage, the mathematical model in the test stage is as follows:

[0165] y(n+1)=0.72y(n)+0.025y(n-1)u(n-1)+0.01u 2 (n-2)+0.2u(n-3)

[0166]

[0167] Example 3: The debutanizer process is an important part of the desulfurization and naphtha separation unit in the petroleum refining process. This process uses 7 auxiliary variables, namely: tower top temperature u1, tower top pressure u2, tower top reflux u3, tower top product outflow u4, sixth layer tower plate temperature u5, tower bottom temperature 1u6, tower bottom temperature 2u7, to predict the butane concentration y at the bottom of the tower. Its nonlinear model can be described as:

[0168] y(n)=f(u1(n),u2(n),u3(n),u4(n),u5(n),u5(n-1),u5(n-2),u5(n-3),(u6(n)+u7(n)) / 2,y(n-1),y(n-2),y(n-3),y(n-4))

[0169] Considering the unknown order, we reset the input of the above model, that is,

[0170] y(n)=f(u1(n),u2(n),u3(n),u4(n),u5(n),u6(n),u7(n),y(n-1))

[0171] In the experiment of the embodiment of the present invention, ESN, LSTM, SCR, PESN and recursive random configuration network (RSCN) were selected for comparison with the method proposed in the present invention (RRSCN). Normalized root-meansquare error (NRMSE) was selected as the evaluation index of network prediction performance.

[0172] Example 1: Mackey-Glass (MG) chaotic system

[0173]

[0174] Analyzing the experimental results, LTSM has the largest prediction error, while RRSCN has the smallest prediction error and uses the least number of nodes. Figure 3 It is the fitting effect of RRSCN on MG dynamic data, which verifies that the method proposed in the present invention constructs the reserve pool in a supervised manner and introduces regularization constraints, which can effectively improve the prediction performance of the model.

[0175] Example 2: Nonlinear System Identification

[0176] Figure 4 It is the fitting effect diagram of each method, from which it can be seen that the model output of RRSCN can effectively fit the target value, and the prediction accuracy is significantly higher than other methods, which verifies the effectiveness of the method proposed in this invention in nonlinear system identification.

[0177] Example 3: Soft measurement of butane concentration at the bottom of debutanizer

[0178] Figure 5 It is the prediction effect of each neural network model. It can be seen that the prediction error of RRSCN is the smallest, and the fit between the model output and the target value is higher, which verifies the effectiveness of the method proposed in this invention in the soft measurement of industrial process parameters.

[0179] Another specific optimization scheme provided by the present invention is:

[0180] S1, data preprocessing:

[0181] In this phase, the raw data is cleaned and transformed through a series of preprocessing steps. These steps include:

[0182] Clean or fill missing values: handle missing data in a dataset by deleting rows or columns with missing values, or by filling missing values ​​using statistical methods (such as mean, median, etc.).

[0183] Feature Scaling: Make all input features have the same scale by normalizing (scaling the data to between 0 and 1) or standardizing (scaling the data to have a mean of 0 and a standard deviation of 1). This can help the model converge faster.

[0184] It also includes steps such as data discretization, feature selection, and feature encoding (for example, converting categorical features into one-hot encoding).

[0185] S2, regularized recursive random configuration network initialization:

[0186] Initialize the structure and parameters of the neural network.

[0187] Determine the basic structure of the network, for example, the number of neurons in the input layer (usually the same as the number of features), the number of neurons in the output layer (usually the same as the dimension of the target variable).

[0188] Initialize the pool. The pool is a collection of all candidate hidden layer neurons. Initially, the pool can be empty or contain some randomly initialized neurons.

[0189] Initialize the output weights. These weights can be initialized randomly or to zero.

[0190] S3, randomly configures weights and biases, and uses a new method to construct candidate neurons in the reserve pool:

[0191] For each neuron in the reserve pool, its weights and biases are randomly assigned. These weights and biases are usually randomly drawn from a given distribution (such as uniform distribution or normal distribution).

[0192] A specific method is used to construct new candidate neurons. For example, a new neuron can be constructed based on a specific activation function (such as ReLU, sigmoid, or tanh) or connection mode (such as full connection or local connection).

[0193] S4, supervised selection of candidate neurons based on regularized inequality constraints:

[0194] According to certain rules or criteria, candidate neurons are selected from the reserve pool. For example, the selection can be based on the output error of the neuron, or the complexity of the neuron (such as the size of the weight matrix or the connection density of the neuron).

[0195] This selection process usually needs to meet certain constraints. For example, the number of neurons selected can be limited, or the total complexity of the selected neurons can be limited.

[0196] S5, determine the best candidate neuron and add it to the reserve pool:

[0197] The best neuron is selected from the candidate neurons, usually the one that minimizes the prediction error.

[0198] Add this optimal neuron to the reserve pool.

[0199] S6, calculate the network output weight by combining the target output and the reserve pool output:

[0200] The output weights of the network are calculated based on the outputs of all neurons in the reserve pool and the target output. This step usually requires solving an optimization problem, for example, the output weights can be solved by minimizing the output error.

[0201] S7, based on the projection algorithm, updates the trained output weights online according to real-time data and calculates the real-time output:

[0202] During the model operation, the output weights are dynamically updated based on the real-time input data and target output data using optimization methods such as projection algorithms.

[0203] The projection algorithm is an iterative optimization algorithm that can iteratively optimize the weights while satisfying certain constraints (for example, the norm of the weights is restricted to a range).

[0204] After updating the output weights, the real-time output of the network is calculated based on the current input data and the updated weights. This output is the network's prediction result for the current input data.

[0205] The above is the detailed working principle of the dynamic modeling method based on the regularized recursive random configuration network. The key to this method is that it can dynamically adjust the structure and parameters of the network to adapt to changes in input data, thereby achieving better prediction performance.

[0206] Example 1: Predicting stock market trends

[0207] 1. Data preprocessing (S1): Normalize and process missing values ​​of historical stock market data.

[0208] 2. Initialization (S2): Initialize the structure and parameters of the regularized recursive random configuration network.

[0209] 3. Construct the reserve pool (S3): Randomly configure weights and biases, and use a specific algorithm to construct candidate neurons in the reserve pool.

[0210] 4. Select candidate neurons (S4, S5): Apply regularized inequality constraints to select supervised candidate neurons and add the best candidate neurons to the reserve pool.

[0211] 5. Calculate network output weights (S6): Use the target output and the reserve pool output to train the network output weights.

[0212] 6. Online update (S7): Use the projection algorithm to update the output weights of real-time stock market data online.

[0213] The embodiments of the present invention can accurately and efficiently predict stock market dynamics, and are helpful for risk assessment and decision support.

[0214] Example 2: Energy Demand Forecasting

[0215] 1. Data preprocessing (S1): Collect and process past energy consumption data, including normalization and seasonal adjustment.

[0216] 2. Initialization (S2): Similar to Example 1, network initialization is performed.

[0217] 3. Constructing a reserve pool (S3): Constructing specific candidate neurons for the reserve pool according to the characteristics of the energy data.

[0218] 4. Select candidate neurons (S4, S5): Perform supervised selection by regularizing inequality constraints and determine the optimal candidate neurons.

[0219] 5. Calculate network output weights (S6): Use historical energy demand and reserve pool output to train network output weights.

[0220] 6. Online update (S7): Perform online update of output weights based on real-time energy consumption data.

[0221] The embodiments of the present invention can accurately predict future energy demand, thereby achieving more effective energy management and conservation.

[0222] These two examples demonstrate the application potential of this method in the financial and energy fields, respectively, reflecting its high accuracy and adaptability.

[0223] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0224] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A dynamic modeling method based on regularized recursive random configuration network, characterized in that: The method first preprocesses the data, then initializes a regularized randomly configured network, and then uses a novel method to randomly configure weights and biases and construct candidate neurons in the reserve pool; regularized inequality constraints are used to supervisely select candidate neurons, and the optimal neurons are added to the reserve pool; Then, the output weights of the network are calculated by combining the target output and the reserve pool output. Finally, the projection algorithm is used to update the trained output weights online according to the real-time data and calculate the real-time output. First, data preprocessing was performed to collect historical operating data of the debutanizer, including seven auxiliary variables: tower top temperature, tower top pressure, tower top reflux rate, tower top product outflow rate, sixth-layer tower plate temperature, tower bottom temperature 1 and tower bottom temperature 2, as well as the corresponding tower bottom butane concentration data; Next, the collected data is cleaned, standardized or normalized to eliminate the dimension effect and potential data bias, ensuring that the impact of different variables on the model is at the same level, thereby improving the model training effect; Then, a dynamic modeling system based on the regularized recursive random configuration network (RRSCN) is initialized; appropriate initial parameters are selected, including the size of the reserve pool, the distribution of input weights and connection weights, and the expected output error of the model; Use the collected historical data to train the network model; In each iteration, the weights and biases are randomly configured, suitable candidate neurons are selected in a supervised manner according to the regularized inequality constraints, and the network output weights are calculated; by continuously adjusting and optimizing these parameters, the prediction error of the model is reduced; Once the model training is completed, the model is used to predict real-time data; by inputting the real-time tower top temperature, tower top pressure, tower top reflux flow, tower top product outflow, 6th layer tower plate temperature, tower bottom temperature 1 and tower bottom temperature 2, the model will output a predicted tower bottom butane concentration; Finally, the projection algorithm is applied to update the output weight of the model online to ensure that the model adapts to the dynamic changes in the process and ensures the accuracy and stability of the prediction; The RRSCN model can effectively perform soft measurement of butane concentration at the bottom of the debutanizer, helping operators control and optimize the production process.

2. The dynamic modeling method based on regularized recursive random configuration network according to claim 1, characterized in that: include: S1, data preprocessing; S2, regularized recursive random configuration network initialization; S3, randomly configures weights and biases, and uses a new method to construct candidate neurons in the reserve pool; S4, supervised selection of candidate neurons based on regularized inequality constraints; S5, determine the best candidate neurons and add them to the reserve pool; S6, calculate the network output weight by combining the target output and the reserve pool output; S7, based on the projection algorithm, updates the trained output weights online according to the real-time data and calculates the real-time output.

3. The dynamic modeling method based on regularized recursive random configuration network according to claim 2, characterized in that: S1 specifically includes: given a set of time series data, input samples: {u(1),u(2),...,u(n max )}={(y(1),u(1)),(y(2),u(2)),...,(y(n max ),u(n max ))}, Target output: T = {t(2), t(3), ..., t(n max +1)}, where y and u are the system output and the controlled input respectively, and the output y(n+1) at the next moment is predicted only by the current moment input (y(n), u(n)), n max is the number of samples; set the maximum allowable number of neurons in the reserve pool N max , the maximum expected output error allowable value ε, the maximum number of candidate neurons generated G max , input weight distribution parameter γ={λ min ,λ min +Δλ,...,λ max }, regularization coefficient C.

4. The dynamic modeling method based on regularized recursive random configuration network according to claim 2, characterized in that: S2 specifically includes: initializing the output error vector e0=T, the model output error scaling factor 0<r<1, assuming that there is already a reserve pool containing N neurons, the network model at this time is: x(n)=g(W in,N u(n)+W r,N x(n-1)+b N ) Where W in,N , W r,N and W out is the input weight, the pool connection weight and the output weight, b N is the bias, x(n) is the state of the reservoir at time n, X = [x(1), x(2), ... x(n)] is the reservoir state matrix, and g is the activation function.

5. The dynamic modeling method based on regularized recursive random configuration network according to claim 2, characterized in that: S3 specifically includes: min ,λ min +Δλ,...,λ max }, randomly select the input weight W in,N+1 , the connection weight of the reserve pool is W r,N+1 , bias b N+1 , substitute the activation function x(n)=g(W in,N+1 u(n)+W r,N+1 x(n-1)+b N+1 ) to get G max Group candidate neurons S4 specifically includes: max Substitute the candidate neurons into the inequality constraints of the regularized random configuration algorithm: Where m is the output dimension, and the non-negative real number sequence {μ N+1 }satisfy And μ N+1 ≤(1-r), from which candidate neurons that satisfy the inequality constraints are selected.

6. The dynamic modeling method based on regularized recursive random configuration network according to claim 2, characterized in that: Determining candidate neurons in S5 specifically includes: defining a set of variables: Select the equations that satisfy the inequality constraints and make ξ N+1 The largest candidate neuron is used as the optimal reserve pool candidate neuron.

7. The dynamic modeling method based on regularized recursive random configuration network according to claim 2, characterized in that: The addition to the reserve pool in S5 specifically includes: adding the optimal candidate neurons in the reserve pool to the network model, and directly calculating the output weight W of the neural network model according to the target output out , and calculate the output root mean square error of the neural network model, use the root mean square error as the loss function, and update the output error of the model e0: = e N+1 And the number of neurons in the reserve pool N=N+1.

8. The dynamic modeling method based on regularized recursive random configuration network according to claim 2, characterized in that: S6 specifically includes: if the output root mean square error ||e0||2 of the current network model is greater than the maximum expected output error tolerance ε, and the number of neurons in the reserve pool N is less than the maximum allowable number of neurons in the reserve pool N max , then repeat S3 to S5; if the output root mean square error of the current network model ||e0||2 is less than the maximum expected output error tolerance ε, or the number of neurons in the reserve pool N is greater than the maximum allowable number of neurons in the reserve pool N max , the training ends and a network model that meets the constraints of random configuration theory is obtained.

9. The dynamic modeling method based on regularized recursive random configuration network according to claim 2, characterized in that: S7 specifically includes: based on the projection algorithm, online updating of the trained output weights according to real-time data, and calculation of real-time output: y(n)=W out (n)g(n-1) Where 0<a<1, c>0 are two constants, g(n) is the output of the reserve pool at time n, and e(n) is the output error before correction.

10. A dynamic modeling system based on a regularized recursive random configuration network using the dynamic modeling method based on a regularized recursive random configuration network as claimed in any one of claims 1 to 9, characterized in that: include: A preprocessing module, used for data preprocessing; Initialization module, used to initialize the regularized recursive random configuration network; Random configuration module, used to randomly configure weights and biases, and adopt a new method to construct candidate neurons in the reserve pool; Neuron selection module, used to select candidate neurons in a supervised manner according to regularized inequality constraints; The optimal candidate determination module is used to determine the optimal candidate neuron and add it to the reserve pool; Output weight calculation module, used to calculate the network output weight by combining the target output and the reserve pool output; The online update module is used to update the trained output weights online according to real-time data based on the projection algorithm and calculate the real-time output.

Images