A method for predicting the centroid motion trajectory of ion clusters based on an improved echo state network.

Abstract

This application relates to the field of neural network technology, and more particularly, to a method for predicting the centroid motion trajectory of ion clusters based on an improved echo state network. The method includes acquiring historical time-series data of the ion cluster centroid motion trajectory, constructing multiple sample data for network training; obtaining a first target vector after random discarding; obtaining a second target vector through a 1*1 convolution bypass; performing a residual connection between the first and second target vectors through a residual connection layer to obtain a third target vector; multiplying the third target vector with the output coefficient matrix of the output layer to obtain the network output vector; and adjusting the output coefficient matrix based on the error between the network output vector and the expected output data. This application adds a random discarding layer and a residual connection layer, enhancing the network's generalization ability and improving the accuracy of the network in the application of ion cluster centroid motion trajectory prediction.

Description

Technical Field

[0001] This application relates to the field of neural network technology, and in particular, to a method for predicting the trajectory of the centroid of an ion cluster based on an improved echo state network. Background Technology

[0002] The trajectory of the centroid of an ion cluster is a crucial parameter for understanding and controlling its behavior, and its prediction has wide applications in numerous fields. For example, in structural analysis, the trajectory of the centroid can reflect the symmetry and stability within the structure. Predicting the trajectory of the centroid allows for a better understanding of the geometric configuration and spatial distribution of ion clusters. In molecular dynamics simulations, predicting the centroid's trajectory helps describe the motion state of ion clusters and is significant for studying their trajectories and dynamic behaviors under different conditions. In nanomaterial design, ion clusters are the fundamental building blocks of many nanomaterials, and predicting their centroid trajectories helps optimize the structural design, improving their performance and stability. In biomedicine, the prediction of the centroid's trajectory can be used to study the structure and function of biomolecules, such as protein folding and drug molecule binding sites.

[0003] While predicting the trajectory of the center of mass of an ion cluster has important applications in many fields, currently, there are few effective prediction methods. One approach is to analyze the forces acting on the ion cluster and predict its trajectory from a molecular dynamics perspective. However, the forces acting on an ion cluster are extremely complex. The cluster is affected not only by external factors such as electric, magnetic, and gravitational fields, but also by the interactions between ions within the cluster, such as electrostatic forces (Coulomb forces) and van der Waals forces between dipoles. Therefore, the trajectory of the center of mass of an ion cluster is an extremely complex nonlinear dynamic system. Numerical simulation of such a system, starting from its fundamental principles and equations, is a highly variable calculation; even subtle changes in initial factors can introduce significant prediction errors.

[0004] Since predicting the trajectory of the ion cluster's center of mass can also be viewed as a time series prediction problem, time series prediction methods have been applied to predicting the trajectory of the ion cluster's center of mass. However, mainstream time series prediction methods, such as ARMA, are linear prediction methods and are not suitable for predicting nonlinear systems such as molecular dynamics systems. Artificial neural networks, due to the nonlinearity of their neuron activation functions, possess powerful nonlinear modeling capabilities and have achieved widespread and profound success. For example, deep learning is based on artificial neural networks. However, currently, there are very few techniques for directly applying artificial neural networks to predicting the trajectory of the ion cluster's center of mass.

[0005] Echo-state networks are a type of artificial neural network proposed before deep learning. They are favored by the industry due to their simple structure and dynamic memory function, and are still widely used today. This application utilizes some important technical ideas from the development of deep learning to modify the structure of echo-state networks, forming a new type of echo-state network, and applying it to the prediction of the centroid motion trajectory of ion clusters. Summary of the Invention

[0006] This application provides a method for predicting the trajectory of the centroid of an ion cluster based on an improved echo state network, in order to solve one or more technical problems existing in the prior art, and at least provide a beneficial option or create conditions.

[0007] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0008] According to one aspect of the embodiments of this application, a method for predicting the centroid motion trajectory of an ion cluster based on an improved echo state network is provided. The improved echo state network includes an input layer, a dynamic reservoir, a random dropout layer, a 1*1 convolutional bypass, a residual connection layer, and an output layer. The method includes:

[0009] Historical time-series data of the centroid motion trajectory of the ion cluster are obtained, and multiple sample data for network training are constructed based on the historical time-series data. Each sample data includes the input data of the improved echo state network and the expected output data of the improved echo state network.

[0010] The input data of the improved echo state network is used as the input vector, multiplied with the input coefficient matrix of the input layer and sent to the dynamic reservoir. After transformation by the dynamic reservoir, the output vector of the dynamic reservoir is obtained. The output vector of the dynamic reservoir is then randomly discarded by the random discarding layer to obtain the first target vector after random discarding.

[0011] The input data of the improved echo state network is input into the 1*1 convolution bypass to obtain a second target vector with the same dimension as the first target vector.

[0012] The first target vector and the second target vector are residually joined by the residual connection layer to obtain the third target vector;

[0013] The third target vector is multiplied by the output coefficient matrix of the output layer, and the result of the multiplication is activated by the activation function of the output layer neuron to obtain the network output vector of the improved echo state network.

[0014] The output coefficient matrix is ​​adjusted based on the error between the network output vector and the expected output data to reduce the error between the network output vector and the expected output data. Through iterative training on each of the sample data, the error between the network output vector and the expected output data is adjusted to a preset error range.

[0015] In one embodiment of this application, based on the foregoing scheme, the step of using the input data of the improved echo state network as an input vector, multiplying it with the input coefficient matrix of the input layer, and then sending it to the dynamic reservoir, and obtaining the output vector of the dynamic reservoir after transformation by the dynamic reservoir, includes:

[0016] The result of multiplying the input vector of the improved echo state network at the current time with the input coefficient matrix of the input layer is used as the first component of the dynamic reservoir transformation.

[0017] The result of multiplying the output vector of the dynamic reservoir at the previous moment with the state coefficient matrix of the dynamic reservoir is taken as the second component of the dynamic reservoir transformation.

[0018] The result of multiplying the output vector of the improved echo state network at the previous moment with the feedback coefficient matrix is ​​taken as the third component of the dynamic reservoir transformation.

[0019] The first component, the second component, and the third component are input into the activation function of the dynamic reserve pool neuron to obtain the result of the dynamic reserve pool transformation, which is the output of the dynamic reserve pool at the current time.

[0020] The first nonlinear activation function is either the hyperbolic tangent function or the sigmoid function.

[0021] In one embodiment of this application, based on the foregoing scheme, the state coefficient matrix of the dynamic reserve pool is obtained through the following steps:

[0022] An initial matrix is ​​randomly generated within a preset range of matrix element values;

[0023] The spectral radius of the initial matrix is ​​determined, and the coefficients of the initial matrix are adjusted according to the spectral radius without random sparsification of the adjusted coefficients. The state coefficient matrix of the dynamic reserve pool is determined according to the spectral radius and the coefficients.

[0024] In one embodiment of this application, based on the foregoing scheme, the step of multiplying the third target vector with the output coefficient matrix of the output layer, and activating the result of the multiplication through the activation function of the output layer neurons to obtain the network output vector of the improved echo state network, includes:

[0025] The result of multiplying the third target vector with the output coefficient matrix of the output layer is input into the activation function of the output layer neuron using the second nonlinear activation function, so that the adjustment of the output coefficient matrix is ​​converted from matrix pseudo-inverse operation to steepest gradient descent operation.

[0026] The second nonlinear activation function is either the hyperbolic tangent function or the sigmoid function.

[0027] The beneficial effects of this application are as follows: This application proposes a novel echo-state network model architecture, namely an improved echo-state network, which consists of an input layer, a dynamic reservoir, a random dropout layer, a 1*1 convolutional bypass, a residual connection layer, and an output layer. Sample data is constructed by acquiring historical time-series data of the ion cluster centroid motion trajectory. This sample data includes the input data and the expected output data of the improved echo-state network. The input data can be used as the input vector of the input layer, which is then fed into the dynamic reservoir and the 1*1 convolutional bypass. After residual connections in the residual connection layer, the prediction result of the input data is output through the output layer, i.e., the network output vector described in this application. By continuously adjusting the output coefficient matrix of the output layer based on the error between the network output vector and the expected output data, the prediction result of the input data becomes more accurate, thus minimizing the error between the network output vector and the expected output data.

[0028] Specifically, the input data of the improved echo state network is used as the input vector, multiplied by the input coefficient matrix of the input layer, and then sent to the dynamic reservoir. After transformation by the dynamic reservoir, the output vector of the dynamic reservoir is obtained. The output vector of the dynamic reservoir is then randomly discarded by the random discarding layer to obtain the first target vector after random discarding.

[0029] The output vector of the dynamic reservoir is randomly discarded by the random dropout layer to obtain the first target vector. The input data of the improved echo state network is then input into the 1*1 convolution bypass to obtain a second target vector with the same dimension as the first target vector. The input vector can bypass the dynamic reservoir and obtain the second target vector through the 1*1 convolution bypass. That is, the second target vector is guided to the residual connection layer and added to the first target vector to obtain the third target vector. In this way, during training, the output coefficient matrix can not only learn the features of the input vector extracted by the dynamic reservoir, but also directly learn the features of the input vector itself (the input vector bypasses the dynamic reservoir and only performs feature extraction).

[0030] The third target vector is a vector after random discarding through a dropout layer, meaning some components are randomly discarded and set to 0. Since the third target vector is multiplied by the output coefficient matrix of the output layer, the result becomes the network output vector. The output coefficient matrix corresponds to the connection coefficients between neurons in the dynamic pool; that is, the connection coefficients between neurons are used as multiple weighting coefficients to weight the components in the third target vector. Because some components are randomly discarded and set to 0, the corresponding weighting coefficients are also zero. Therefore, these weighting coefficients are not adjusted in this training round, thus avoiding overfitting. Since training is based on the error iteration between the network output vector and the expected output data, and the random dropout layer randomly discards some components of the output vector of the dynamic reservoir (i.e., randomly discards some components of the third target vector) in each round of training, some components of the first target vector can be set to 0, and thus some components of the third target vector can also be set to 0. Therefore, the connection coefficients between each neuron have the opportunity to be trained during the iterative training process. Thus, each neuron in the trained dynamic reservoir can participate in the training process, making the performance of the improved echo state network better and the prediction of the ion cluster centroid motion trajectory more accurate.

[0031] In summary, this improved echo-state network adds a random dropout layer and a 1*1 convolutional bypass, which, unlike the traditional echo-state network, allows all neurons in the dynamic reservoir to participate in the training process, resulting in better performance of the improved echo-state network.

[0032] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0034] Figure 1 This is a diagram of the architecture of a traditional echo state network;

[0035] Figure 2 This is an architecture diagram of the improved echo state network model proposed in this application;

[0036] Figure 3This is a flowchart illustrating an ion cluster centroid motion trajectory prediction method based on an improved echo state network, according to an embodiment of this application. Detailed Implementation

[0037] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0038] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0039] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller node devices.

[0040] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0041] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0042] The following describes the structure of echo state networks in existing related technologies:

[0043] like Figure 1 As shown, Figure 1The existing Echo State Network (ESN) structure involves a feedforward signal entering from the input layer, flowing through the intermediate layer (dynamic reservoir) to the output layer, and then feeding the output signal back to the dynamic reservoir. The trajectory of the ion cluster's centroid is influenced by various factors, such as external fields (electric, magnetic, gravitational fields, etc.), inter-ion interactions (Coulomb forces, van der Waals forces, etc.), and the material composition of the ions (metals, nanomaterials, biomaterials, etc.). Therefore, the ion cluster centroid trajectory is a highly complex molecular dynamics system. However, in existing ESNs, although the activation functions of the dynamic reservoir neurons are nonlinear, the input coefficient matrix, the dynamic reservoir state coefficient matrix, and the feedback coefficient matrix—which are the independent variables of the activation functions—are predetermined and do not change during network training. In existing ESNs, the activation functions of the output layer neurons are typically set to linear, making the training process a linear regression, which significantly reduces the ability of classical ESNs to model the nonlinear trajectory of the ion cluster centroid.

[0044] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0045] According to one aspect of this application, a method for predicting the centroid motion trajectory of an ion cluster based on an improved echo state network is provided. Figure 2 The architecture diagram for the improved echo state network includes an input layer, a dynamic reservoir, a random dropout layer, a 1*1 convolutional bypass, a residual connection layer, and an output layer. Figure 3 The flowchart below illustrates a method for predicting the trajectory of the centroid of an ion cluster based on an improved echo state network, according to an embodiment of this application. This method includes at least steps S1 to S6, which are detailed below:

[0046] In step S1, historical time series data of the centroid motion trajectory of the ion cluster are obtained, and multiple sample data for network training are constructed based on the historical time series data. Each sample data includes the input data of the improved echo state network and the expected output data of the improved echo state network.

[0047] In step S2, the input data of the improved echo state network is used as the input vector, multiplied with the input coefficient matrix of the input layer and sent to the dynamic reservoir. After transformation by the dynamic reservoir, the output vector of the dynamic reservoir is obtained. The output vector of the dynamic reservoir is then randomly discarded by the random discarding layer to obtain the first target vector after random discarding.

[0048] In step S3, the input data of the improved echo state network is input into the 1*1 convolution bypass to obtain a second target vector with the same dimension as the first target vector.

[0049] In step S4, the first target vector and the second target vector are residually joined through the residual connection layer to obtain the third target vector;

[0050] In step S5, the third target vector is multiplied by the output coefficient matrix of the output layer, and the result of the multiplication is activated by the activation function of the output layer neuron to obtain the network output vector of the improved echo state network.

[0051] In step S6, the output coefficient matrix is ​​adjusted based on the error between the network output vector and the expected output data to reduce the error between the network output vector and the expected output data. Through iterative training of each sample data, the error between the network output vector and the expected output data is adjusted to a preset error range.

[0052] First, the dynamic reservoir is a core component of ESN. In existing ESNs, the state coefficient matrix of the dynamic reservoir is a sparse matrix, and its sparsity and position are predetermined and do not change during network training. This fixed sparsity means that the connection coefficients between some neurons are fixed at 0. Therefore, during the training of existing ESNs, the neurons corresponding to these fixed 0 connection coefficients do not participate in the training process, resulting in poor performance. Therefore, this application's embodiment incorporates the idea of ​​dropout in deep learning, abandoning the sparsity of the state coefficient matrix in traditional ESNs and instead performing dropout (randomly discarding) on ​​the output vector of the dynamic reservoir. Figure 2 The random dropout layer shown randomly drops the input vector, essentially setting some components of the dynamic buffer to zero. Following the deep learning approach, each time the network is trained using a single sample (i.e., one sample corresponds to one round of input data for the input layer), the output vector of the dynamic buffer is dropout-freeed again. This causes the output vector of the dynamic buffer to continuously change during training, allowing some components of the first target vector to be randomly set to 0, and consequently, some components of the third target vector. Therefore, the connection coefficients between neurons have a chance to be trained during iterative training. As a result, each neuron in the trained dynamic buffer can participate in the training process, leading to better performance of the improved echo-state network and more accurate prediction of the ion cluster centroid trajectory.

[0053] Residual connection is a concept in deep learning that allows the network to learn the residual (difference) between the input and the hidden layer output. In other words, it bypasses the dynamic buffer and directly learns the features of the input vector itself, rather than directly learning a complex mapping function. This application introduces a residual connection layer into the echo state network model. While guiding the input vector to the input of the dynamic buffer, it also bypasses the dynamic buffer and guides it to the output of the dynamic buffer (i.e., the residual connection layer). After being added to the output vector of the dynamic buffer (the first target vector), it is then connected to the output layer. This allows the output coefficient matrix to learn not only the features of the input vector extracted by the dynamic buffer, but also the input vector itself, especially the state information contained in the components that are zeroed out by dropout. This state information is directly the value of the input vector itself.

[0054] In the application of predicting the centroid motion trajectory of ion clusters, the sample data is the motion trajectory of the ion cluster centroid in a historical time series. Therefore, this application constructs sample data by obtaining historical time series data of the ion cluster centroid motion trajectory. For example, the historical time series consists of time A to time E, arranged chronologically, with time A being the earliest and time E being the latest. For example, time A to time C can be used as input data, i.e., as the input vector of the input layer, to train the improved echo state network. That is, time A to time C can be used as training samples, while time D to time E can be used as the expected output data of the improved echo state network, i.e., validation samples.

[0055] By continuously training with training samples, the error between the improved echo state network output vector (i.e., the prediction result obtained based on the training samples) and the validation samples is continuously reduced, thereby making the improved echo state network more accurate in predicting the trajectory of the centroid of the ion cluster.

[0056] Since the dimension of the input vector of the improved echo state network is inconsistent with the dimension of the output vector of the dynamic reservoir (i.e., the first target vector), in order to achieve residual connection, this application uses 1*1 convolution to transform the dimension of the input vector and then leads it to the output of the dynamic reservoir, thereby realizing the component-wise addition of the input of the improved echo state network and the output of the dynamic reservoir.

[0057] In one embodiment, the step of using the input data of the improved echo state network as an input vector, multiplying it with the input coefficient matrix of the input layer, and then sending it to the dynamic reservoir, followed by transformation by the dynamic reservoir to obtain the output vector of the dynamic reservoir, includes:

[0058] The result of multiplying the input vector of the improved echo state network at the current time with the input coefficient matrix of the input layer is used as the first component of the dynamic reservoir transformation.

[0059] The result of multiplying the output vector of the dynamic reservoir at the previous moment with the state coefficient matrix of the dynamic reservoir is taken as the second component of the dynamic reservoir transformation.

[0060] The result of multiplying the output vector of the improved echo state network at the previous moment with the feedback coefficient matrix is ​​taken as the third component of the dynamic reservoir transformation.

[0061] The first component, the second component, and the third component are input into the activation function of the dynamic reserve pool neuron to obtain the result of the dynamic reserve pool transformation, which is the output of the dynamic reserve pool at the current time.

[0062] The first nonlinear activation function is either the hyperbolic tangent function or the sigmoid function.

[0063] Figure 2 The input in this process is the input vector described in this application, and the output is the network output vector. Figure 2 The dashed line in the middle represents the feedback data.

[0064] The first nonlinear activation function is either the hyperbolic tangent function or the sigmoid function.

[0065] In one embodiment of this application, based on the foregoing scheme, the state coefficient matrix of the dynamic reserve pool is obtained through the following steps:

[0066] An initial matrix is ​​randomly generated within a preset range of matrix element values;

[0067] The spectral radius of the initial matrix is ​​determined, and the coefficients of the initial matrix are adjusted according to the spectral radius without random sparsification of the adjusted coefficients. The state coefficient matrix of the dynamic reserve pool is determined according to the spectral radius and the coefficients.

[0068] Specifically, an initial matrix W0 is randomly generated within the range of [-1,1] or [0,1] (i.e., the preset range of matrix element values ​​described in this application).

[0069] Calculate the spectral radius |λ of W0 max |, that is, the maximum value of the modulus of the eigenvalues ​​of W0;

[0070] Therefore, the state matrix W is:

[0071] Here, 0 < α < 1, and α is a preset coefficient.

[0072] The resulting spectral radius of W is less than 1.

[0073] The step of multiplying the third target vector with the output coefficient matrix of the output layer, and then activating the result through the activation function of the output layer neurons to obtain the network output vector of the improved echo state network includes:

[0074] The result of multiplying the third target vector with the output coefficient matrix of the output layer is input into the activation function of the output layer neuron using the second nonlinear activation function, so that the adjustment of the output coefficient matrix is ​​converted from matrix pseudo-inverse operation to steepest gradient descent operation.

[0075] The second nonlinear activation function is either the hyperbolic tangent function or the sigmoid function.

[0076] The embodiments of this application are further explained below:

[0077] 1. Detailed implementation of hyperparameters

[0078] The activation function f(·) for the dynamic reservoir:

[0079] f(·) employs a nonlinear function, thereby enabling the entire network model to perform nonlinear modeling. The hyperbolic tangent function or a sigmoid function can be used as the activation function f(·) for the dynamic reservoir, i.e.

[0080] or

[0081] The activation function f of the output layer out (·):

[0082] In traditional ESN, to reduce computational complexity, f out (·) Usually a linear function is used, i.e., f out (x) = x, but in this application, given that the trajectory of the ion cluster's centroid is an extremely complex nonlinear dynamic system, in order to enhance the nonlinear modeling capability of ESN, this application sets f... out (·) is also a non-linear function (i.e., the preset second non-linear activation function), and uses the ReLU function, which is currently the most commonly used in deep learning: ReLU(x) = max(0,x).

[0083] The dynamic reservoir contains the following number of neurons, N:

[0084] N represents the number of neurons. Neurons are an important hyperparameter of ESN. The larger N is, the more complex the dynamic system that ESN can describe, and the more accurate the description. For example... Figure 2The numbers x1 to xn shown represent neurons. However, an excessively large N will consume significant computational resources and degrade network performance. A value typically between 200 and 1000 is chosen. Based on experimental results on a dataset, this application exemplarily sets N = 600.

[0085] Input coefficient matrix W in and feedback coefficient matrix W back :

[0086] W in and W back The elements of these two matrices are determined according to the principles of randomness and sparsity. The elements are randomly generated within the range of [-1,1] or [0,1], and a sparsity ratio is set, randomly setting some elements in the matrices to zero.

[0087] State coefficient matrix W:

[0088] In traditional ESNs, W (i.e., the connection coefficients between neurons) is generated randomly and sparsly. In the improved ESN proposed in this application, dropout is used instead of sparsity, achieving the same sparsity effect. However, in traditional ESNs, the sparsity of W is constant (e.g., randomly setting certain coefficients to 0 does not change the subsequent training process, leading to some neurons not participating in the training process, which is not conducive to describing complex dynamic systems and has low accuracy). In the improved ESN of this application, the output vector of the dynamic reservoir (i.e., the first target vector) and the third target vector are constantly changing, ensuring that each neuron participates in the computation during training, and the connection coefficients of each neuron can potentially affect the output. That is, the network output vector can change according to the changes in the connection coefficients between neurons. Therefore, in this application, W is mainly generated randomly.

[0089] Based on the improved ESN architecture of this application, the specific implementation method is as follows:

[0090] 1. Construct training samples:

[0091] (u(1),y(1)),(u(2),y(2)),...(u(T train ),y(T train ))

[0092] Here, u(t) is the K-dimensional input vector, i.e., the input vector, and y(t) is the L-dimensional output vector, i.e., the expected output data, which is in vector form. K and L can be set according to the actual situation. trainLet y(t) represent the number of sample data points. For example, if there are 5 sample data points, then 5 training iterations are needed; if there are 100 sample data points, then 100 training iterations are needed. The meaning of (u(t), y(t)) is: train the ESN such that if the input of the ESN is u(t), then the desired output of the ESN is y(t), t = 1, ..., T. train .

[0093] 2. Set the initial values ​​for the training process: t = 0, x(0) = 0,

[0094] 3. Calculate the first output data for the dynamic reserve pool:

[0095] Based on the improved state equation of ESN, the output x(t+1) of the dynamic reservoir of DR is calculated:

[0096] u(t+1) represents the sample data at the current time (current round), i.e., the input vector at the current time (current round), W in u(t+1) is the first component, Wx(t) is the second component, and x(t) is the state coefficient matrix, i.e., the connection coefficients between neurons. The third component, The result is the product of the output vector and the feedback coefficient matrix of the improved echo state network at the previous time step (previous round).

[0097] 4. Randomly discard the components of the output x(t+1) of the dynamic reserve pool (some components are randomly forced to be 0), i.e., dropout (to improve the generalization ability of the network and make it applicable to more scenarios), and then obtain the first target vector.

[0098] 5. Perform a 1*1 convolution on u(t+1) to obtain the second target vector, so that the dimension of the second target vector u(t+1) is consistent with the dimension of x(t+1) (the first target vector).

[0099] 6. Residual connection

[0100] x c (t+1) = x(t+1) + u(t+1), where x c (t+1) represents the third target vector described in this application.

[0101] 7. Calculate the output of ESN. out The state coefficient matrix (composed of connection coefficients between neurons) is the only parameter that needs to be modified during training; the initial W... out It can be set randomly.

[0102]

[0103] 8. Calculation error

[0104] Let y(t+1) be the network output vector, and y(t+1) be the desired output data.

[0105] 9. Based on the principle of error backpropagation and the gradient descent method, use the error e to modify W. out That is, the output coefficient matrix is ​​adjusted based on the error between the network output vector and the expected output data as described in this application.

[0106] 10. If t≠T train If the result is -1, then t = t + 1, and repeat steps 3 to 9; otherwise, end the training.

[0107] In one embodiment, a test embodiment of an improved echo-state network is also included:

[0108] The purpose of the test is to use test samples to test whether the trained ESN meets the application requirements. The specific implementation method is as follows:

[0109] First, construct the test samples:

[0110] (v(1),z(1)),(v(2),z(2)),...(v(T test ),z(T test ))

[0111] Here, v(t) is the input K-dimensional vector (K and L are determined according to the actual situation), z(t) is the desired output L-dimensional vector, and T test The number of test samples differs from the number of training samples.

[0112] Second, calculate the network output vector of the improved ESN (i.e., the improved echo state network described in this application), i.e., the output;

[0113]

[0114] Here t = 0, ..., T test -1, and x(0) = 0.

[0115] Third, calculate the test error.

[0116]

[0117] This is the actual output.

[0118] In summary, this application, specifically for the prediction of the centroid motion trajectory of ion clusters, utilizes concepts such as dropout, residual connections, and channel transformation from deep learning to modify the structure of traditional ESNs (see...). Figure 2 This allows all neurons in the dynamic reservoir to participate in the training process, resulting in continuous changes in the output of the dynamic reservoir and dynamic adjustments in the neuron connection coefficients, increasing the describability of some complex dynamic systems. Consequently, the accuracy of predicting the trajectory of the ion cluster's centroid is also improved.

[0119] The above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this application and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0120] It should be understood that this application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for predicting the trajectory of the centroid of an ion cluster based on an improved echo state network, characterized in that, The improved echo-state network includes an input layer, a dynamic reservoir, a random dropout layer, a 1*1 convolutional bypass, a residual connection layer, and an output layer. The method includes: Historical time-series data of the centroid motion trajectory of the ion cluster are obtained, and multiple sample data for network training are constructed based on the historical time-series data. Each sample data includes the input data of the improved echo state network and the expected output data of the improved echo state network. The input data of the improved echo state network is used as the input vector, multiplied with the input coefficient matrix of the input layer and sent to the dynamic reservoir. After transformation by the dynamic reservoir, the output vector of the dynamic reservoir is obtained. The output vector of the dynamic reservoir is then randomly discarded by the random discarding layer to obtain the first target vector after random discarding. The input data of the improved echo state network is input into the 1*1 convolution bypass to obtain a second target vector with the same dimension as the first target vector. The first target vector and the second target vector are residually joined by the residual connection layer to obtain the third target vector; The third target vector is multiplied by the output coefficient matrix of the output layer, and the result of the multiplication is activated by the nonlinear activation function of the output layer neurons to obtain the network output vector of the improved echo state network. The output coefficient matrix is ​​adjusted based on the error between the network output vector and the expected output data to reduce the error between the network output vector and the expected output data. Through iterative training on each of the sample data, the error between the network output vector and the expected output data is adjusted to a preset error range.

2. The method for predicting the centroid motion trajectory of an ion cluster based on an improved echo state network according to claim 1, characterized in that, The step of using the input data of the improved echo state network as an input vector, multiplying it with the input coefficient matrix of the input layer, and then sending it to the dynamic reservoir, followed by transformation by the dynamic reservoir to obtain the output vector of the dynamic reservoir, includes: The result of multiplying the input vector of the improved echo state network at the current time with the input coefficient matrix of the input layer is used as the first component of the dynamic reservoir transformation. The result of multiplying the output vector of the dynamic reservoir at the previous moment with the state coefficient matrix of the dynamic reservoir is taken as the second component of the dynamic reservoir transformation. The result of multiplying the output vector of the improved echo state network at the previous moment with the feedback coefficient matrix is ​​taken as the third component of the dynamic reservoir transformation. The first component, the second component, and the third component are input into the first nonlinear activation function of the dynamic reserve pool neuron to obtain the result of the dynamic reserve pool transformation. The result of the dynamic reserve pool transformation is the output of the dynamic reserve pool at the current time. The first nonlinear activation function is either the hyperbolic tangent function or the sigmoid function.

3. The method for predicting the centroid motion trajectory of an ion cluster based on an improved echo state network according to claim 2, characterized in that, The state coefficient matrix of the dynamic reserve pool is obtained through the following steps: An initial matrix is ​​randomly generated within a preset range of matrix element values; The spectral radius of the initial matrix is ​​determined, and the coefficients of the initial matrix are adjusted according to the spectral radius without random sparsification of the adjusted coefficients. The state coefficient matrix of the dynamic reserve pool is determined according to the spectral radius and the coefficients.

4. The method for predicting the centroid motion trajectory of an ion cluster based on an improved echo state network according to claim 3, characterized in that, The step of multiplying the third target vector with the output coefficient matrix of the output layer, and then activating the result through the nonlinear activation function of the output layer neurons to obtain the network output vector of the improved echo state network includes: The result of multiplying the third target vector with the output coefficient matrix of the output layer is input into the activation function of the output layer neuron using the second nonlinear activation function, so that the adjustment of the output coefficient matrix is ​​converted from matrix pseudo-inverse operation to steepest gradient descent operation. The second nonlinear activation function is either the hyperbolic tangent function or the sigmoid function.

Images