Iterative learning control optimization method for direct current motor with non-repetitive disturbance

Abstract

The invention discloses an iterative learning control optimization method for a direct current motor with non-repetitive disturbance, and relates to the technical field of direct current motor control, and the method comprises the steps: lifting a nominal model of a nonlinear system to a high-dimensional linear space through a deep Koopman operator, and obtaining the description of a high-dimensional state space model of the nonlinear system; converting the state space model into a linear repetition process model, and obtaining a linear matrix inequality condition of system convergence in a high-dimensional space by using a two-dimensional iterative learning control method; further, the intelligent agent is trained in a high-dimensional state space model iterative learning control system with model mismatch and a non-repetitive uncertainty nonlinear system, and finally the intelligent agent is applied to an actual system. The control law gain obtained by the method can better adapt to an original nonlinear system, a deep reinforcement learning algorithm is designed to compensate non-repetitive disturbance, and zero-error tracking of expected output is basically achieved.

Description

Technical Field

[0001] The invention relates to the technical field of direct current motor control, in particular to an iterative learning control optimization method for a direct current motor with non-repetitive disturbance. Background Art

[0002] Iterative learning control (ILC) is a control method specifically designed for batch processes. It can use historical batch information and partial information of the current batch and compare it with the expected output to correct the control signal of the previous batch, so that the actual output of the system can be adjusted through trial and error and continuously approach the expected output.

[0003] The actual DC motor control process is basically nonlinear. How to obtain the iterative learning controller gain of nonlinear DC motors has become a difficulty in applying iterative learning control to actual industrial production processes. At the same time, considering that under actual production conditions, the performance of iterative learning control systems is easily deteriorated due to unknown disturbances such as system environment changes, part wear, aging, etc. that change with time and batches. Therefore, for nonlinear DC motor systems with non-repetitive uncertain disturbances, how to obtain the iterative learning controller gain and compensate for non-repetitive disturbances is a research with practical significance.

[0004] Existing solutions mainly include two methods: dynamic iterative linearization and compensatory iterative learning control. Dynamic iterative linearization is a method of linearizing the system based on the historical batch information of the system and obtaining the controller gain. The control algorithm is simple and easy to understand, but it has many irrelevant parameters and is difficult to adjust. Not only that, its control effect is difficult to guarantee under non-repetitive uncertain disturbances. Compensatory iterative learning control methods such as neural networks and reinforcement learning are all aimed at non-repetitive uncertainties, and the iterative learning controller gain is still obtained by linearization near the steady-state point. Summary of the invention

[0005] The purpose of the present invention is to solve the trajectory tracking problem of a nonlinear DC motor with non-repetitive uncertain disturbances, and proposes an iterative learning control optimization method for a DC motor with non-repetitive disturbances. In this method, a Koopman operator network is designed to linearize the nonlinear system, and a two-dimensional iterative learning control method is used to obtain the linear matrix inequality convergence condition of the system, and then a deep reinforcement learning algorithm is designed to compensate for the non-repetitive disturbances. The technical solution of the present invention is as follows:

[0006] A DC motor iterative learning control optimization method with non-repetitive disturbances comprises the following steps:

[0007] The first step is to establish a dynamic model of the DC motor driven single-pole system, including:

[0008] The DC motor driven single-rod system is a DC motor driving a single rigid link through gears. A photoelectric encoder is installed on one side of the rigid link to measure the angular position of the link. The dynamic model of the system is represented by the following second-order differential equation:

[0009]

[0010] in, They represent the inertia coefficient, damping coefficient and angle of the DC motor respectively, J1, B1 and θ1 represent the inertia coefficient, damping coefficient and angle of the rigid connecting rod respectively; n0 represents the gear ratio, u represents the torque, M represents the mass of the rigid connecting rod, g represents the acceleration of gravity, and l represents the length from the center of mass of the rigid connecting rod to the rotation axis;

[0011] The system (1) is discretely sampled by Euler approximation, and the motor angle and its differential are the states The angular velocity of the rigid link is output The torque is the input u(t); thus, equation (1) is converted into the state equation form:

[0012]

[0013] in,

[0014]

[0015] f(·) represents the vector function related to the state vector in the system, and its dimension matches that of other vectors; h is the sampling time interval, t=0,1,2,..., T represents the corresponding t-th sampling point, T is the length of sampling points from the start of the system to the end of the current batch, x0 represents the initial state of the system, d(k,t) is a non-repeating uncertainty disturbance related to both time and batch;

[0016] The second step is to build a deep neural network data-driven model based on the Koopman operator, including:

[0017] For dynamic nonlinear systems without external control input:

[0018] x(t+1)=f(x(t)) (3)

[0019] Function f: is a nonlinear function, the Koopman operator is defined as:

[0020]

[0021] Among them, the function ψ:R n →R is called the Koopman eigenfunction, which is represented by the symbol Represents the lifting function space, Rn represents an n-dimensional vector space, where n is the dimension of x(t); Koopman operator Acting on the eigenfunction ψ, it is transformed in the function space so that the transformation result can obtain the state value of the next moment through the mode ν; however, the Koopman operator in equation (4) is an infinite-dimensional linear operator. In practical application, a finite-dimensional operator is used to approximate the Koopman operator in equation (4). The approximate Koopman operator Defined as:

[0022]

[0023] in μ i represents the i-th eigenvalue of this linear operator, i = 1, 2, ..., N, N >> n; ψ i represents the eigenfunction corresponding to the i-th eigenvalue;

[0024] Through the modality ν will be in the lifting space The state vector Ψ composed of eigenfunctions is mapped back to the original state space

[0025]

[0026] Among them, ν i Denotes the i-th mode. Based on equations (5) and (6), we get Figure 1 And the nonlinear dynamic system it represents:

[0027]

[0028] For the controlled system, the Koopman theory can be extended to the controlled nonlinear system. First, for the nominal system of (2), its state vector is expanded to:

[0029]

[0030] The extended state vector is the concatenation of the original state vector and the input vector, so the extended state vector at the next moment is:

[0031]

[0032] in, is the forward time-shift operator, Define the eigenfunction as the following expanded form:

[0033] φ i (x(t),u(t))=ψ i (x(t))+ζ(u(t)) (10)

[0034] Among them, ζ:R r →R r represents linear transformation, R r represents the r-dimensional vector space, where r is the dimension of the input vector u(t); ψ i is the i-th eigenfunction of x(t), φ i is the i-th eigenfunction of the extended state vector X(t); define the Koopman operator under the extended state vector for:

[0035]

[0036] Among them, the Koopman lifting function is:

[0037]

[0038] The finite-dimensional approximation of the Koopman operator in equation (10) is defined as:

[0039]

[0040] in, And N>>n; let:

[0041] Ψ(X(t))=[ψ1(x(t)) ψ2(x(t)) … ψ N (x(t))] T

[0042] Then formula (13) is simplified to:

[0043]

[0044] where A∈R N×N , by the matrix The first N rows and N columns of N×r , by the matrix The first N rows are composed of columns N+1 to N+r. After using the lifting function to lift the low-dimensional space to the high-dimensional space, the original state variables are retained to ensure the continuity of the high-dimensional state variables, so the new lifted state vector is defined:

[0045] χ(t)=[x T (t) Ψ T (x(t))] T

[0046] Then formula (14) is simplified to:

[0047]

[0048] Among them, A* ∈R (n+N)×(n+N) , B * ∈R (n+N)×1 , C * ∈R m×(N+n) , m is the dimension of the output variable y(t); As a new Koopman operator, the linear Koopman model in the lifting space is obtained as shown in formula (15); according to the system structure shown in formula (15), the Koopman-based neural network data-driven model of system (2) is created as follows Figure 2 As shown in the figure, the data-driven model mainly includes a data buffer and a deep Koopman operator network; the data in the data buffer is randomly sampled according to the time series control quantity to obtain a large number of batches of nonlinear system input and output information; in the deep Koopman operator network, the original system is upgraded and predicted by replacing the lifting function with a multi-layer neural network and replacing the Koopman operator with a single-layer neural network;

[0049] The third step is to establish the loss function of the deep neural network based on the Koopman operator, including:

[0050] In order to take advantage of the high expressiveness and robustness of deep neural networks, we use neural network N en (X|w) replaces the eigenfunction Ψ(X(t)) and uses deep learning to solve N en The weight parameter w and Koopman operator of (X|w) The initial state is selected according to the system working conditions and actual environmental conditions, and the control amount is randomly sampled and averaged to obtain K batches of training data. The length of each batch is T, and the data in the data buffer is continuously updated during the training process; the optimization loss function is established from three aspects: the ability to reconstruct the intrinsic function, the ability to predict forward, and the controllability of the system in the improvement space:

[0051]

[0052] Among them, ω i Represents the loss function L in three aspects i The weight of ; through the above optimization loss function, the training data is generated according to the discretization model of the nonlinear system of formula (2), and the control quantity is randomly sampled to obtain the data-driven deep EDMD model (15);

[0053] Step 4: Construct a linear repetitive process model in high-dimensional space, including:

[0054] According to formula (15), we can get:

[0055]

[0056] Where k = 0, 1, 2... is the operation batch of the system, let is the state error vector of adjacent batches of the system, that is:

[0057]

[0058] Using the error PD type control law:

[0059]

[0060] where e k (t) = y r (t)-y k (t), y r (t) is the desired output of the system, K1, K2, K3 are the control gain matrices;

[0061] The iterative learning control system is a system that runs repeatedly in a finite time, which is consistent with the characteristics of the linear repetitive process model. Therefore, the system (17) is transformed into a linear repetitive process model, and equation (19) and system (17) are substituted into equation (18) to obtain:

[0062]

[0063] Among them, K = K2-K3; the difference in output error between the previous and next batches of the system at the same time:

[0064] e k (t)-e k-1 (t) = y r (t)-y k (t)-y r (t)+y k-1 (t)

[0065] Substituting equations (17), (18), and (19) into the above equations, we obtain:

[0066]

[0067] Therefore, the linear repetitive process model of the nominal system (20) is as follows:

[0068]

[0069] make:

[0070]

[0071]

[0072] Then the linear repeated process model shown in formula (22) is simplified to the following form:

[0073]

[0074] Step 5: Obtain the linear matrix inequality convergence conditions of the system under iterative learning control, including:

[0075] In order to obtain the convergence condition of the output error of the system under the control law (19), Lemma 1 is given:

[0076] Schur's complement lemma: Given matrices χ1, χ2, χ3 of appropriate dimensions, and χ2>0, then It is equivalent to the following two equations:

[0077] (1)

[0078] (2)

[0079] Theorem 1: In the linear repetitive process model (23) under the control law (19), when k→∞, the sufficient condition for the stability of the system is that there exist positive definite matrices W1>0, W2>0, W3>0 and matrices R1, R2, R3 such that the following linear matrix inequality holds:

[0080]

[0081] Then the control gain matrix of control law (19) is:

[0082]

[0083] Step 6: Design a deep reinforcement learning algorithm to compensate for iterative learning control, including:

[0084] Since there are disturbances that change with time and batches in the actual industrial batch process, and the model based on the Koopman operator cannot accurately restore the nonlinear system, this application uses the deep reinforcement learning SoftActor-Critic (SAC) algorithm to compensate for the two-dimensional iterative learning control. The control block diagram of the system is as follows: Figure 4 As shown, the control quantity of the input system includes the output of the iterative learning controller and the output of the reinforcement learning compensator. The model mismatch matrix and non-repetitive uncertainty disturbance are introduced into the system shown in (18):

[0085]

[0086] in represents the model mismatch matrix, E1, E2, F1, F2 are constant coefficient matrices related to the system, is a non-repeating uncertainty disturbance related to both time and batch; H(t,k) = [d T(k, t)…00]; χ0 represents the initial state of the initial state x0 in the high-dimensional space after the Koopman operator upgrades the dimension;

[0087] The control signal of the system consists of two parts:

[0088]

[0089] in is the control signal generated by the two-dimensional iterative learning control ILC, is the compensation signal generated by the deep reinforcement learning SAC agent; in the process of interaction between the deep reinforcement learning agent and the two-dimensional iterative learning control environment, Figure 5 As shown, where s k (t)∈S is the state of the SAC agent, S is the set of historical batch information of the system; a k (t) is the control compensation signal generated by the SAC agent; the reward r k (t) is the product of the output error obtained by the system feedback after the agent applies the control compensation signal and the gain-β; the SAC algorithm takes into account both long-term rewards and exploration degree, and its Bellman equation is as follows:

[0090]

[0091] in, Indicates k (t) obeys the sampling probability distribution D, a k (t) is the value calculated by the policy function π, is the entropy of the policy function π, γ is the reward discount factor, and α is the relative importance factor of entropy, which indicates the importance of the agent's exploration ability relative to the reward; the state-action value function is defined as follows:

[0092]

[0093] The policy network under the SAC algorithm is trained by the following formula:

[0094]

[0095] in:

[0096]

[0097] is the output value of the state-action value network; θ1 and θ2 are the parameters of the two state-action value evaluation networks, and are the parameters of the two state-action-value target networks, and ε is the parameter of the policy network; in order to make the policy function π converge to the optimal policy π more smoothly* , define a target policy network Then formula (31) is updated as:

[0098]

[0099] The parameter update method of the state-action value target network and the target policy network is as follows:

[0100]

[0101] Among them, τ is the weight of the original network parameters when the parameters are updated;

[0102] After the state-action value network is optimized by the above formula, the policy network will maximize the reward and policy entropy by the following formula:

[0103]

[0104] Among them, the input parameter of the policy network is the state value, and the output parameter is the normal distribution parameter mean μ and variance σ, that is, the action value will be sampled according to the normal distribution with mean μ and variance σ; in the early stage of agent training, the agent needs to be encouraged to explore, and the weight α should be slightly larger at this time. However, when the agent can gradually find the optimal control compensation signal during training, exploration is no longer encouraged, and the weight α should be smaller. Therefore, α can be automatically adjusted through the following loss function:

[0105]

[0106] in, is the minimum strategy entropy constant;

[0107] Next, the control and feedback process of the SAC agent and the two-dimensional ILC is described as a Markov decision process. By training the SAC agent, the control signal of the two-dimensional ILC is compensated; let the state be:

[0108] s k (t)=[…,q k-1 (t-1),q k-1 (t),q k-1 (t+1),…,q k (t-1),q k (t)] (35)

[0109] in:

[0110]

[0111] Let the action be:

[0112]

[0113] Considering that the goal of control is to achieve zero-error tracking of the desired trajectory while stabilizing the system quickly, the reward of the agent is designed as:

[0114] r k (t)=-β(‖e k (t)‖2+‖x k (t)-x k-1 (t) ‖2) (37)

[0115] Where ‖·‖2 represents the two-norm; in order to evaluate the performance of the strategy in each batch, the average reward is defined as:

[0116]

[0117] Step 6: Design a phased training method for the SAC agent, including:

[0118] In order to evaluate the performance of the control system, the root mean square error of the system output is defined as follows:

[0119]

[0120] In the early stage of agent training, the strong randomness of the strategy may lead to poor system performance. At the same time, the online training method will take more time to collect data because the amount of data generated per unit time is relatively small. Therefore, the training process is divided into two stages. In the first stage of training, the agent is trained on a high-dimensional linear system model (26) with model mismatch and non-repeating uncertainty until Or the number of training rounds is greater than a certain value N1; in the second stage of training, the agent is trained on the nonlinear system (2) with non-repeating uncertainty until Or the number of training rounds is greater than a specific value N2; Generally, a smaller positive number is used, and N1 and N2 generally take larger integers;

[0121] Step 7: Implement tracking control of the reinforcement learning compensation iterative learning control method, including:

[0122] Using the two-dimensional ILC controller obtained in the above steps and the SAC agent trained in stages under the two-dimensional ILC control system environment, the control signal generated by the agent is synthesized with the control signal of the two-dimensional ILC to perform trajectory tracking control on the actual nonlinear batch process with non-repetitive uncertainties.

[0123] Its further technical solution is to establish the loss function L from three aspects i include:

[0124] (1) Eigenfunction reconstruction capability

[0125] According to the Koopman basic principle, the lifting function maps the low-dimensional state vector to the high-dimensional space is reversible, so there is a network N de (Ψ|w de )like Figure 3 The result of remapping the high-dimensional state vector to the low-dimensional space is Close to the actual value of the state vector in the low-dimensional space; define the first loss function:

[0126] L1(w)=||x(t)-N de ((N en x(t)|w)|w de )|| (38)

[0127] (2) Forward prediction capability

[0128] The established data-driven model requires accurate prediction of the next moment's lifting state vector and the state vector in the low-dimensional space of the system. Therefore, the second loss function is defined:

[0129]

[0130] (3) System controllability

[0131] In order to ensure that the established lifting state space model is controllable, the third loss function is defined:

[0132] L3(A * ,B * )=N-rank(R c ) (40)

[0133] Among them, R c =[B * A * B * (A * ) 2 B * …(A * ) N-1 B * ].

[0134] A further technical solution is that the process of proving that the inequality in formula (24) is true includes:

[0135] According to the essence of the two-dimensional system of iterative learning control, the following Lyapunov function is constructed:

[0136] V(k,t)=V1(t,k)+V2(k,t)

[0137]

[0138] Among them, V1(t,k) represents the energy change of the system along the time direction during a certain iteration, V2(k,t) represents the energy change along the batch direction, S and S3 are diagonal matrices with matching dimensions; Substituting equation (23) into the above equation, the energy increment of the system in the two directions is:

[0139]

[0140] The total energy increase is:

[0141]

[0142] in:

[0143]

[0144] In formula (41), H k+1 (t) is a parameter related to the system, and H k+1 (t)≠0, then the sufficient condition for the system to converge is:

[0145] ΔV(k,t)<0

[0146] Right now:

[0147] Φ T PΦ-P<0

[0148] According to formula (2) in Lemma 1, the above formula is equivalent to:

[0149]

[0150] Substituting formula (42) into the above formula, we get:

[0151]

[0152] Multiply the left and right matrices of the above inequality at the same time make:

[0153]

[0154] That is, we get formula (24), and the proof is complete.

[0155] The beneficial technical effects of the present invention are:

[0156] For the DC motor system with non-repetitive uncertainty, the nominal model of the nonlinear system is promoted to a high-dimensional linear space using the deep Koopman operator, and the high-dimensional state space model description of the nonlinear system is obtained, which can achieve accurate approximation of the original nonlinear system. Then the state space model is converted into a linear repetitive process model, and then the controller gain is obtained using the two-dimensional iterative learning control method. The gain can better adapt to the original nonlinear system and ensure the basic control performance of the system. For non-repetitive disturbances, the interaction process between the reinforcement learning agent and the two-dimensional iterative learning control system is analyzed and the staged training process of the agent is designed to achieve accurate compensation for non-repetitive disturbances. Using the deep reinforcement learning method to compensate for the iterative learning control signal can eliminate the influence of non-repetitive uncertainty disturbances in the nonlinear batch processing process, and basically achieve zero-error tracking of the expected output. BRIEF DESCRIPTION OF THE DRAWINGS

[0157] Figure 1 It is the framework diagram of the Koopman operator in this application.

[0158] Figure 2 This is a structural diagram of a deep neural network constructed based on the Koopman operator in this application.

[0159] Figure 3 This is a diagram of the neural network structure of the intrinsic function reconstruction capability in this application.

[0160] Figure 4 This is the reinforcement learning compensation two-dimensional iterative learning control block diagram in this application.

[0161] Figure 5 It is a schematic diagram of the interaction process between the intelligent agent and the two-dimensional iterative learning control system in this application.

[0162] Figure 6 This is a comparison chart of the output of the linearized model in this application and the original nonlinear model.

[0163] Figure 7 It is the output response diagram of the control algorithm proposed in this application on the DC motor.

[0164] Figure 8 It is a comparison chart of the root mean square error of the algorithm proposed in this application and two dynamic iterative linearizations as they change with batches.

[0165] Fig. 9 It is a curve chart showing the change of the reward of the algorithm proposed in this application during the training process with the number of training rounds. DETAILED DESCRIPTION

[0166] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings.

[0167] The present application provides an iterative learning control optimization method for a DC motor with non-repetitive disturbances. The specific implementation steps refer to the contents of the first to seventh steps in the invention content. The nonlinear DC motor model shown in formula (2) is used to set the various parameters in the DC motor drive single-pole system as follows: J l =0.44m -1 , B l =0.25N·s / m, M=0.5kg, g=9.8m / s 2 , l = 0.1m; In addition, there is a gear ratio n0 = 1.6, a rigid connecting rod angle The system operation cycle is T = 3s, and the sampling time interval is h = 0.05s. k,1 (t),d k,2 (t) are non-repeating uncertainty disturbances related to both time and batch:

[0168] d k,1 (t) = 0, d k,2 (t)=0.1(n1 sin(2.5tπ)+n2 sin(0.1kπ))

[0169] Where n1, n2 are normal distribution random noises with mean μ = 0 and variance σ = 1. The initial state of the system is x0 = [0 0] T , the initial control amount u0 = 0. The control target is:

[0170]

[0171] First, using the Figure 1 The constructed Koopman operator network is used to linearize the system, and a high-dimensional state space model can be obtained as shown in formula (17), where:

[0172] C * =[0 1 0 0 0 0 0 0],

[0173] in:

[0174]

[0175] Starting from the initial state x0, the control amount changes from [-10 3 ,10 3 ], the output trajectory of the high-dimensional linearized model obtained based on the Koopman operator is compared with the four groups of output trajectories of the original nonlinear model. Figure 6As shown in the figure, the output trajectories of the two models in each group are basically the same, indicating that the high-dimensional linear model has a higher accuracy. Then, based on Theorem 1, the control law gain under the control law of formula (19) is obtained as:

[0176] K1=[K 1,1 K 1,2 ], K2=7.12, K3=7.12,

[0177] K 1,1 =[-5.46×10 1 -1.28×10 1 -8.02×10 0 -1.38×10 2 3.30×10 1 2.13×10 2 ],

[0178] K 1,2 =[-2.50×10 1 -3.87×10 0 -2.78×10 2 1.83×10 2 -2.08×10 2 -7.29×10 1 ].

[0179] Assume that the E1 and E2 matrices are 10% of the nominal system matrices A and B respectively, and F1 and F2 are unit matrices. The non-repeating uncertainty disturbance in the model mismatch in equation (26) is:

[0180]

[0181] The state, action and reward of the SAC agent shown in formula (37) are designed as follows:

[0182]

[0183] r k (t) = -20(‖e k (t)‖2+‖x k (t)-x k-1 (t) ‖2),

[0184] In addition, the reward discount factor γ in formula (28) is designed to be 0.99, the original parameter network ratio τ is 0.995 when the agent is updated, and the training cutoff condition in the sixth step is N1 = 300000, N2 = 1200000. The training method in the sixth step is used to train the SAC agent in stages until the corresponding conditions are met. The reinforcement learning agent compensation two-dimensional iterative learning control method trained under the Koopman operator is applied to the nonlinear DC motor system with non-repetitive disturbances (2). The output response diagram is shown in Figure 7 As shown in Figure 2, this shows that the control method solves the nonlinear and non-repetitive uncertainty problems at the same time, so that the system output achieves excellent tracking performance of the expected output. Compared with the dynamic iterative linearization method, the root mean square error changes with the batch as shown in Figure 2. Figure 8 As shown, after adding non-repetitive uncertainty, the root mean square error of the iterative dynamic linearization method shows a small oscillation trend with the batch, while the method proposed in this application can quickly converge to near zero. Fig. 9 This is the curve of the average reward of the agent during the training process changing with the number of training rounds. As the number of training rounds increases, the reward gradually increases and converges to near zero, and the output error of the surface system also gradually converges to near zero.

[0185] The present application provides an iterative learning control optimization method for a DC motor with non-repetitive disturbances. The input and output data of the non-DC motor are sampled in batches according to a time series, and the nonlinear DC motor model is promoted to a high-dimensional space using the Koopman operator, which can achieve an accurate approximation of the original nonlinear system, and the obtained control law gain can better adapt to the original nonlinear system. The iterative learning control signal is compensated using a deep reinforcement learning method, which can eliminate the influence of non-repetitive uncertainty disturbances in the nonlinear batch processing process, and basically achieve zero-error tracking of the desired output.

[0186] The above is only a preferred embodiment of the present application, and the present invention is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present invention should be considered to be included in the protection scope of the present invention.

Claims

1. A DC motor iterative learning control optimization method with non-repetitive disturbances, characterized in that: The method comprises: The first step is to establish a dynamic model of the DC motor driven single-pole system, including: The DC motor driven single-rod system is a DC motor driving a single rigid connecting rod through gears, and a photoelectric encoder is installed on one side of the rigid connecting rod to measure the angular position of the connecting rod; the dynamic model of the system is represented by the following second-order differential equation: in, They represent the inertia coefficient, damping coefficient and angle of the DC motor respectively, J1, B1 and θ1 represent the inertia coefficient, damping coefficient and angle of the rigid connecting rod respectively; n0 represents the gear ratio, u represents the torque, M represents the mass of the rigid connecting rod, g represents the acceleration of gravity, and l represents the length from the center of mass of the rigid connecting rod to the rotation axis; The system (1) is discretely sampled by Euler approximation, and the motor angle and its differential are the states The angular velocity of the rigid link is output The torque is the input u(t); thus, equation (1) is converted into the state equation form: in, f(·) represents the vector function related to the state vector in the system, and its dimension matches that of other vectors; h is the sampling time interval, t=0,1,2,..., T represents the corresponding t-th sampling point, T is the length of sampling points from the start of the system to the end of the current batch, x0 represents the initial state of the system, d(k,t) is a non-repeating uncertainty disturbance related to both time and batch; The second step is to build a deep neural network data-driven model based on the Koopman operator, including: For dynamic nonlinear systems without external control input: x(t+1)=f(x(t)) (3) function is a nonlinear function, the Koopman operator is defined as: Among them, the function ψ:R n →R is called the Koopman eigenfunction, which is represented by the symbol Represents the lifting function space, R n represents an n-dimensional vector space, where n is the dimension of x(t); Koopman operator Act on the eigenfunction ψ and transform it in the function space so that the transformation result can obtain the state value of the next moment through the mode ν; a finite-dimensional operator is used to approximate the Koopman operator in equation (4). The approximate Koopman operator is Defined as: in μ i represents the i-th eigenvalue of this linear operator, i = 1, 2, ..., N, N >> n; ψ i represents the eigenfunction corresponding to the i-th eigenvalue; Through the modality ν will be in the lifting space The state vector Ψ composed of eigenfunctions is mapped back to the original state space Among them, ν i Denotes the i-th mode. Based on equations (5) and (6), the nonlinear dynamic system based on Koopman theory is obtained: The Koopman theory is extended to controlled nonlinear systems. First, for the nominal system in (2), its state vector is expanded to: The extended state vector is the concatenation of the original state vector and the input vector, so the extended state vector at the next moment is: in, is the forward time-shift operator, Define the eigenfunction as the following expanded form: f i (x(t),u(t))=ψ i (x(t))+ζ(u(t)) (10) Among them, ζ:R r →R r represents linear transformation, R r represents the r-dimensional vector space, where r is the dimension of the input vector u(t); ψ i is the i-th eigenfunction of x(t), φ i is the i-th eigenfunction of the extended state vector X(t); define the Koopman operator under the extended state vector for: Among them, the Koopman lifting function is: The finite-dimensional approximation of the Koopman operator in definition (10) is: in, And N>>n; let: Ψ(X(t))=[ψ1(x(t)) ψ2(x(t)) … ψ N (x(t))] T Then formula (13) is simplified to: where A∈R N×N , by the matrix The first N rows and N columns of N×r , by the matrix The first N rows are composed of columns N+1 to N+r. After using the lifting function to lift the low-dimensional space to the high-dimensional space, the original state variables are retained to ensure the continuity of the high-dimensional state variables, so the new lifted state vector is defined: χ(t)=[x T (t) Ψ T (x(t))] T Then formula (14) is simplified to: Among them, A * ∈R (n+N)×(n+N) , B * ∈R (n+N)×1 , C * ∈R m×(N+n) , m is the dimension of the output variable y(t); As a new Koopman operator, a linear Koopman model in the lifting space is obtained as shown in formula (15); according to the system structure shown in formula (15), a Koopman-based neural network data-driven model of system (2) is created, and the data-driven model mainly includes a data buffer and a deep Koopman operator network; the data in the data buffer is randomly sampled according to the time series control quantity to obtain a large number of batches of nonlinear system input and output information; in the deep Koopman operator network, the original system is upgraded and predicted by replacing the lifting function with a multi-layer neural network and replacing the Koopman operator with a single-layer neural network; The third step is to establish the loss function of the deep neural network based on the Koopman operator, including: Using neural network N en (X|w) replaces the eigenfunction Ψ(X(t)) and uses deep learning to solve N en The weight parameter w and Koopman operator of (X|w) The initial state is selected according to the system working conditions and actual environmental conditions, and the control amount is randomly averaged to obtain K batches of training data, each batch has a length of T, and the data in the data buffer is continuously updated during the training process; an optimization loss function is established from three aspects: the ability to reconstruct the eigenfunction, the ability to predict forward, and the controllability of the system in the improvement space: Among them, ω i Represents the loss function L in three aspects i The weight of ; through the above optimization loss function, the training data is generated according to the discretization model of the nonlinear system of formula (2), and the control quantity is randomly sampled to obtain the data-driven deep EDMD model (15); Step 4: Construct a linear repetitive process model in high-dimensional space, including: According to formula (15), we can get: Where k = 0, 1, 2... is the operation batch of the system, let is the state error vector of adjacent batches of the system, that is: Using the error PD type control law: where e k (t) = y r (t)-y k (t), y r (t) is the expected output of the system, K1, K2, K3 are the control gain matrices; transform the system (17) into a linear repetitive process model, and substitute equation (19) and system (17) into equation (18) to obtain: Among them, K = K2-K3; the difference in output error between the previous and next batches of the system at the same time: e k (t)-e k-1 (t)=y r (t)-y k (t)-y r (t)+y k-1 (t) Substituting equations (17), (18), and (19) into the above equations, we obtain: Therefore, the linear repetitive process model of the nominal system (20) is as follows: make: Then the linear repeated process model shown in formula (22) is simplified to the following form: Step 5: Obtain the linear matrix inequality convergence conditions of the system under iterative learning control, including: In order to obtain the convergence condition of the output error of the system under the control law (19), Lemma 1 is given: Schur's complement lemma: Given matrices χ1, χ2, χ3 of appropriate dimensions, and χ2>0, then It is equivalent to the following two equations: (1) (2) Theorem 1: For the linear repetitive process model (23) under the control law (19), when k→∞, the sufficient condition for the stability of the system is that there exist positive definite matrices W1>0, W2>0, W3>0 and matrices R1, R2, R3 such that the following linear matrix inequality holds: Then the control gain matrix of control law (19) is: Step 6: Design a deep reinforcement learning algorithm to compensate for iterative learning control, including: The model mismatch matrix and non-repeating uncertainty disturbance are introduced into the system shown in (18): in represents the model mismatch matrix, E1, E2, F1, F2 are constant coefficient matrices related to the system, is a non-repeating uncertainty disturbance related to both time and batch; H(t,k) = [d T (k, t)…00]; χ0 represents the initial state of the initial state x0 in the high-dimensional space after the Koopman operator upgrades the dimension; The control signal of the system consists of two parts: in is the control signal generated by the two-dimensional iterative learning control ILC, is the compensation signal generated by the deep reinforcement learning SAC agent; in the process of interaction between the deep reinforcement learning agent and the two-dimensional iterative learning control environment, s k (t)∈S is the state of the SAC agent, S is the set of historical batch information of the system; a k (t) is the control compensation signal generated by the SAC agent; the reward r k (t) is the product of the output error obtained by the system feedback after the agent applies the control compensation signal and the gain-β; the SAC algorithm takes into account both long-term rewards and exploration degree, and its Bellman equation is as follows: in, Indicates k (t) obeys the sampling probability distribution D, a k (t) is the value calculated by the policy function π, is the entropy of the policy function π, γ is the reward discount factor, and α is the relative importance factor of entropy, which indicates the importance of the agent's exploration ability relative to the reward; the state-action value function is defined as follows: The policy network under the SAC algorithm is trained by the following formula: in: is the output value of the state-action value network; θ1 and θ2 are the parameters of the two state-action value evaluation networks, and are the parameters of the two state-action value target networks, and ε is the parameter of the policy network; in order to make the policy function π converge to the optimal policy π more smoothly * , define a target policy network Then formula (31) is updated as: The parameter update method of the state-action value target network and the target policy network is as follows: Among them, τ is the weight of the original network parameters when the parameters are updated; After the state-action value network is optimized by the above formula, the policy network will maximize the reward and policy entropy by the following formula: Among them, the input parameter of the policy network is the state value, and the output parameter is the normal distribution parameter mean μ and variance σ, that is, the action value will be sampled according to the normal distribution with mean μ and variance σ; the following loss function is used to automatically adjust α: in, is the minimum strategy entropy constant; Next, the control and feedback process of the SAC agent and the two-dimensional ILC is described as a Markov decision process. By training the SAC agent, the control signal of the two-dimensional ILC is compensated; let the state be: s k (t)=[…,q k-1 (t-1),q k-1 (t),q k-1 (t+1),…,q k (t-1),q k (t)] (35) in: Let the action be: Considering that the goal of control is to achieve zero-error tracking of the desired trajectory while stabilizing the system quickly, the reward of the agent is designed to be: r k (t)=-β(||e k (t)||2+||x k (t)-x k-1 (t)||2) (37) Where ‖·‖2 represents the two-norm; in order to evaluate the performance of the strategy in each batch, the average reward is defined as: Step 6: Design a phased training method for the SAC agent, including: In order to evaluate the performance of the control system, the root mean square error of the system output is defined as follows: In the first stage of training, the agent is trained on a high-dimensional linear system model (26) with model mismatch and non-repeated uncertainty until Or the number of training rounds is greater than a certain value N1; in the second stage of training, the agent is trained on the nonlinear system (2) with non-repeating uncertainty until Or the number of training rounds is greater than a specific value N2; Step 7: Implement tracking control of the reinforcement learning compensation iterative learning control method, including: Using the two-dimensional ILC controller obtained in the above steps and the SAC agent trained in stages under the two-dimensional ILC control system environment, the control signal generated by the agent is synthesized with the control signal of the two-dimensional ILC to perform trajectory tracking control on the actual nonlinear batch process with non-repetitive uncertainties.

2. The iterative learning control optimization method for a DC motor with non-repetitive disturbance according to claim 1, characterized in that: Establish the loss function L from three aspects i include: (1) Eigenfunction reconstruction capability According to the Koopman basic principle, the lifting function maps the low-dimensional state vector to the high-dimensional space is reversible, so there is a network N de (Ψ|w de ) makes the high-dimensional state vector remapped to the low-dimensional space Close to the actual value of the state vector in the low-dimensional space; define the first loss function: L1(w)=||x(t)-N de ((N en x(t)|w)|w de )|| (38) (2) Forward prediction capability The established data-driven model requires accurate prediction of the next moment's lifting state vector and the state vector in the low-dimensional space of the system. Therefore, the second loss function is defined: (3) System controllability In order to ensure that the established lifting state space model is controllable, the third loss function is defined: L3(A * ,B * )=N-rank(R c ) (40) Among them, R c =[B * A * B * (A * ) 2 B * …(A * ) N-1 B * ].

3. The iterative learning control optimization method for a DC motor with non-repetitive disturbance according to claim 1, characterized in that: The proof process of the inequality in formula (24) includes: According to the essence of two-dimensional system of iterative learning control, the following Lyapunov function is constructed: V(k,t)=V1(t,k)+V2(k,t) Among them, V1(t,k) represents the energy change of the system along the time direction during a certain iteration, V2(k,t) represents the energy change along the batch direction, S and S3 are diagonal matrices with matching dimensions; Substituting equation (23) into the above equation, the energy increment of the system in the two directions is: The total energy increase is: in: In formula (41), H k+1 (t) is a parameter related to the system, and H k+1 (t)≠0, then the sufficient condition for the system to converge is: ΔV(k,t)<0 Right now: F T PΦ-P<0 According to formula (2) in Lemma 1, the above formula is equivalent to: Substituting formula (42) into the above formula, we get: Multiply the left and right matrices of the above inequality at the same time make: That is, we get formula (24), and the proof is complete.

Images