Iterative learning control optimization method for direct current motor with non-repetitive disturbance

Abstract

The present invention relates to the technical field of direct current motor control. Disclosed is an iterative learning control optimization method for a direct current motor with non-repetitive disturbance. The method comprises: using a deep Koopman operator to lift a nominal model of a nonlinear system to a high-dimensional linear space, so as to obtain a high-dimensional state space model description of the nonlinear system; then converting the state space model into a form of a linear repetitive process model; and using a two-dimensional iterative learning control method to obtain a linear matrix inequality condition for system convergence in the high-dimensional space, so that an agent is separately trained under iterative learning control of the high-dimensional state space model with model mismatch and under the nonlinear system with non-repetitive uncertainty, and finally, the agent is applied to an actual system. The controller gain obtained by the method can be better adapted to the original nonlinear system, and a deep reinforcement learning algorithm is designed to compensate for non-repetitive disturbance, thereby basically achieving zero-error tracking for a desired output.

Description

An Iterative Learning Control Optimization Method for DC Motors with Non-Repetitive Perturbations Technical Field

[0001] This invention relates to the field of DC motor control technology, and in particular to an iterative learning control optimization method for DC motors with non-repetitive disturbances. Background Technology

[0002] Iterative Learning Control (ILC) is a control method specifically designed for batch processes. It can use historical batch information, partial information of the current batch, and compare them with the desired output to correct the control signal of the previous batch, so that the actual output of the system is adjusted through trial and error and continuously approaches the desired output.

[0003] In practical DC motor control processes, the behavior is predominantly nonlinear. Obtaining the gain of the iterative learning controller for a nonlinear DC motor has become a significant challenge in applying iterative learning control to real-world industrial production. Furthermore, under actual production conditions, unknown disturbances such as system environmental changes, component wear, and aging—which vary over time and batch-wise—can easily degrade the performance of the iterative learning control system. Therefore, for nonlinear DC motor systems with non-repetitive uncertain disturbances, determining the gain of the iterative learning controller and compensating for these disturbances is a research area of ​​practical significance.

[0004] Existing solutions mainly include two methods: dynamic iterative linearization and compensated iterative learning control. Dynamic iterative linearization is a method that linearizes the system based on historical batch information to obtain the controller gain. The control algorithm is simple and easy to understand, but it has many irrelevant parameters that are difficult to adjust. Moreover, its control effect is difficult to guarantee under non-repetitive uncertain disturbances. Compensated iterative learning control methods, such as neural networks and reinforcement learning, are all designed for non-repetitive uncertainties, and they still rely on linearization near the steady-state point to obtain the iterative learning controller gain. Summary of the Invention

[0005] The purpose of this invention is to solve the trajectory tracking problem of a nonlinear DC motor with non-repetitive uncertain disturbances. A proposed iterative learning control optimization method for DC motors with non-repetitive disturbances is presented. This method designs a Koopman operator network to linearize the nonlinear system, uses a two-dimensional iterative learning control method to obtain the convergence condition of the system's linear matrix inequality, and then designs a deep reinforcement learning algorithm to compensate for the non-repetitive disturbances. The technical solution of this invention is as follows:

[0006] An iterative learning control optimization method for DC motors with non-repetitive perturbations includes the following steps:

[0007] The first step is to establish a dynamic model of the DC motor-driven single-bar system, including:

[0008] The DC motor-driven single-link system consists of a DC motor driving a single rigid link via gears. A photoelectric encoder is installed on one side of the rigid link to measure its angular position. The dynamic model of the system is represented by the following second-order differential equation:

[0009] in, θ1, θ2, and θ3 represent the inertia coefficient, damping coefficient, and angle of the DC motor, respectively; J1, B1, and θ1 represent the inertia coefficient, damping coefficient, and angle of the rigid connecting rod, respectively; n0 represents the gear ratio; u represents the torque; M represents the mass of the rigid connecting rod; g is the acceleration due to gravity; and l represents the length from the center of mass of the rigid connecting rod to the rotation axis.

[0010] The system (1) is discretely sampled using the Euler approximation method, and the motor angle and its differential are states. The rigid connecting rod angular velocity is the output. The torque is the input u(t); therefore, equation (1) is transformed into a state equation form:

[0011] in,

[0012] f(·) represents the vector function related to the state vector in the system, and its dimension matches that of other vectors; h is the sampling time interval, t = 0, 1, 2, ..., T represents the corresponding t-th sampling point, T is the sampling point length from the start of the system to the end of the current batch, x0 represents the initial state of the system, and d(k,t) is the non-repetitive uncertainty disturbance related to both time and batch.

[0013] The second step is to construct a deep neural network data-driven model based on the Koopman operator, including:

[0014] For a dynamic nonlinear system without external control input: x(t+1)=g(x(t)) (3)

[0015] function It is a non-linear function, the Koopman operator. The definition is: (KΨ)(x)=Ψ(g(x)) (4)

[0016] Wherein, the function ψ:R n →R is called the Koopman eigenfunction, which is determined by the symbol... The lifting function space represented by R n Represents an n-dimensional vector space, where n is the dimension of x(t); Koopman operator The Koopman operator in equation (4) is applied to the eigenfunction ψ and transformed in the function space, so that the transformation result is used to obtain the state value at the next moment through the mode ν; however, the Koopman operator in equation (4) is an infinite-dimensional linear operator. In practical applications, a finite-dimensional operator is used to approximate the Koopman operator in equation (4). Defined as:

[0017] in μ i Let ψ represent the i-th eigenvalue of this linear operator, i = 1, 2, ..., N, N >> n; i This represents the eigenfunction corresponding to the i-th eigenvalue;

[0018] Modality ν will be in a state of enhancement. The state vector Ψ, composed of eigenfunctions, is mapped back to the original state space.

[0019] Where, ν i Let the i-th mode be represented. Based on equations (5) and (6), we obtain the nonlinear dynamic system based on the Koopman theoretical framework, as shown in Figure 1 and its representation:

[0020] For controlled systems, Koopman theory can be extended to controlled nonlinear systems; first, for the nominal system of (2), its state vector is extended as:

[0021] The extended state vector is the concatenation of the original state vector and the input vector. Therefore, the extended state vector at the next time step is:

[0022] in, For the forward time shift operator, Su(t) = u(t+1), the eigenfunction is defined in the following extended form:

[0023] φ i (x(t),u(t))=ψ i (x(t))+ζ(u(t)) (10)

[0024] Where, ζ:R r →R r Represents a linear transformation, R r Let ψ represent an r-dimensional vector space, where r is the dimension of the input vector u(t); i Let φ be the i-th eigenfunction of x(t). i Let X(t) be the i-th eigenfunction of the extended state vector; define the Koopman operator under the extended state vector. For: φ i (X(t+1))=Kφ i (X(t)) (11)

[0025] The Koopman promotion function is as follows:

[0026] The finite-dimensional approximation of the Koopman operator in equation (10) is defined as follows:

[0027] in, And N>>n; let: Ψ(X(t))=[ψ1(x(t)) ψ2(x(t))…ψ N (x(t))] T

[0028] Equation (13) simplifies to:

[0029] Where A∈R N×N , by matrix The first N rows and N columns form the first N rows and N columns, B∈R N×r , by matrix The first N rows, columns N+1 to N+r, form the first N rows, where S∈R. r×r After using a lifting function to lift the low-dimensional space to a high-dimensional space, the original state variables are retained to ensure the continuity of the high-dimensional state variables. Therefore, a new lifted state vector is defined: χ(t)=[x T (t)Ψ T (x(t))] T

[0030] Equation (14) simplifies to:

[0031] Among them, A * ∈R (n+N)×(n+N) B * ∈R (n+N)×1 C * ∈R m×(N+n) , where m is the dimension of the output variable y(t); As a new Koopman operator, the linear Koopman model in the boosting space is obtained as shown in Equation (15); according to the system structure shown in Equation (15), the Koopman-based neural network data-driven model of system (2) is created as shown in Figure 2. This data-driven model mainly includes a data buffer and a deep Koopman operator network; the data in the data buffer is randomly sampled according to the time series control quantity to obtain a large number of batches of nonlinear system input and output information; in the deep Koopman operator network, the boosting function is replaced by a multi-layer neural network and the Koopman operator is replaced by a single-layer neural network to achieve the dimensionality increase and prediction of the original system;

[0032] The third step is to establish the loss function for the deep neural network based on the Koopman operator, including:

[0033] To leverage the high expressiveness and robustness of deep neural networks, we utilize neural network N... en Replacing the eigenfunction Ψ(X(t)) with (X|w), N is solved using deep learning. en The weight parameter w and the Koopman operator K of (X|w) are used; the initial state is selected according to the system operating conditions and actual environmental conditions, and the control quantity is randomly averaged to obtain K batches of training data, each batch having a length of T, and the data in the data buffer is continuously updated during the training process; an optimized loss function is established from three aspects: eigenfunction reconstruction capability, forward prediction capability, and system controllability in the improvement space.

[0034] Where, ω i L represents the loss function in three aspects i The weights; by optimizing the loss function as described above, training data is generated according to the discretized model of the nonlinear system in equation (2), and the control quantity is randomly sampled to obtain the data-driven deep EDMD model (15);

[0035] Step 4: Construct a linear repeating process model in a high-dimensional space, including:

[0036] According to equation (15):

[0037] Where k = 0, 1, 2... represents the system's running batches, let Let be the state error vector of adjacent batches in the system, i.e.:

[0038] Using error-based PD-type control laws:

[0039] Where e k (t)=y r (t)-y k(t), y r (t) represents the desired output of the system, and K1, K2, and K3 are the control gain matrices;

[0040] Iterative learning control systems are systems that operate repeatedly within a finite time, which aligns with the characteristics of linear repetitive process models. Therefore, system (17) is transformed into a linear repetitive process model, and by substituting equation (19) and system (17) into equation (18), we obtain:

[0041] Where K = K2 - K3; the difference in output error between the previous and next batches at the same time: e k (t)-e k-1 (t)=y r (t)-y k (t)-y r (t)+y k-1 (t)

[0042] Substituting equations (17), (18), and (19) into the above equation, we get:

[0043] Therefore, the linear repeating process model of the nominal system (20) is obtained as follows:

[0044] make:

[0045] The linear repetitive process model shown in equation (22) simplifies to the following form:

[0046] Step 5: Obtain the convergence conditions of the linear matrix inequalities of the system under iterative learning control, including:

[0047] To derive the convergence condition of the output error of the system under control law (19), Lemma 1 is given:

[0048] Schur's complement lemma: Given matrices χ1, χ2, χ3 of appropriate dimensions, where χ2 > 0, then Equivalent to the following two formulas:

[0049] Theorem 1: For a linear repetitive process model (23) under control law (19), the sufficient condition for system stability as k→∞ exists for the existence of positive definite matrices W1>0, W2>0, W3>0 and matrices R1, R2, R3 such that the following linear matrix inequalities hold:

[0050] The control gain matrix of control law (19) is:

[0051] Step 6: Design a deep reinforcement learning algorithm to compensate for the iterative learning control, including:

[0052] Because in actual industrial batch processes, there are disturbances that change simultaneously with time and batch, and the model obtained based on the Koopman operator cannot accurately reproduce the nonlinear system, this application uses the deep reinforcement learning SoftActor-Critic (SAC) algorithm to compensate for the two-dimensional iterative learning control. The control block diagram of the system is shown in Figure 4. The control quantities input to the system include the output of the iterative learning controller and the output of the reinforcement learning compensator. A model mismatch matrix and non-repetitive uncertainty disturbances are introduced into the system shown in (18):

[0053] in Let E1, E2, F1, and F2 represent the model mismatch matrix, where E1, E2, F1, and F2 are constant coefficient matrices related to the system. For non-repeating uncertainty perturbations that are simultaneously related to time and batch; H(t,k)=[d T [(k,t)...00];χ0 represents the initial state x0 in the high-dimensional space after the initial state x0 is increased in dimension by the Koopman operator;

[0054] The system's control signals consist of two parts:

[0055] in The control signal generated by the two-dimensional iterative learning control ILC. The compensation signal is generated by the deep reinforcement learning SAC agent; the interaction process between the deep reinforcement learning agent and the two-dimensional iterative learning control environment is shown in Figure 5, where s k (t)∈S represents the state of the SAC agent, and S is the set of historical batch information of the system; a k (t) represents the control compensation signal generated by the SAC agent; reward r k (t) is the product of the output error and the gain -β obtained by the system feedback after the agent applies the control compensation signal; the SAC algorithm takes into account both long-term reward and exploration degree, and its Bellman equation is as follows:

[0056] in, s k (t) follows a sampling probability distribution D, a k (t) is the value calculated using the policy function π, where H is the entropy of the policy function π, γ is the reward discount factor, and α is the relative importance factor of entropy, representing the importance of the agent's exploration ability relative to the reward; the state-action value function is defined as follows:

[0057] The policy network under the SAC algorithm is trained using the following formula:

[0058] in:

[0059] θ1 represents the output value of the state-action value network; θ2 and θ1 are the parameters of the two state-action value evaluation networks, respectively. and ε represents the parameters of the two state-action value target networks, and ε represents the parameters of the policy network; to ensure the policy function π converges more smoothly to the optimal policy π. * Define a target policy network Then equation (31) is updated to:

[0060] The parameter update methods for the state-action value target network and the target policy network are as follows:

[0061] Where τ is the weight of the original network parameters during parameter update;

[0062] After the state-action value network is optimized using the above formula, the policy network will maximize the reward and policy entropy using the following formula:

[0063] The policy network's input parameters are state values, and its output parameters are the mean μ and variance σ of a normal distribution. That is, action values ​​will be sampled according to a normal distribution with mean μ and variance σ. In the early stages of agent training, the agent needs to be encouraged to explore, so the weight α should be slightly larger. However, as the agent gradually finds the optimal control compensation signal during training, further exploration is no longer necessary, and the weight α should be smaller. Therefore, α can be automatically adjusted using the following loss function:

[0064] in, The minimum policy entropy constant;

[0065] Next, the control and feedback process of the SAC agent and the two-dimensional ILC is described as a Markov decision process. By training the SAC agent, compensation for the control signal of the two-dimensional ILC is achieved; let the state be: s k (t)=[…,q k-1 (t-1),q k-1 (t),q k-1 (t+1),…,q k (t-1),q k (t)] (35)

[0066] in:

[0067] The action is:

[0068] Considering that the goal of control is to achieve zero-error tracking of the desired trajectory while rapidly stabilizing the system, the reward for the agent is designed as: r k (t)=-β(||e k (t)||2+||x k (t)-x k-1 (t)||2) (37)

[0069] Where ||·||2 represents the L2 norm; to evaluate the performance of the policy in each batch, the average reward is defined as:

[0070] Step 6: Design a phased training method for the SAC agent, including:

[0071] To evaluate the performance of the control system, the root mean square error of the system output is defined as follows:

[0072] In the early stages of agent training, the high randomness of the policy may lead to poor system performance. Furthermore, online training methods consume a significant amount of time for data collection due to the limited amount of data generated per unit time. Therefore, the training process is divided into two stages. In the first stage of training, the agent is trained on a high-dimensional linear system model (26) with model mismatch and non-repetition uncertainty until… Or the number of training rounds is greater than a specific value N1; in the second phase of training, the agent is trained on a nonlinear system (2) with non-repetitive uncertainty until Or the number of training rounds is greater than a specific value N2; Generally, smaller positive numbers are chosen, while N1 and N2 are generally larger integers.

[0073] Step 7: Implement tracking control using the reinforcement learning compensation iterative learning control method, including:

[0074] Using the two-dimensional ILC controller obtained from the above steps and the SAC agent trained in stages in the two-dimensional ILC control system environment, the control signals generated by the agent and the control signals of the two-dimensional ILC are combined to perform trajectory tracking control on the actual nonlinear batch processing process with non-repetitive uncertainty.

[0075] Its further technical solution is to establish the loss function L from three aspects. i include:

[0076] (1) Eigenfunction reconstruction capability

[0077] According to the basic principles of Koopman, the lifting function's mapping of a low-dimensional state vector to a high-dimensional space is invertible; therefore, there exists a network N... de (Ψ|w de As shown in Figure 3, the result of remapping the high-dimensional state vector to a low-dimensional space is... Approximates the actual value of the state vector in the low-dimensional space; define the first loss function: L1(w)=||x(t)-N de ((N en x(t)|w)|w de (38)

[0078] (2) Forward prediction capability

[0079] The established data-driven model requires accurate prediction of the system's boosted state vector and the state vector in the low-dimensional space at the next time step. Therefore, a second loss function is defined:

[0080] (3) System controllability

[0081] To ensure the controllability of the established lifting state-space model, a third loss function is defined: L3(A * B * ) = N - rank(R c (40)

[0082] Among them, R c =[B * A * B * (A * ) 2 B * …(A * ) N-1 B * ].

[0083] The further technical solution is that the proof process for the inequality (24) to hold includes:

[0084] Based on the nature of two-dimensional systems controlled by iterative learning, a Lyapunov function of the following form is constructed: V(k,t)=V1(t,k)+V2(k,t)

[0085] Where V1(t,k) represents the energy change of the system along the time direction during a certain iteration, V2(k,t) represents the energy change along the batch direction, and S and S3 are dimension-matched diagonal matrices; substituting equation (23) into the above equation, the energy increment of the system in the two directions is:

[0086] The total energy increment is:

[0087] in:

[0088] In equation (41), H k+1 (t) represents system-related parameters, and H k+1 If ΔV(k,t)≠0, then the sufficient condition for the system to converge is: ΔV(k,t)<0

[0089] That is: Φ T PΦ-P<0

[0090] According to Lemma 1, equation (2) is equivalent to:

[0091] Substituting equation (42) into the above equation, we get:

[0092] Multiply the matrix on the left side of the above inequality by both left and right multiplication. make:

[0093] Thus, equation (24) is obtained, and the proof is complete.

[0094] The beneficial technical effects of this invention are:

[0095] For DC motor systems with non-repetitive uncertainties, the nominal model of the nonlinear system is elevated to a high-dimensional linear space using the deep Koopman operator, resulting in a high-dimensional state-space model description of the nonlinear system. This allows for an accurate approximation of the original nonlinear system. This state-space model is then transformed into a linear repetitive process model, and a two-dimensional iterative learning control method is used to obtain the controller gain. This gain better adapts to the original nonlinear system, ensuring the system's basic control performance. For non-repetitive disturbances, the interaction process between the reinforcement learning agent and the two-dimensional iterative learning control system is analyzed, and a phased training process for the agent is designed, achieving accurate compensation for non-repetitive disturbances. Using deep reinforcement learning to compensate for the iterative learning control signal eliminates the influence of non-repetitive uncertainty disturbances during nonlinear batch processing, essentially achieving zero-error tracking of the desired output. Attached Figure Description

[0096] Figure 1 is a framework diagram of the Koopman operator in this application.

[0097] Figure 2 is a diagram of the deep neural network structure constructed based on the Koopman operator in this application.

[0098] Figure 3 is a diagram of the neural network structure for eigenfunction reconstruction capability in this application.

[0099] Figure 4 is a block diagram of the two-dimensional iterative learning control for reinforcement learning compensation in this application.

[0100] Figure 5 is a schematic diagram of the interaction process between the intelligent agent and the two-dimensional iterative learning control system in this application.

[0101] Figure 6 is a comparison of the output of the linearized model obtained in this application and the original nonlinear model.

[0102] Figure 7 is the output response diagram of the control algorithm proposed in this application on a DC motor.

[0103] Figure 8 is a comparison of the root mean square error of the algorithm proposed in this application and two dynamic iterative linearization methods with batch variation.

[0104] Figure 9 is a graph showing the change of reward during the training process of the algorithm proposed in this application as a function of the number of training rounds. Detailed Implementation

[0105] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0106] This application provides an iterative learning control optimization method for a DC motor with non-repetitive perturbations. The specific implementation steps are described in steps one through seven of the invention description. Specifically, using the nonlinear DC motor model shown in equation (2), the parameters in the DC motor-driven single-bar system are set as follows: J l =0.44m -1 , B l =0.25N·s / m, M=0.5kg, g=9.8m / s 2 l = 0.1m; in addition, there is the gear ratio n0 = 1.6, and the rigid connecting rod angle. The system operating cycle is T = 3s, and the sampling time interval is h = 0.05s. Where d k,1 (t),d k,2 (t) represents the non-repetitive uncertainty disturbance that is simultaneously related to both time and batch: d k,1 (t)=0, d k,2 (t)=0.1(n1sin(2.5tπ)+n2sin(0.1kπ))

[0107] Where n1 and n2 are random noises following a normal distribution with mean μ = 0 and variance σ = 1. The initial state of the system is x0 = [0 0]. T The initial control variable u0 = 0. The control objective is:

[0108] First, the system is linearized using the Koopman operator network constructed based on Figure 1, resulting in a high-dimensional state-space model as shown in equation (17), where:

[0109] C * =[0 1 0 0 0 0 0 0],

[0110] in:

[0111] Starting from the initial state x0, the control quantity changes from [-10]. 3 10 3 Figure 6 shows a comparison of the output trajectories of the high-dimensional linearized model obtained by random sampling based on the Koopman operator with the four sets of output trajectories of the original nonlinear model. The images show that the output trajectories of the two models in each group are basically consistent, indicating that the high-dimensional linearized model has high accuracy. Then, based on Theorem 1, the control law gain under the control law of equation (19) is obtained as: K1=[K 1,1 K 1,2 ], K2=7.12, K3=7.12, K 1,1 = [-5.46×10 1 -1.28×10 1 -8.02×10 0 -1.38×10 2 3.30×10 1 2.13×10 2 ], K 1,2 = [-2.50×10 1 -3.87×10 0 -2.78×10 2 1.83×10 2 -2.08×10 2 -7.29×10 1 ].

[0112] Let E1 and E2 matrices be 10% of the nominal system matrices A and B, respectively, and F1 and F2 be the identity matrices. The non-repetitive uncertainty disturbance in the model mismatch of equation (26) is:

[0113] The states, actions, and rewards of the SAC agent, as shown in equation (37), are as follows:

[0114] Furthermore, in design formula (28), the reward discount factor γ = 0.99, the original parameter network proportion τ = 0.995 when the agent updates, and the training cutoff condition in step six... N1 = 300000, N2 = 1200000. The SAC agent is trained in stages using the training method in step 6 until the corresponding conditions are met. The reinforcement learning agent trained under the Koopman operator is applied to a nonlinear DC motor system with non-repetitive disturbances (2). The output response diagram is shown in Figure 7, which shows that the control method solves the nonlinear and non-repetitive uncertainty problems at the same time, so that the system output achieves excellent tracking performance of the expected output. Compared with the dynamic iterative linearization method, the root mean square error changes with the batch as shown in Figure 8. After adding non-repetitive uncertainty, the root mean square error of the iterative dynamic linearization method shows a small oscillation trend with the batch, while the method proposed in this application can quickly converge to near zero. Figure 9 shows the curve of the average reward of the agent during the training process changing with the number of training rounds. As the number of training rounds increases, the reward gradually increases and converges to near zero, indicating that the system output error also gradually converges to near zero.

[0115] This application provides an iterative learning control optimization method for DC motors with non-repetitive disturbances. The method samples the input and output data of the non-DC motor in batches according to a time series. Using the Koopman operator, the nonlinear DC motor model is elevated to a high-dimensional space, achieving an accurate approximation of the original nonlinear system. The resulting control law gain better suits the original nonlinear system. Deep reinforcement learning is used to compensate the iterative learning control signal, eliminating the influence of non-repetitive uncertainty disturbances during the nonlinear batch processing, essentially achieving zero-error tracking of the desired output.

[0116] The above descriptions are merely preferred embodiments of this application, and the present invention is not limited to the above embodiments. It is understood that other improvements and variations directly derived or conceived by those skilled in the art without departing from the spirit and concept of the present invention should be considered to be included within the protection scope of the present invention.

Claims

1. A direct current motor iterative learning control optimization method with non-repetitive disturbance, characterized in that, The method includes: The first step is to establish a dynamic model of the DC motor-driven single-bar system, including: The direct current motor driving single rod system is a direct current motor driving a single rigid connecting rod through a gear, and an optical encoder is installed on one side of the rigid connecting rod to measure the angular position of the connecting rod; the dynamic model of the system is represented by the following second-order differential equation: wherein θ1, θ2, and θ3 represent the inertia coefficient, damping coefficient, and angle of the DC motor, respectively; J1, B1, and θ1 represent the inertia coefficient, damping coefficient, and angle of the rigid connecting rod, respectively; n0 represents the gear ratio; u represents the torque; M represents the mass of the rigid connecting rod; g is the acceleration due to gravity; and l represents the length from the center of mass of the rigid connecting rod to the rotation axis. The system (1) is discretely sampled by Euler approximation method, and the motor angle and its differential are states Rigid link angular velocity is output The torque is the input u(t); from which, equation (1) is converted into a state equation form: wherein f(·) represents the vector function related to the state vector in the system, and its dimension matches that of other vectors; h is the sampling time interval, t = 0, 1, 2, ..., T represents the corresponding t-th sampling point, T is the sampling point length from the start of the system to the end of the current batch, x0 represents the initial state of the system, and d(k,t) is the non-repetitive uncertainty disturbance related to both time and batch. The second step is to construct a deep neural network data-driven model based on the Koopman operator, including: For dynamic nonlinear systems without external control input: x(t+1)=g(x(t)) (3) Function f: is a nonlinear function, the Koopman operator The definition of is: (KΨ)(x)=Ψ(g(x)) (4) where the function ψ: R n → R is called the Koopman eigenfunction, is given by the symbol represent the lifted function space, R n denotes the n-dimensional vector space, n being the dimension of x(t); Koopman operator Acting on the eigenfunction ψ, a transformation is performed on the function space, so that the transformation result passes through the mode v to obtain the state value at the next time; a finite-dimensional operator is used to approximate the Koopman operator in formula (4), and the approximate Koopman operator is defined as: wherein μ i denotes the i-th eigenvalue of this linear operator, i = 1, 2,..., N, N » n; ψ i denotes the eigenfunction corresponding to the i-th eigenvalue; By modality v to be in the elevated space the state vector Ψ composed of the eigenfunctions maps back into the original state space wherein v i represents the i-th mode, based on the formula (5) and the formula (6), a nonlinear dynamic system based on the Koopman theory is obtained: The Koopman theory is extended to controlled nonlinear systems. First, for the nominal system of (2), the state vector is extended as The extended state vector is the splicing of the original state vector and the input vector, and the extended state vector at the next time is: wherein, For the forward time shift operator, Su(t) = u(t+1), the eigenfunction is defined in the following extended form: φ i (x(t),u(t)) = ψ i (x(t))+ζ(u(t)) (10) where ζ: R r → R r denotes a linear transformation, R r denotes an r-dimensional vector space, r being the dimension of the input vector u(t); ψ i is the i-th eigenfunction of x(t), φ i is the i-th eigenfunction of the extended state vector X(t); define the Koopman operator under the extended state vector for: φ i (X(t+1)) = Kφ i (X(t)) (11) The Koopman promotion function is as follows: The finite-dimensional approximation of the Koopman operator of definition (10) is: wherein, And N >> n; Let: Ψ(X(t)) = [ψ1(x(t)) ψ2(x(t))... ψ N (x(t)) T Then equation (13) is simplified as: where A e R N×N by the matrix The first N rows of B consist of the first N columns of A, B e R N×r The matrix The first N rows of the matrix A consist of the columns from N+1 to N+r, S e R r×r After lifting the low-dimensional space to the high-dimensional space by the lifting function, the original state variable is reserved to ensure the continuity of the high-dimensional state variable, and thus a new lifting state vector is defined: χ(t) = [x T (t)Ψ T (x(t)) T Then equation (14) is simplified as: where A * ∈ R (n+N)×(n+N) , B * ∈ R (n+N)×1 , C * ∈ R m×(N+n) , m is the dimension of the output variable y(t); and As a new Koopman operator, the linear Koopman model in the boosting space is obtained as shown in Equation (15); according to the system structure shown in Equation (15), a Koopman-based neural network data-driven model of system (2) is created. This data-driven model mainly includes a data buffer and a deep Koopman operator network; the data in the data buffer is randomly sampled according to the time series control quantity to obtain a large number of batches of nonlinear system input and output information; in the deep Koopman operator network, the dimensionality of the original system is increased and predicted by replacing the boosting function with a multi-layer neural network and replacing the Koopman operator with a single-layer neural network; The third step is to establish the loss function for the deep neural network based on the Koopman operator, including: Using neural networks N en (X|w) instead of the eigenfunction Ψ(X(t)), using a deep learning approach to solve for the weight parameters w and Koopman operator en (X|w) instead of the eigenfunction Ψ(X(t)), using a deep learning approach to solve for the weight parameters w and Koopman operator According to the system working condition and actual environment condition, an initial state is selected, a random average sampling control amount is obtained, K batches of training data are obtained, the length of each batch is T, and the data in the data buffer are updated during the training process; an optimization loss function is established from three aspects of intrinsic function reconstruction ability, forward prediction ability and system controllability in the promotion space: Wherein, ω i Indicate the weight of three aspects loss function L i ; Through the optimization of the above loss function, the training data is generated according to the discrete model of the nonlinear system of formula (2), the control quantity is randomly sampled, and the deep EDMD model (15) based on data driving is obtained; Step 4: Construct a linear repeating process model in a high-dimensional space, including: According to equation (15) we obtain: where k = 0, 1, 2... is the running batch of the system, let The state error vector for the system adjacent batch, i.e., Using an error PD-type control law: where e k (t) = y r (t) - y k (t), y r (t) is the desired output of the system, K1, K2, K3 are control gain matrices; the system (17) is converted into a linear repetitive process model, and formula (19) and the system (17) are substituted into formula (18) to obtain: Where K = K2 - K3; is the difference in output error between the previous and next batches at the same time. e k (t)-e k-1 (t) = y r (t)-y k (t)-y r (t)+y k-1 (t) Substituting formula (17), (18), (19) into the above formula respectively gives: The linear repetitive process model of the nominal system (20) is thus given by Let: The linear repetitive process model represented by equation (22) is then reduced to the following form: Step 5: Obtain the convergence conditions of the linear matrix inequalities of the system under iterative learning control, including: To derive the convergence condition of the output error of the system under control law (19), Lemma 1 is given: Schur complement lemma: Given matrices χ1, χ2, χ3 of appropriate dimensions, and χ2 > 0, then Equivalent to the following two formulas: (1) (2) Theorem 1: A sufficient condition for the linear repetitive process model (23) under the control law (19) to be stable as k—>∞ is that there exist positive definite matrices W1>0, W2>0, W3>0 and matrices R1, R2, R3 such that the following linear matrix inequalities hold: The control gain matrix of the control law (19) is: Step 6: Design a deep reinforcement learning algorithm to compensate for the iterative learning control, including: Introduce the model mismatch matrix and non-repetitive uncertainty disturbance to the system shown in (18): wherein denotes a model mismatch matrix, E1, E2, F1, F2 are constant matrices related to the system, H(t, k) = [d T (k, t)... 00]; x0represents the initial state x0after the Koopman operator dimensionality increase in the high-dimensional space The control signal of the system consists of two parts: wherein a control signal generated for two-dimensional iterative learning control ILC, This is the compensation signal generated by the deep reinforcement learning SAC agent; during the interaction between the deep reinforcement learning agent and the two-dimensional iterative learning control environment, s k (t)∈S represents the state of the SAC agent, and S is the set of historical batch information of the system; a k (t) represents the control compensation signal generated by the SAC agent; reward r k (t) is the product of the output error and the gain -β obtained by the system feedback after the agent applies the control compensation signal; the SAC algorithm takes into account both long-term reward and exploration degree, and its Bellman equation is as follows: wherein s k (t) is subject to a sampling probability distribution D, a k (t) is a value computed by a policy function p, H is the entropy of the policy function p, g is a reward discount factor, a is a relative importance factor of the entropy, which represents the importance of the exploration ability of the agent relative to the reward; the state-action value function is defined as follows: The policy network under the SAC algorithm is trained by the following formula: wherein: are the output values of the state-action value networks; θ1and θ2are the parameters of the two state-action value estimation networks, respectively, and parameters of two state-action value target networks, respectively, and e is a parameter of the policy network; to make the policy function p more smoothly converge to the optimal policy p * , define a target policy network Equation (31) is then updated as: The parameter updating method of the state-action value target network and the target policy network is as follows: Where τ is the weight of the original network parameters during parameter update; After the state-action value network is optimized by the above formula, the policy network will maximize the reward and policy entropy by the following formula: Wherein, the input parameter of the policy network is the state value, and the output parameter is the mean value μ and the variance σ of the normal distribution, that is, the action value will be sampled according to the normal distribution with the mean value μ and the variance σ; the loss function is used to automatically adjust α as follows: wherein, The minimum policy entropy constant; Next, the control and feedback process of the SAC agent and the two-dimensional ILC is described as a Markov decision process. By training the SAC agent, compensation for the control signal of the two-dimensional ILC is achieved; let the state be: s k (t) = [..., q k-1 (t - 1), q k-1 (t), q k-1 (t + 1),..., q k (t - 1), q k (t)] (35) wherein: Let the action be: Considering that the goal of control is to achieve zero-error tracking of the desired trajectory while rapidly stabilizing the system, the reward for the agent is designed as follows: r k (t) = -β(||e k (t)||2+||x k (t) - x k-1 (t)||2) (37) where ||·||2denotes the two-norm; to evaluate the performance of the policy at each batch, the average reward is defined as: Step 6: Design a phased training method for the SAC agent, including: To evaluate the performance of the control system, the root mean square error of the system output is defined as follows: In the first phase of training, the agent is trained on a high-dimensional linear system model (26) with model mismatch and non-repetitive uncertainty until or the number of training rounds is greater than a certain value N1; in a second phase of training, the agent is trained on the nonlinear system (2) with non-repetitive uncertainty until Or the number of training rounds is greater than a specific value N2; Step 7: Implement tracking control using the reinforcement learning compensation iterative learning control method, including: Using the two-dimensional ILC controller obtained from the above steps and the SAC agent trained in stages in the two-dimensional ILC control system environment, the control signals generated by the agent and the control signals of the two-dimensional ILC are combined to perform trajectory tracking control on the actual nonlinear batch processing process with non-repetitive uncertainty.

2. The direct current motor iterative learning control optimization method with non-repetitive disturbance according to claim 1, characterized in that, A loss function L is established from three aspects i includes: (1) Eigenfunction reconstruction capability According to Koopman's fundamental principle, the lifting function maps the low-dimensional state vector to a high-dimensional space reversibly, so there exists a network N de (Ψ|w de ) such that the high-dimensional state vector is re-mapped to the low-dimensional space by It approximates the actual value of the low-dimensional state vector; define the first loss function: L1(w) = ||x(t) - N de ((N en x(t)|w)|w de )|| (38) (2) Forward prediction capability The established data-driven model requires accurate prediction of the next time's lifting state vector and the state vector in the low-dimensional space, so a second loss function is defined: (3) System controllability To ensure the controllability of the established lifting state-space model, a third loss function is defined: L3(A * ,B * ) = N-rank(R c ) (40) wherein R c = [B * A * B * (A * ) 2 B * … (A * ) N-1 B * ].

3. The method of claim 1, wherein the non-repetitive disturbance is a step change in the DC motor load. The proof of the inequality (24) includes: Based on the nature of iterative learning-controlled two-dimensional systems, a Lyapunov function of the following form is constructed: V(k, t) = V1(t, k) + V2(k, t) where V1(t, k) represents the energy change in the time direction during the iteration process, V2(k, t) represents the energy change in the batch direction, and S and S3 are diagonal matrices with matching dimensions; substituting equation (23) into the above equation, the energy increment of the system in the two directions is: The total energy increment is: wherein: H k+1 (t) is a parameter term related to the system, and H k+1 (t)≠0, then the sufficient condition for the system convergence is: ΔV(k,t)<0 Right now: Φ T PΦ-P < 0 According to equation (2) in Lemma 1, the above equation is equivalent to: Substituting formula (42) into the above formula gives: Simultaneously left and right multiply the matrix on the left side of the above inequality by Let: Thus, equation (24) is obtained, and the proof is complete.

Images