Motor adaptive control method, recording medium and system

Abstract

The invention belongs to the technical field of motor control, and particularly relates to a motor adaptive control method, which comprises the following steps of: establishing an unconventional motor active disturbance rejection mechanism, and performing equivalent compensation on a control quantity through an error feedback control rate according to a received differential signal, a motor control system state and total disturbance influencing system output; parameter optimization in the mechanism is completed through combination of intelligent in-vivo neural network deep learning and reinforcement learning. The method is easy to adapt to a rapidly changing and uncertain working environment, can break through the limitation of traditional parameter adjustment, realizes a more intelligent and personalized control strategy, improves the control precision and adaptability of the linear motor, and is especially suitable for speed control of the linear motor. The invention also provides a non-transitory readable recording medium storing the program of the method and a system comprising the medium, and the program can be called through a processing circuit to execute the method.

Description

Technical Field

[0001] The invention belongs to the technical field of motor control and discloses a motor adaptive control method, recording medium and system. Background Art

[0002] In the field of contemporary automation and precision manufacturing, linear motors have many advantages over traditional rotary motors: their direct drive mode eliminates the need for mechanical transmission, thereby reducing energy loss and system complexity, while significantly improving dynamic response speed and positioning accuracy. These characteristics enable linear motors to achieve high-speed linear motion while achieving micron or even nanometer-level precision control, greatly meeting the stringent requirements of high-end application scenarios such as semiconductor chip manufacturing, precision machining, and high-speed automated conveying systems.

[0003] However, linear motors face a series of challenges in practical applications. Linear motor control systems are often affected by various internal and external disturbances during operation, such as load changes, temperature fluctuations, mechanical wear, etc. These factors have an adverse effect on the control performance of linear motors. Although traditional Active Disturbance Rejection Controller (ADCC) technology has solved these problems to a certain extent, using a PID (proportional-integral-differential) control mechanism, its performance is limited by fixed parameter settings and is difficult to adapt to rapidly changing and uncertain working environments.

[0004] With the rapid development of artificial intelligence technology, especially the application of deep reinforcement learning (DRL) in the field of automatic control, new possibilities have been brought to the optimization of linear motor control parameters. However, how to combine and utilize the various functions of ADRC and DRL tools to break through the limitations of traditional parameter adjustment, realize more intelligent and personalized control strategies, and improve the control accuracy and adaptability of linear motors is a problem currently faced by technical personnel in this field. Summary of the invention

[0005] In view of the above problems, the present invention provides a motor adaptive control method, comprising the following steps: using a differential tracker to obtain a smooth differential signal from an original motor state signal containing random noise; the differential tracker is expressed as: ; ; In the formula, k 1 is a constant coefficient, x is the signal variable, x * is the signal given value, is the signal observation value, e1 is the difference between the given value of the signal and the observed value of the signal. fal is a nonlinear function, m 1 is the nonlinear factor, n 1 is the filtering factor, z 11 is the first-order derivative of the signal variable with respect to time; The extended state observer is used to observe the state of the motor control system and the total disturbance affecting the system output; the extended state observer is expressed as: ; In the formula, k2, k3 are constant coefficients, m2, m3 are nonlinear factors, n2, n3 are filter factors, b0 is the compensation coefficient, is the perturbed observation, z 21 is the first-order derivative of the signal observation value with respect to time; u is the control input variable; According to the received differential signal, the state of the motor control system and the total disturbance affecting the system output, the control amount is compensated by the error feedback control rate, and the error feedback control rate is expressed as: ; In the formula, f is the disturbance variable, k 4， k 5 is a constant coefficient, m 4， n 4 is the nonlinear factor, m 5， n 5 is the filter factor, b 0 is the compensation coefficient, u0 is the control input given value; the parameter optimization involved in the formulas of the differential tracker, extended state observer, and error feedback control rate adopts an artificial intelligence learning method and is set through multiple rounds of cyclic network iterative training.

[0006] Preferably, the artificial intelligence learning method includes reinforcement learning and deep learning coupled to an intelligent agent, wherein the reinforcement learning adopts a Markov decision process, wherein the state, action, reward function and change relationship parameters of the state change probability involved are optimized through the iteration of the recurrent neural network of deep learning. In each control cycle, the optimization results of deep learning and reinforcement learning are interactively updated, and the control parameters finally obtained are input into the motor controller to control the motor.

[0007] Preferably, the value function optimization is the optimization of the mean of the value functions obtained based on evaluations of two groups of different dimensions.

[0008] Preferably, a particle swarm optimization algorithm is used to optimize the neural network structure of the deep learning.

[0009] Preferably, the weight coefficient of the particle swarm optimization algorithm is formed by the addition of three parts, the first part is the reference weight, the second part is the product of the speed function ratio of the previous and next adjacent moments and the speed weight, and the third part is the product of the position function ratio of the previous and next adjacent moments and the position weight.

[0010] Preferably, the linear motor motion equation with disturbance is as follows: ;in, F fri is the friction force, F load is the load force, F E is the electromagnetic force, F dis It is the external disturbance force. M is the mass of the mover; in the speed loop, the feedback control rate and the motor motion equation are combined to select the optimized artificial intelligence learning method k 3, k 4, k 5 Generate the q-axis current reference value in the motor control quantity: ; where the disturbance appears in the form of force F, the parameter with a superscript ^ represents the observed value of the parameter, and the parameter with a superscript * represents the reference value of the parameter, F 0 =F fri +F load ; Perturbed Observations ; The external disturbance is compensated to the motor current through the above formula to adapt to the smooth control of the motor in different environments, thereby reducing speed and thrust fluctuations.

[0011] Preferably, the reward function is a calculation of the steady-state and dynamic performance of the linear motor, and the dynamic performance includes speed overshoot σ, rise time t r and adjustment time t s , steady-state performance includes speed steady-state error v error and position steady-state error x error ; Reward function , where p 1, p 2, p 3, p 4, p 5 is the reward function weight coefficient.

[0012] Another embodiment of the present invention is to provide a non-transitory readable recording medium for storing one or more programs including a plurality of instructions. When the instructions are executed, the processing circuit will execute the above-mentioned motor adaptive control method.

[0013] Another embodiment of the present invention is to provide a motor adaptive control system, including a processing circuit and a memory electrically coupled thereto, wherein the memory is configured to store at least one program, wherein the program includes a plurality of instructions, and the processing circuit runs the program to execute the above-mentioned motor adaptive control method.

[0014] Compared with the prior art, the motor adaptive control method, recording medium and system provided by the present invention have the following beneficial effects: (1) an intelligent agent is used to autonomously learn the optimal ADRC controller parameters without relying on experience and trial and error, and can adapt to different operating conditions and disturbance environments, thereby improving the control accuracy and stability of the linear motor.

[0015] (2) The present invention adopts an improved particle swarm algorithm. After calculating the average value of the loss function, it infers the shortcomings of the current network strategy, updates the control strategy in real time, and adjusts the position and speed of the particles to more accurately approximate the optimal control strategy, ensuring that the control parameters can quickly adapt to the dynamic changes of the linear motor.

[0016] (3) The PSO was improved to learn and select strategies in the Actor-Critic framework, introduce partitioned inertia weights, recycle global optimal historical data, optimize the neural network structure parameters, improve the search efficiency and optimization accuracy of the control strategy, and achieve a balance between motor control accuracy and system stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is the flow chart of the conventional linear motor speed loop control; Figure 2 is a flow chart of the interaction between an intelligent agent and an environment in an embodiment of the present invention; Figure 3 It is a schematic diagram of the linear motor APADRC reinforcement learning principle in an embodiment of the present invention; Figure 4 It is a control block diagram of the linear motor speed loop APADRC in an embodiment of the present invention; Figure 5 It is an updated strategy diagram of the improved DRLPSO algorithm in an embodiment of the present invention; Figure 6 It is a schematic diagram of APADRC control parameters for the training part in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be described below in conjunction with the drawings in the embodiments of the present invention. The described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making innovative efforts are within the scope of protection of the present invention.

[0019] An embodiment of a motor adaptive control method provided by the present invention comprises the following steps: This method optimizes the APADRC controller parameters through DRLPSO to adapt to different operating conditions and external disturbances. The agent calculates the reward and value functions during the interaction with the environment. After calculating the average value of the loss function, the improved PSO algorithm uses it as the optimization target value to infer the shortcomings of the current network strategy, update the control strategy in real time, and adjust the position and speed of the particles to more accurately approach the optimal control strategy, ensuring that the control parameters can quickly adapt to the dynamic changes of the linear motor. The improved PSO performs strategy learning and selection on the Actor-Critic framework, introduces partitioned inertia weights, recycles global optimal historical data, optimizes the neural network structure parameters, and improves the search efficiency and optimization accuracy of the control strategy. It mainly includes the design of the APADRC controller of the linear motor, the design of the interface module and reward function of the APADRC controller and deep reinforcement learning, and the improved DRLPSO inverse network strategy algorithm.

[0020] The design of the linear motor APADRC controller can combine the motor equation and the ADRC principle to derive the relationship between the motor current compensation term and the thrust disturbance and speed. Figure 1 As shown, the speed loop control equation of the linear motor with disturbance is as follows: (1) In the formula F fri is the friction force, F load is the load force, F E is the electromagnetic force, F dis It is the external disturbance force. M is the mass of the mover; B is the friction coefficient.

[0021] The differential tracker in ADRC is expressed as: (2) (3) In the formula, k 1 is a constant coefficient, v * is the given speed, is the observed speed, e 1 is the difference between the given speed and the actual speed. fal is a nonlinear function, m 1 is the nonlinear factor, n 1 is the filtering factor, z 11 is the first-order derivative of velocity.

[0022] The extended state observer in ADRC is expressed as: (4) In the formula, k 2， k 3 is a constant coefficient, m 2， m 3 is the nonlinear factor, n 2， n 3 is the filter factor, b 0 is the compensation coefficient. z 22 and is a perturbation observation. z 21 It is the first-order derivative of the position observation signal with respect to time.

[0023] The error feedback control rate in ADRC is expressed as: (5) In the formula, k 4， k 5 is a constant coefficient, m 4， n 4 is the nonlinear factor, m 5， n 5 is the filter factor, b 0 is the compensation coefficient.

[0024] All control parameters can be optimized through deep reinforcement learning training to adapt to different operating conditions and disturbance environments.

[0025] In the speed loop, combine the ADRC formula and the motor motion equation to select k 3, k 4, k 5 Generate available current reference value: (6) (7) (8) In the formula is the observed value of the disturbance force.

[0026] From the above formula, we can know that the motor thrust is positively correlated with the speed. The external disturbance is compensated to the motor current through APADRC to adapt to the smooth control of the motor in different environments, thereby reducing speed and thrust fluctuations.

[0027] like Figure 2 As shown in the figure, the agent receives the electrical signal from the motor output and the set reward function, and improves the coefficients of the previous training through the Actor network and the Critic network. It continuously interacts and evaluates with the environment until the reward function R converges. The agent uses a deep reinforcement learning algorithm to update the network weights according to the reward signal and learn the optimal APADRC controller parameters.

[0028] The design of the APADRC controller, deep reinforcement learning interface module and reward function is as follows: Figure 3-5 As shown in Figure 2. The core of reinforcement learning is the Markov decision process (MDP), where s is the state set (motor current, speed, etc.), a For action sets, r is the reward function, P For the status s Take action a Go to next state s’ probability.

[0029] Strategies are usually expressed as: (9) In the following, π represents the random strategy. * Represents a deterministic strategy.

[0030] The state value function and action value function are (using the Bellman equation): (10) In the formula, V is the state value function, E is the expected value in the mathematical formula, Q is the action value function, S is the state set, A is the action set, γ represents the discount factor in reinforcement learning, the subscript π represents the corresponding item when using a specific strategy, the subscript t represents the corresponding item of the specific control cycle, and the Actor target network is used to estimate the action in the next state: (11) In the formula, the superscript ’ represents the next one, θ refers to the neural network parameters, and μ is the function that converts the probability of state occurrence under the corresponding neural network parameters into behavior. Since this function is only related to the actor network, it is used θ μ represents the actor network parameters, a refers to the action corresponding to the probability of the actor output, and this article refers to the APADRC control parameters.

[0031] π * =argmax π Q’ ( s',a ) (12) Actor loss function: (13)

[0032] in, θ Q are the critic network parameters.

[0033] Critic loss function: (14)

[0034] Figure 3 The Actor and Critic networks are divided into the current network and the target network. The Actor current network selects action a in the current state s and executes action a to obtain the new state s′ and reward r. Then the sample (s, a, R, s′) is put into the experience replay pool. When the experience replay pool accumulates to a certain amount, the Actor current network samples the next state s′, selects action a′, and inputs it into the Critic target network to calculate the Q value (value function value). The Critic target network is responsible for calculating the target Q value based on the experience replay pool and inputting it into the current network. The current network regularly updates the network parameters and selects the maximum Q value and the corresponding strategy to output to the Actor network. The motor state, ADRC coefficient, and reward arrays obtained from the linear motor control system are put into the experience replay pool. After a series of training, the Actor-Critic and its neural network in the Agent (intelligent body) find the optimal control coefficient and input it into the motor control system to form a closed loop.

[0035] like Figure 5 As shown in the figure, the traditional DRL is based on the Actor-Critic framework, which will cause the problem of overestimation of Q value. An inaccurate Q value will lead to wrong strategy selection, and as time goes by, the cumulative error will be large. Although the use of two critic networks can reduce the error, the effect is not very obvious. There will still be wrong strategies and reinforcement learning. The present invention proposes a double Q average back-inference algorithm based on the improved PSO to obtain the optimal network structure parameters. The value functions Q1 and Q2 brought by the control parameters are evaluated by the critic network and the loss function is calculated. L 1, L 2.

[0036] (15) The average of the two loss functions is: (16) By calculating the average Q mean Closer to Q target This ensures that the policy update is neither too aggressive, resulting in instability, nor too conservative, resulting in slow learning.

[0037] Figure 3 In the invention, the linear motor control system includes a linear motor body, a position sensor, a motor controller, a deep reinforcement learning controller, etc. As an intelligent agent, the deep reinforcement learning network interacts with the linear motor control system, receives state information and environmental disturbance information from the sensor, and obtains the coefficients of the next moment through training to control the linear motor. In the present invention, the entire linear motor control system is regarded as the environment (Environment), the motor output information is regarded as the state (State), and the motor control effect is regarded as the reward (Reward). The input layer of the intelligent agent receives the motor output current, speed and position of the linear motor and its error and environmental disturbance, and the output layer outputs the optimized APADRC controller parameters for the next cycle.

[0038] The control performance indicators of linear motors include dynamic performance and steady-state performance. Dynamic performance mainly includes speed overshoot σ, rise time t r and adjustment time t s , the steady-state performance mainly includes the speed steady-state error v error and position steady-state error x error ; Reward Function (17) In the formula p 1, p 2, p 3, p 4, p 5 is the reward function weight coefficient.

[0039] The improved DRLPSO back-stepping network strategy algorithm is as follows: Figure 5 As shown, the improved PSO performs strategy learning and selection on the Actor-Critic framework, based on the average value L mean Adjust the position and speed of the particles, and infer the current network strategy structure to more accurately approach the optimal control strategy. Each particle in the particle swarm algorithm represents a possible strategy, and the fitness of the particle is determined by the reward in the reinforcement learning environment. The particle swarm algorithm uses the average value of the loss function given by the Critic network Lmean To optimize the control strategy in reinforcement learning, in each control cycle, the improved PSO is used to search for the best strategy in all possible control sequences and apply it to the system. The two solutions interact iteratively until the optimal parameters of the control system are obtained.

[0040] In the conventional particle swarm algorithm, the initial position of each particle is randomly distributed, and the historical optimal position of each particle and the global optimal position of the entire particle swarm are recorded in each iteration. The traditional particle position X and velocity V are updated according to the following formula: (18) The traditional particle swarm weight coefficient formula is: (19) In the formula, w is the inertia weight, w start is the initial weight coefficient, w end is the final weight coefficient, t max is the number of iterations of the algorithm, t is the current iteration number. c 1 and c 2 is the learning factor, which is a non-negative constant; r 1 and r 2 is a random number distributed in the interval [0,1].

[0041] Traditional particle swarm optimization is prone to local optimality. In order to maintain a good balance between the global and local search capabilities of particles, the present invention adopts a partition inertia weight setting method, that is, the weight coefficient w It is divided into three parts. The first part is the reference value, the second part affects the particle speed, and the third part affects the particle position. That is: (20) In the formula, w is the inertia weight, w 0 is the base value, w v is the particle velocity weight value, w x is the particle position weight value. Among them: l is the ratio of the velocity function of the particle group at the previous moment to that at this moment, g It is the ratio of the position function of the particle swarm at the previous moment to that at this moment. The speed and position function values ​​corresponding to each time series are recycled, and the evolution speed and position of the particles are analyzed through the global optimal historical data, so as to achieve a good balance between global and local search capabilities.

[0042] Compiling the above method steps into a program and then storing it on a hard disk or other non-transitory storage medium constitutes an embodiment of "a non-transitory readable recording medium" of the present invention; and electrically connecting the storage medium to a computer processor to complete the adaptive control of the motor through data processing constitutes an embodiment of "a motor adaptive control system" of the present invention.

[0043] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computers, available storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer program code.

[0044] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0045] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0046] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0047] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A motor adaptive control method, characterized in that: The following steps are involved: A differential tracker is used to obtain a smooth differential signal from the original motor state signal containing random noise; the differential tracker is expressed as: ； ； In the formula, k 1 is a constant coefficient, x is the signal variable, x * is the signal given value, is the signal observation value, e 1 is the difference between the given value of the signal and the observed value of the signal; fal is a nonlinear function, m 1 is the nonlinear factor, n 1 is the filter factor, z 11 is the first-order derivative of the signal variable with respect to time; The extended state observer is used to observe the state of the motor control system and the total disturbance affecting the system output; the extended state observer is expressed as: ； In the formula, k2, k3 are constant coefficients, m2, m3 are nonlinear factors, n2, n3 are filter factors, b0 is the compensation coefficient, is the perturbed observation, z 21 is the first-order derivative of the signal observation value with respect to time; u is the control input variable; According to the received differential signal, the state of the motor control system and the total disturbance affecting the system output, the control amount is compensated by the error feedback control rate, and the error feedback control rate is expressed as: ； In the formula, f is the disturbance variable, k 4， k 5 is a constant coefficient, m 4， n 4 is the nonlinear factor, m 5， n 5 is the filter factor, b 0 is the compensation coefficient, u0 is the control input given value; the parameter optimization involved in the formulas of the differential tracker, extended state observer, and error feedback control rate adopts an artificial intelligence learning method and is set through multiple rounds of cyclic network iterative training.

2. A motor adaptive control method according to claim 1, characterized in that: The artificial intelligence learning method includes reinforcement learning and deep learning coupled to an intelligent agent. The reinforcement learning adopts a Markov decision process, in which the change relationship parameters of the state, action, reward function and state change probability involved are optimized through the iteration of the recurrent neural network of deep learning. In each control cycle, the optimization results of deep learning and reinforcement learning are interactively updated, and the finally obtained control parameters are input into the motor controller to control the motor.

3. A motor adaptive control method according to claim 2, characterized in that: The value function optimization is to optimize the mean of the value functions obtained based on evaluations of two groups of different dimensions.

4. A motor adaptive control method according to claim 3, characterized in that: A particle swarm optimization algorithm is used to optimize the neural network structure of the deep learning.

5. A motor adaptive control method according to claim 4, characterized in that: The weight coefficient of the particle swarm optimization algorithm is formed by adding three parts. The first part is the reference weight, the second part is the product of the ratio of the speed function of the adjacent moments and the speed weight, and the third part is the product of the ratio of the position function of the adjacent moments and the position weight.

6. A motor adaptive control method according to claim 5, characterized in that: The equation of motion for a linear motor with perturbations is as follows: ;in, F fri is the friction force, F load is the load force, F E is the electromagnetic force, F dis It is the external disturbance force. M is the mass of the mover; in the speed loop, the feedback control rate and the motor motion equation are combined to select the optimized artificial intelligence learning method k 3, k 4, k 5 Generate the q-axis current reference value in the motor control quantity: ; where the disturbance appears in the form of force F, the parameter with a superscript ^ represents the observed value of the parameter, and the parameter with a superscript * represents the reference value of the parameter, F 0 =F fri +F load ; Perturbed Observations ; The external disturbance is compensated to the motor current through the above formula to adapt to the smooth control of the motor in different environments, thereby reducing speed and thrust fluctuations.

7. A motor adaptive control method according to claim 6, characterized in that: The reward function is a calculation of the steady-state and dynamic performance of the linear motor. The dynamic performance includes speed overshoot σ, rise time t r and adjustment time t s , steady-state performance includes speed steady-state error v error and position steady-state error x error ; Reward function , where p 1, p 2, p 3, p 4, p 5 is the reward function weight coefficient.

8. A non-transitory readable recording medium for storing one or more programs including a plurality of instructions, characterized in that: When the instruction is executed, the processing circuit is caused to execute a motor adaptive control method according to any one of claims 1 to 7.

9. A motor adaptive control system, comprising a processing circuit and a memory electrically coupled thereto, characterized in that: The memory configuration stores at least one program, wherein the program includes a plurality of instructions, and the processing circuit runs the program to execute a motor adaptive control method according to any one of claims 1-7.

Images