Intelligent control method of permanent magnet synchronous motor servo system in semi-closed loop scenario

Abstract

The present invention discloses a control method for a permanent magnet synchronous motor drive servo system in a semi-closed loop scenario. The method targets a semi-closed loop control scenario in which only motor feedback signals such as the rotor angle of the permanent magnet synchronous motor can be measured, but the actual position of the load mechanism cannot be measured. Taking into account the high-order nonlinear characteristics of the load model, the method proposes an edge intelligent control algorithm driven by deep reinforcement learning to achieve high-precision signal tracking under the permanent magnet synchronous motor FOC control framework, uses a proximal strategy optimization algorithm to train and tune a strategy network, and integrates a traditional three-loop control strategy to enable it to observe the three-loop instructions and feedback quantities of the permanent magnet synchronous motor and output the control voltage of the motor, so as to improve the control accuracy of the traditional three-loop control when facing a high-order nonlinear load model, while ensuring safety during operation.

Description

Technical Field

[0001] The present invention belongs to the control technology of permanent magnet synchronous motors, relates to computer control system technology, and specifically to a position control algorithm of permanent magnet synchronous motors in the industrial Internet, and especially to an intelligent control method for a permanent magnet synchronous motor servo system in a semi-closed loop scenario. Background Art

[0002] Permanent magnet synchronous motors have been widely used in the fields of industrial Internet, electric transportation, industrial robots, aerospace, etc. due to their compact structure, high efficiency and power density, and good speed regulation performance. The field oriented control (FOC) strategy commonly used in permanent magnet synchronous motor position servo systems is often based on the PID three-loop controller to control the motor rotation angle. Although it has been widely used in industrial practice due to the low complexity and stability of the algorithm, the PID three-loop controller also faces many challenges and shortcomings when it faces some complex and special driving scenarios:

[0003] First, when the servo motor drives some high-order nonlinear load mechanisms, the parameters of the PID controller are difficult to determine through models and indicators, so it is necessary to rely on manual experience for trial and error adjustment, and it is difficult to have good performance in terms of dynamic response. Secondly, considering the motor system's speed and current limits and the inverter output voltage threshold limits, when the controller receives highly dynamic position, speed, and current instructions, the limit of the instructions will further reduce the performance of the controller. The most challenging thing is that the special equipment facing the present invention cannot install external sensors or reliably measure the actual position of the load mechanism due to the special working environment, and the feedback obtained by the controller is only the motor shaft angle obtained by the magnetic encoder detection, that is, a semi-closed loop control system. Compared with the closed-loop control that monitors the final actuator, the semi-closed loop scenario will cause a large performance degradation in the traditional PID controller that is controlled based on feedback errors. For example, the present invention considers that the motor shaft drives the screw to feed or retract through the gear, thereby causing the swing mechanism connected to the triangular connecting rod structure to deflect a certain angle. The swing angle of the swing mechanism cannot be measured by sensors, and the motor rotation angle and the swing angle of the swing mechanism are affected by the elastic motion of the screw rod, involving high-order nonlinear dynamic equations, which are difficult to express simply with functions. Therefore, the control algorithm cannot obtain actual angle feedback to form a closed-loop control.

[0004] The prior art also includes a method for dynamic tuning of electric servo position feedback based on deep reinforcement learning in a semi-closed loop scenario (publication number CN117335700A). Based on the PID three-loop control, this technology determines the state space and action settings of the reinforcement learning agent, and then uses simulation software to model the system model and use deep reinforcement learning methods to obtain the tuning strategy network of the motor position feedback to output the optimal tuning value by improving the tracking accuracy and response speed of the load swing angle to the command target angle under semi-closed loop control. Taking this scheme as an example, it can be learned that the three-loop control scheme usually adds a limit to the output position of each loop to ensure control safety under extreme conditions. However, this will also limit the role of the agent, because once one of the three loops reaches the limit, the agent cannot optimize the original three-loop control scheme through tuning. Summary of the invention

[0005] Purpose of the invention: In view of the shortcomings and problems of the above-mentioned PID three-loop control method when facing high-order nonlinear loads in the semi-closed loop situation of permanent magnet synchronous motors in the industrial Internet, the present invention provides an industrial Internet servo system control method driven by deep reinforcement learning in a semi-closed loop scenario.

[0006] Technical solution: An intelligent control method for a permanent magnet synchronous motor servo system in a semi-closed loop scenario, which includes constructing a permanent magnet synchronous motor operation model based on the FOC control framework and a mathematical model of the load and its transmission mechanism, and then using a three-loop structure as the basis to determine the state space and action settings using a reinforcement learning agent, with the goal of improving the tracking accuracy and response speed of the load swing angle to the command target angle in a semi-closed loop scenario, using simulation software to model the system model and using a deep reinforcement learning method to obtain a tuning strategy network for the motor position feedback to output the optimal control voltage;

[0007] The method also pre-trains the model through empirical data collected by the sensor during the test process, and then optimizes the policy network based on the double-delay depth determination policy gradient algorithm, so that the policy network can use the real-time observable feedback value or computable value of the motor as the state quantity, and thereby calculate the optimal PMSM control voltage, and the permanent magnet synchronous motor drives the transmission device to drive the servo system to respond to the instruction;

[0008] In the above process of using reinforcement learning agent to determine the state space and action settings, the anti-disturbance algorithm is used in the position loop to improve the anti-disturbance ability of the control system, PI control is used in the speed loop, and the reinforcement learning algorithm is used in the current loop to train the strategy network. By observing the feedback position of the position loop input and the reference speed of the output, the feedback speed of the speed loop input and the reference current and feedback current of the output, the decision output voltage applied to the PMSM is made to improve the system performance, and the PPO reinforcement learning algorithm is used for agent optimization;

[0009] This method uses the PPO reinforcement learning algorithm with continuous state and action space to optimize the intelligent agent, so that it observes the gap between the current PMSM state and the given reference value, and predicts its difference from the actual screw precession length θ when only relying on the approximate screw precession length as input. loc The deviation between the position feedback error and the control accuracy is alleviated by outputting the voltage to control the PMSM, so as to optimize the error between the system and the given command during the motor control process and alleviate the problem of insufficient control accuracy caused by the position feedback error.

[0010] Furthermore, the implementation steps of the method include:

[0011] S1. Construct a permanent magnet synchronous motor operation model based on the FOC control framework. The FOC strategy of the permanent magnet synchronous motor decomposes the phase variable of the motor into magnetic field component and torque component, and controls them independently to achieve precise control of the motor magnetic field and torque;

[0012] Under FOC control, the motor system takes the voltage command values ​​of the d and q axes from the controller as input, and inputs the dynamic torque to the load mechanism, which is expressed as the rotation angle of the motor rotor;

[0013] S2. Construct a mathematical model of the load and its transmission mechanism. The motor load model takes into account the elastic deformation of the transmission mechanism, the dynamic equation of the swing mechanism, and also considers nonlinear factors such as Coulomb friction and torque transmission in the triangular linkage mechanism;

[0014] S3. Considering the PMSM and high-order nonlinear load models in the above modeling, a deep reinforcement learning-driven edge intelligent control solution is constructed on the FOC control framework;

[0015] In the control scheme, the ADRC controller receives the external input reference value of the controlled object, and outputs the reference value of the position control and its rate of change through the differential controller; at the same time, the ADRC controller receives the feedback of the measurement element of the controlled object. Since there is a certain amount of noise and oscillation in the time domain during the measurement of the feedback, these will affect the correct output of the subsequent deep reinforcement learning agent. Therefore, the ADRC controller first inputs the feedback signal into the state expansion observer to predict the noise contained therein, and uses the corresponding feedforward mechanism to remove it; finally, the ADRC controller will make a difference between the outputs of the differential controller and the state expansion observer to output the reference value of the next loop;

[0016] The PI controller makes a difference between the reference value of the ADRC and the feedback value of the measuring element, and linearly superimposes the difference and the integral value over time as the output control quantity to the next actuator;

[0017] S4. Build a permanent magnet synchronous motor control solution driven by deep reinforcement learning, use PPO for training, and the intelligent agent outputs the control voltage of the PMSM to drive the transmission device to drive the servo system to move, so as to minimize the long-term error between the reference command and the servo system;

[0018] The value observed by the agent is determined as the differential controller output θ of the ADRC v1 (t),θ v2 (t), the output of the reference signal generator e1(t), e2(t), the command deflection angle α * (t) The converted screw precession length With feedback The error e θ (t), reference speed ω * (t) and reference current And the corresponding speed feedback ω(t) and current feedback i q (t).

[0019] Considering that although the ADRC controller can predict the noise in the feedback signal and use feedforward to compensate for the impact of the noise, the aforementioned agent observations still have different degrees of oscillation in the time domain. This oscillation mainly comes from the fast changing speed ω and current i q , which will cause the output of the agent to oscillate; in addition, even if the input itself does not oscillate, the output of the reinforcement learning agent itself will also oscillate in the time domain, because the agent aims to minimize the reward function and ignores the potential safety hazards that may occur when used as the control voltage of the PMSM. Therefore, it is necessary to provide feedback quantities ω and i q A Kalman filter is added and a low-pass filter is added to the agent output to smooth the control voltage of the PMSM.

[0020] Based on the above scheme, the method includes the following optimization process:

[0021] In order to better optimize the training algorithm, the input value is dimensionalized and limited. At the same time, the two feedback quantities ω and i q Add a Kalman filter and add a low-pass filter to the output of the agent;

[0022] In order to comprehensively improve the performance of the controller, the tasks faced by the reinforcement learning controller should be as rich as possible to include all possible command patterns, including random step commands, low-frequency sine commands, and high-frequency sine commands;

[0023] In order to reduce the swing mechanism angle α and the command angle α ref The error is calculated by taking the square of the difference between the reference value of each ring and the feedback value of each ring and taking the opposite number as the reward value for each time step, and multiplying it by a fixed coefficient to balance the weights between the three rings:

[0024]

[0025] Status Using the optimization strategy function Enables the agent to output tuning values To this end, it is necessary to establish two Q networks to evaluate the value of the output of the policy network and use the evaluation value to perform gradient optimization on the actor network.

[0026] Further, in step S1, under FOC control, the motor system takes the voltage command values ​​of the d and q axes from the controller as input, and inputs the torque to the load mechanism, which is expressed as the rotation angle of the motor rotor; the specific operation is as follows:

[0027] S11, obtaining the rotor position and speed information of the motor for use in the transformation calculation between the phase variable in the stationary coordinate system and the relative rotation variable of the dq coordinate;

[0028] In the dq coordinate system, the motor equation can be described as:

[0029]

[0030] Where: R s is the stator resistance; v d , v q are d-axis and q-axis voltages respectively; i d ,i q are d-axis and q-axis current respectively; L d , L q are the d-axis and q-axis inductances respectively; m is the d-axis magnetic flux of the permanent magnet; T e , T L are the output torque and load torque respectively; B is the bearing viscosity coefficient; J is the total moment of inertia of the motor and the load; ω m ,ω r are the rotor mechanical angular velocity and the rotor electromagnetic angular velocity respectively, p is the number of permanent magnet logarithms, satisfying ω r =p×ω m ; Input dq axis voltage, according to the external load torque T L , output motor angle θ m and angular velocity ω r The dq axis current is detected by the current detector;

[0031] S12. According to actual needs, the motor type is determined to be a surface-mounted PMSM. Considering that the permanent magnet synchronous motor operating in a steady state must be kept under certain operating limits, the method requires that the rotor speed and the stator current should be kept within a threshold range, satisfying:

[0032] L q =L d

[0033] |ω r |≤ω limit

[0034]

[0035] Among them, ω limit is the maximum rotor speed; i limit is the maximum value of stator current.

[0036] Furthermore, step S2 specifically includes:

[0037] For the high-order load mechanism with nonlinear factors, the motor shaft torque is transmitted to the screw by gears, and the triangular connecting rod mechanism is pushed and pulled by the screw stroke to form a force arm, which pushes the swing mechanism to deflect the angle. In the process of modeling, in order to fit the mechanical properties of the actual load, the elastic motion of the screw, the motion equation of the swing mechanism and the nonlinear Coulomb friction during the motion are considered;

[0038] The dynamic equation of this load can be described as:

[0039]

[0040] Among them, θ m is the rotor mechanical angle, g r is the gear reduction coefficient, n r is the screw reduction coefficient, L is the screw retraction / extension, K s is the combined rotation stiffness, F is the force exerted on the screw by the torque; M e is the mass of the screw, B e is elastic damping, ΔL is the screw compression stroke; T L is the motor shaft load torque, effi is the transmission efficiency; M is the swing torque, K p is the combined precession stiffness, r is the length of the lever arm; α is the swing angle of the swing mechanism, J b is the swing inertia, B b is the swing damping, K delta is the position resistance torque, M f is the friction torque, which is modeled as Coulomb friction and is expressed as follows:

[0041]

[0042] The above load model takes into account the elastic deformation of the screw. If we only consider the geometric relationship of the triangular linkage mechanism, we can form an equivalent triangular structure between the screw side, the swing mechanism and the fixed fulcrum; without considering the elastic deformation of the screw, the relationship between the retraction / extension amount L of the screw and the deflection angle α can be approximately expressed as:

[0043]

[0044] Among them, a and b are the lengths of two adjacent sides OA and OB of the swing angle in the triangular linkage mechanism; α0 and L0 are the swing center angle ∠AOB and the length of side AB when the load deflection angle is 0, respectively.

[0045] Furthermore, step S3 uses three-loop control to perform basic control on the final position angle for the semi-closed-loop electric servo system under the FOC control framework to ensure the stability and robustness of the system operation, wherein:

[0046] S31. The current loop for controlling the torque is constructed as a decoupled current controller, which decomposes the d and q axis voltage terms that are originally coupled to each other into linear terms and nonlinear terms:

[0047]

[0048] Among them, v d1 and v q1 can be controlled by a linear PID current controller, while the linear term v d0 and v q0 It can be calculated from the rotor speed value of the encoder:

[0049]

[0050] S32. The relationship between the input error e(t) and the output control value u(t) in the PID controller:

[0051]

[0052] In S33 and ADRC controllers, the differential controller input θ(t) and output θ v1 (t),θ v2 The relationship between (t) is:

[0053]

[0054] in, Represents θ v1 (t) the derivative with respect to time t;

[0055] S34, ADRC controller, the state expansion observer θ(t) and the output θ z1 (t),θ z2 (t),θz3 The relationship between (t) is:

[0056]

[0057] Among them, fal(e(t),α,δ) is the error filter, and the relationship between its input e(t),α,δ and output is:

[0058]

[0059] In S35, ADRC controller, the input of the reference signal generator comes from the output θ of the differential controller v1 (t),θ v2 (t) and the output θ of the state extended observer z1 (t),θ z2 (t),θ z3 (t), the specific relationship is:

[0060]

[0061] Among them, sat(x,x max ) represents a saturation function. When the input x is greater than x max The output is x max , when x is less than -x max The output is -x max , otherwise the output is x;

[0062] S36, the feedback motor speed ω(t) and speed command ω * (t) The difference is used as the input of the speed PI controller, and the output of the controller is used as the reference value of the q-axis current

[0063] S37, based on the speed controller, build a position control loop, specifically based on the load-based triangular connecting rod structure, deflect the command angle α ref Approximate conversion to screw rod precession length Input the differential controller of ADRC and convert it to the precession length of the rotor angle feedback signal θ(t) The state expansion observer of ADRC is input, and the reference speed output by the ADRC reference signal generator is used as the output of the position loop.

[0064] It is further pointed out that the specific steps of the method of the present invention for alleviating the problem of insufficient control accuracy caused by position feedback error include:

[0065] S41, the value observed by the agent is determined as the differential controller output θ of ADRC v1 (t),θ v2(t), the output of the reference signal generator e1(t), e2(t), the command deflection angle α ref The converted screw precession length With feedback The error e θ (t), reference speed ω * (t) and reference current And the corresponding speed feedback ω(t) and current feedback i q (t), and transform its value to determine the state space s of the agent as:

[0066]

[0067] S42, using the output continuous action a of the intelligent body as the q-axis voltage input of the PMSM, so that the PMSM generates an electromagnetic torque under the drive of a given voltage, and drives the servo system to respond to a given instruction;

[0068] S43, in order to improve the comprehensive performance of the controller, random step instructions, low-frequency sine instructions, and high-frequency sine instructions are mixed for training. Each Episode selects a task as an instruction with equal probability and lasts for 10 seconds;

[0069] S44, to reduce the swing mechanism angle α and the command angle α ref The error is calculated by taking the inverse of the square of the difference between the command and the reference value of each loop in the three-loop control as the reward value for each time step:

[0070]

[0071] S45. Considering the deployment requirements on embedded chips, the actor is set to an MLP with 2 hidden layers; the critic network is set to an MLP with 4 hidden layers; during training, the agent sampling time is set to 0.01s.

[0072] Beneficial effect: The method described in the present invention is aimed at the semi-closed-loop control situation where only the motor feedback signals such as the rotation angle of the permanent magnet synchronous motor can be measured, but the actual position of the load mechanism cannot be measured. Taking into account the high-order nonlinear characteristics of the load model, it is proposed to use the traditional PID three-loop controller as the basis under the permanent magnet synchronous motor FOC control framework, and use the double-delay deep determination strategy gradient algorithm to train the tuning strategy network, so that it observes the permanent magnet synchronous motor feedback quantity and outputs the position loop feedback position tuning value, so as to improve the control accuracy and response speed of the traditional PID three-loop control when facing a high-order nonlinear load model. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 It is the overall control process model structure diagram of the present invention;

[0074] Figure 2 It is a Simulink modeling diagram of the permanent magnet synchronous motor in the present invention;

[0075] Figure 3 It is an equivalent triangle structure diagram formed between the screw rod side, the swing mechanism and the fulcrum in the present invention;

[0076] Figure 4 It is a Simulink modeling diagram of the load mechanism in the present invention;

[0077] Figure 5 It is a structural diagram of a decoupling current controller based on PID control in the present invention;

[0078] Figure 6 It is the structure diagram of the speed loop and position loop based on the PID controller in the present invention;

[0079] FIG7 is a response curve diagram of the q-axis current controlled by the current controller to a step current command when the voltage limit is not considered ( FIG7( a ) ) and when the voltage limit is used ( FIG7( b ) ) in the present invention;

[0080] FIG8 is a schematic diagram showing the error convergence between the command angle precession distance and the approximate precession distance of the rotation shaft position in the position loop of the present invention ( FIG8(a) ), and the phenomenon that the actual angle of the swing mechanism and the command angle differ greatly ( FIG8(b) );

[0081] Fig. 9 It is a control logic diagram of the intelligent agent as a PMSM input voltage controller in the present invention, wherein Uq is the control system response instruction of the intelligent agent as a PMSM input voltage controller;

[0082] FIG10 is a schematic diagram of the phenomenon of the oscillation of the output value of the agent (FIG. 10(a)) and the oscillation of the motor state quantity observed by the agent (FIG. 10(b)) in the present invention;

[0083] FIG11 is a comparison of the effects of the PID scheme, the ADRC scheme and the reinforcement learning scheme in the present invention under different instructions (FIG. 11(a)-(d)). DETAILED DESCRIPTION

[0084] In order to illustrate the technical solution disclosed by the present invention in detail, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0085] First of all, the key problem to be solved by the method described in the present invention is how to improve the control algorithm to improve the tracking accuracy of the actual position of the load to the command position in a semi-closed loop control situation where only motor parameters such as the motor rotation angle can be measured but the actual position of the high-order nonlinear load mechanism cannot be measured when the permanent magnet synchronous motor is performing three-loop position control.

[0086] The main design idea of ​​the present invention is to use data-driven deep reinforcement learning, use empirical data that can be collected by sensors during the test process but cannot be observed during the actual operation process to pre-train the model, and use a double-delayed deep deterministic policy gradient algorithm to optimize a policy network, so that the policy network can use the real-time observable feedback value or computable value of the motor as the state quantity, and thereby calculate the optimal PMSM control voltage, and the permanent magnet synchronous motor drives the transmission device to drive the servo system to respond to the instruction. The overall control process provided by this method is as follows Figure 1 shown.

[0087] A construction and training process of an intelligent control method for a permanent magnet synchronous motor servo system in a semi-closed loop scenario includes the following steps:

[0088] Step 1: Construct a mathematical model of permanent magnet synchronous motor under FOC framework

[0089] The rotor in a permanent magnet synchronous motor is composed of a rotor core and permanent magnets arranged around the core. Regardless of the rotor configuration, the magnetic flux (magnetic induction intensity) density distribution generated by its paired magnetic poles in the air gap is similar. In physical modeling, it is usually assumed that the magnetic flux density distribution generated by the permanent magnetic poles installed on the rotor surface or buried in the core in the motor air gap is sinusoidal. Therefore, the fundamental wave of the magnetic flux density curve is taken as the ideal magnetic flux density distribution. At the same time, for this sinusoidal magnetic flux density signal, its coordinate axis is defined as the magnetic field angle θ r Define the number of permanent magnet pairs installed on the rotor core as p, and the mechanical angle of rotor rotation θ m The relationship with the corresponding magnetic field angle is:

[0090]

[0091] At the same time, the polar axis of a permanent magnet (the extreme point of the sine wave) is defined as the d-axis. The q-axis is located between the two magnetic poles, with a 90° difference from the d-axis magnetic field angle, and the magnetic flux is 0.

[0092] The configuration of the rotor leads to the classification of permanent magnet synchronous motors into two categories: salient pole motors and non-salient pole motors. Salient pole motors have internal magnets, while non-salient pole motors have surface mounted magnets. The difference between salient pole motors and non-salient pole motors is of interest because the magnetic permeability of permanent magnets is almost the same as free air, while the magnetic permeability of the iron core is much higher than that of air (ferromagnetic).

[0093] According to Abe's law, the magnetic induction intensity at a point in space is proportional to the magnetic permeability of that point. Therefore, considering the magnetic field of a constant magnetic field strength generated by a energized spiral tube: when the rotor rotates, no matter which direction the surface-mounted motor rotates, the radial length of the iron core through which the magnetic field lines pass is the same, that is, the magnetic resistance on the magnetic circuit is the same; while for the salient pole motor, when the rotor rotates to the d-axis, the magnetic circuit passes the least iron core and the most magnetic resistance; and when it reaches the q-axis, the iron core is the most, the magnetic resistance is the least, and there is an uneven magnetic air gap. This phenomenon is called magnetic salient polarity.

[0094] The stator winding of the motor is essentially an overlapping, energized solenoid in different directions. The coils are distributed in the stator slots outside the stator core with a 120° displacement, and are named A, B, and C phase windings. In actual circuits, the tails of the ABC three-phase windings are usually connected to form a triangle connection circuit. In this case, there are:

[0095] i a +i b +i c =0

[0096] For the above three-phase motor windings, three-phase alternating current with a phase difference of 120° is applied respectively, and the three-phase sinusoidal time-varying variables can synthesize a rotating magnetic field in space.

[0097]

[0098] For a rotating rotor, if the rotating magnetic field can keep the same speed as the rotor and the magnetic phase is constant, the interaction between the magnetic fields can produce a constant torque, namely the magnetic torque. For motors with salient polarity, another torque, namely the reluctance torque, is generated, which drives the rotor to rotate together with the load.

[0099] The direction of the square magnetic field generated by the abc three-phase winding is used as the coordinate axis direction to define the abc reference coordinate system. In this coordinate system, the phase variable in the time domain can be expressed as f a 、f b 、f c , where f can represent phase voltage, phase current and flux. Considering Faraday's law of electromagnetic induction and Ohm's law, the three-phase voltage can be expressed as:

[0100]

[0101] According to the above discussion on the rotor, for the surface-mounted rotor motor, the magnetic permeability of the self-inductance and mutual inductance of each stator coil remains unchanged, and the relationship between the self-inductance coefficient and the mutual inductance coefficient of any configuration of the rotor with respect to the magnetic angle is:

[0102]

[0103] Among them, for surface-mounted motors, L2 = 0. Therefore, the flux value of the three-phase winding of abc is the self-inductance and mutual inductance flux plus the flux leakage of the permanent magnet into the coil:

[0104]

[0105] where λ m is the maximum magnetic flux from the permanent magnet N pole to a coil. Substituting the above two equations into the equation, we get:

[0106]

[0107] The motor model in the above stationary reference frame has the problem of time-varying parameters, which complicates the control system design. This control complexity caused by rotation can be solved by projecting the phase variables of the model into a two-term model in the rotating reference frame. The dq coordinate system has two orthogonal axes fixed to the rotor, namely the d-axis of the permanent magnet pole of the above rotor and the q-axis orthogonal to it.

[0108] Consider transforming the motor model from the three-phase stationary abc reference coordinate system to the two-phase rotating coordinate system: First, we need to know the angle θ of the dq axis relative to the stationary abc coordinate system r , then the three-phase variables are projected onto the d axis to obtain f d , projected onto the q axis to obtain f q Mathematically, the transformation process can be solved using Park transformation, and its matrix form is as follows:

[0109]

[0110] The coefficient This is to ensure that the magnitude of the transformation remains equal after the transformation, and because for the phase variable:

[0111] f a +f b +f c =0

[0112] Therefore, the above transformation matrix is ​​reversible after adding this constraint. The transformation from the dq reference coordinate system to the abc reference coordinate system can be achieved by the following formula:

[0113]

[0114] where f oThe FOC control transforms the three-phase rotating magnetic field into the relative rotation change of the rotor dq axis through Park change. When the motor starts or is suddenly loaded, when the abc three-phase variables are unbalanced sinusoidal signals, the dq axis variables are generally time-varying models; when the motor is running in a steady state, the rotating magnetic field created by the phase variables remains relatively static with the rotor, and the dq axis variables become some DC signals. In this case, controlling the dq axis variables is equivalent to controlling two equivalent solenoids that are always aligned or perpendicular to the magnetic axis, and the corresponding motor control algorithm becomes relatively simple.

[0115] Transforming the above equation by Park change, the voltage equation of the rotor reference system is as follows:

[0116]

[0117] where v d and v q are the stator voltages of the d-axis and q-axis respectively; i d and i q are the stator currents of the d-axis and q-axis respectively; d and λ q They are the stator flux of d-axis and q-axis respectively, and their values ​​are:

[0118]

[0119] Where L d and L q are the d-axis and q-axis inductances, respectively, and λ m is the d-axis magnetic flux facing the magnetic pole:

[0120]

[0121] In the dq coordinate system, the expressions of the self-inductance and mutual inductance coefficients of each phase that originally changed with rotation become constants. Combining the two equations, we get the motor current-voltage equation:

[0122]

[0123] Among them, v d , v q are the system input quantities (control quantities), and their input determines the current and torque. In the rotor coordinate system, the instantaneous input power of the permanent magnet synchronous motor during operation is:

[0124]

[0125] in, This is to make up for the multiplication factor when doing Park transformation. Note that R s i dThe term is the resistance voltage drop term and does not contribute to the final motor output power; The term is the magnetic field voltage drop term. This electrical energy is stored in the magnetic field and therefore does not contribute to the final motor output power. Therefore, the actual value of the electrical energy converted into mechanical power is:

[0126]

[0127] According to the torque theorem:

[0128] M=P / ω

[0129] The electromagnetic torque is:

[0130]

[0131] The permanent magnet synchronous motor is connected to a mechanical load. The dynamics of the mechanical part of the motor is described by the following equation:

[0132]

[0133] Where T L is the load torque, B is the motor bearing viscosity coefficient, and J is the total moment of inertia of the motor and the load. Finally, the permanent magnet synchronous motor as a whole can be built into a simulink submodule for easy calling, such as Figure 2 As shown. The module's corresponding input voltage of the dq axis is based on the external load torque T L , output the motor angle and angular velocity, and obtain the dq axis current through the current detector.

[0134] Based on the above analysis, for the actual motor model, the present invention constructs a permanent magnet synchronous motor operation mathematical model based on the FOC control framework and implements it in Simulink.

[0135] Under FOC control, the motor system takes the voltage command values ​​of the d and q axes from the controller as input, and inputs the torque to the load mechanism, which is expressed as the rotation angle of the motor rotor.

[0136] The FOC strategy of permanent magnet synchronous motor decomposes the phase variable of the motor into magnetic field component and torque component, and controls them independently to achieve precise control of the motor magnetic field and torque. It needs to obtain the rotor position and speed information of the motor for the transformation calculation between the phase variable in the stationary coordinate system and the relative rotation variable in the dq coordinate system. In the dq coordinate system, the motor equation can be described as:

[0137]

[0138] Where: R s is the stator resistance; v d , v qare d-axis and q-axis voltages respectively; i d ,i q are d-axis and q-axis current respectively; L d , L q are the d-axis and q-axis inductances respectively; m is the d-axis magnetic flux of the permanent magnet; T e , T L are the output torque and load torque respectively; B is the bearing viscosity coefficient; J is the total moment of inertia of the motor and the load; ω m ,ω r are the rotor mechanical angular velocity and the rotor electromagnetic angular velocity respectively, p is the number of permanent magnet logarithms, satisfying ω r =p×ω m The module responds to the input voltage of the dq axis according to the external load torque T L , output motor angle θ m and angular velocity ω r The dq axis current is detected by the current detector.

[0139] According to actual needs, the motor type is determined to be a surface-mounted PMSM. At the same time, considering that the permanent magnet synchronous motor operating in a steady state must be kept under certain operating restrictions, the present invention requires that the rotor speed and stator current should be kept within a threshold range, that is, satisfying:

[0140] L q =L d

[0141] |ω r |≤ω limit

[0142]

[0143] Among them, ω limit is the maximum rotor speed; i limit is the maximum value of stator current.

[0144] Step 2: Construct a mathematical model of the load and its transmission mechanism.

[0145] In combination with actual needs, a high-order load mechanism with nonlinear factors is determined. The mechanism uses gears to transmit the torque of the motor shaft to the screw, and pushes and pulls the triangular connecting rod mechanism through the screw stroke to form a force arm, which pushes the swing mechanism to deflect the angle. In the process of modeling, in order to fit the mechanical properties of the actual load, the elastic movement of the screw, the motion equation of the swing mechanism, and the nonlinear Coulomb friction during the movement are considered. The dynamic equation of this load can be described as:

[0146]

[0147] Among them, θm is the rotor mechanical angle, g r is the gear reduction coefficient, n r is the screw reduction coefficient, L is the screw retraction / extension, K s is the combined rotation stiffness, F is the force exerted on the screw by the torque; M e is the mass of the screw, B e is elastic damping, ΔL is the screw compression stroke; T L is the motor shaft load torque, effi is the transmission efficiency; M is the swing torque, K p is the combined precession stiffness, r is the length of the lever arm; α is the swing angle of the swing mechanism, J b is the swing inertia, B b is the swing damping, K delta is the position resistance torque, M f is the friction torque, which is modeled as Coulomb friction and is expressed as follows:

[0148]

[0149] The above load model takes into account the elastic deformation of the screw rod. If we only consider the geometric relationship of the triangular linkage mechanism, the equivalent triangular structure formed by the screw rod side, the swing mechanism and the fixed support point can be expressed as Figure 3 Without considering the elastic deformation of the screw, the relationship between the retraction / extension amount L of the screw and the deflection angle α can be expressed as

[0150]

[0151] Among them, a and b are the lengths of two adjacent sides OA and OB of the swing angle in the triangular linkage mechanism; α0 and L0 are the swing center angle ∠AOB and the length of side AB when the load deflection angle is 0, respectively.

[0152] The overall load is modeled as a high-order nonlinear system. There is a complex differential equation relationship between the load swing angle and the motor observable shaft angle, which makes it difficult for the traditional error-based closed-loop controller to obtain good accuracy and response speed performance in this scenario, leaving room for optimization for data-based artificial intelligence algorithms.

[0153] Build an equivalent load module in Matlab / Simulink to implement the above equations, such as Figure 4 As shown. This module converts the motor rotor angle θ m As input, the output is fed back to the motor load torque T L and control amount α.

[0154] Step 3: Build a basic three-ring controller and solution

[0155] In the present invention, the PID controller makes a difference between the external input reference value of the controlled object and the feedback value of the measuring element, and linearly superimposes the difference, the integral value over time and the differential value as the output control quantity to the next actuator. The relationship between the input error e(t) and the output control value u(t) is:

[0156]

[0157] Where K p , K i , K d It is the proportional coefficient of the three phases, which can be adjusted manually or automatically through parameter setting or other optimization algorithms.

[0158] In the FOC framework, vector control is achieved by controlling i d and i q The method of controlling torque by current. Therefore, vector control is related to the innermost control in the motor drive system, and subsequent speed and position control should be carried out on the basis of current control. The surface-mounted permanent magnet synchronous motors considered in the invention are:

[0159]

[0160] The electromagnetic torque is i q A linear function of d There is no effect on the torque, and any work done by the d-axis current will result in a waste of input power (mainly dissipated in resistance and magnetic field). q Control the torque while maintaining i d =0, the maximum torque-to-current ratio control (MTPA) is realized, that is, the maximum torque output can be achieved under any stator current.

[0161] In the rotor reference frame, the motor model is subject to the speed voltage term (i.e., ω r L q i q and ω r L d i d +ω r λ m ) cross-coupling effect. Especially at high speeds, this term will dominate the voltage equation. This will actually weaken the performance of the PI controller, so a decoupling circuit is needed as a current control solution for vector control. In order to make i d and i q The control linearization can be provided by the combination of the following two signals:

[0162]

[0163] Among them, v d1 and v q1 It can be controlled by linear PI current control, while the linear term v d0 and v q0 It can be calculated from the rotor speed value of the encoder:

[0164]

[0165] Based on the above decoupling current controller, the feedback motor speed ω fb and speed command ω ref The difference is used as the input of the speed PI controller, and the output of the controller is used as the reference value of the q-axis current. At the same time, pay attention to the speed limit of the system and set the motor speed reference value ω ref Limited to (-ω limit ,ω limit ) range. Based on the speed controller, a position control loop is constructed. Considering the triangular connecting rod structure of the load, the deflection angle α of the command swing mechanism is ref , can be approximately converted to the screw precession length using the above cosine formula And compare it with the rotor angle feedback signal θ fb Conversion precession length L fb The error calculation is used as the input of the position loop PI controller, and the speed loop reference value is used as the output.

[0166] The Simulink modeling of the decoupled current control in the above three-loop control is as follows Figure 5 As shown, the modeling of the position loop and the speed loop is as follows Figure 6 As shown. The PMSM three-loop control model is a linear control for controlling the actual swing mechanism angle. While ensuring the system robustness, it can meet certain requirements in terms of accuracy and response speed. It can also ensure that the motor current and speed values ​​will not exceed the system limit. However, the above three-loop control process does not perform well in the final control performance, which is mainly caused by the following problems:

[0167] On the one hand, the equivalent voltage of SVPWM output in the actual system cannot exceed the inverter limit (220V), so it is necessary to Figure 5The output value of the decoupling current controller shown in FIG. 1 is limited, and the limiting function destroys the linearity of the linear term of the decoupling current controller. FIG. 7 shows the response curve of the q-axis current controlled by the controller to the step current command reference value before and after the threshold limit is considered. It can be observed that under the same controller parameter configuration, the q-axis current without threshold limit can track the command size at a faster speed, while after the threshold limit is added, the q-axis current curve is highly irregular, with a large overshoot and a long convergence time. This nonlinearity reduces the response speed of the current loop to a certain extent, and increases the risk of the actual current exceeding the system limit, reducing the performance and reliability of the controller.

[0168] On the other hand, the system cannot observe the actual angle of the swing mechanism and can only use the command angle α ref The rotor position feedback θ is approximately converted into the screw process and Without considering the complex differential equation relationship between the motor rotor angle and the swing mechanism angle, when the system faces a low-frequency signal, the position loop error is much larger than the actual swing angle, as shown in Figure 8.

[0169] Step 4: Build a deep reinforcement learning-driven permanent magnet synchronous motor control solution

[0170] Using PPO for training, the agent directly outputs the PMSM control voltage to drive the transmission device to drive the servo system to move, so as to minimize the long-term error between the reference curve and the actual state of the servo system. The advantage of using the reinforcement learning agent directly as the PMSM voltage controller is that it can maximize the adaptability of the intelligent model in the control process and avoid the three-loop control limiting the output of the agent; the three-loop control scheme usually adds a limit to the output position of each loop to ensure the control safety under extreme conditions, but this will also limit the role of the agent, because once one of the three loops reaches the limit, the agent cannot optimize the original three-loop control output through tuning. The only limitation of the agent as a PMSM voltage controller is that its output should be within the voltage range that the PMSM can withstand, which has stronger flexibility and adaptability than the agent as a feedforward compensator. However, since the reinforcement learning agent itself aims to maximize the reward function, its output has oscillations in the time domain; in addition, since the state quantity of the agent includes rapidly changing speed and current, if the noise and oscillation are not processed, the oscillation of the agent will be further aggravated, which will lead to unsatisfactory control effect and even cause control safety problems. Therefore, when the agent is directly used as a PMSM voltage controller, it is necessary not only to use an ADRC controller with anti-noise interference ability in the position loop to suppress the noise in the feedback signal, but also to use a Kalman filter for the rapidly changing speed and current, and a low-pass filter for the agent output to maximize the suppression of the oscillation of the agent output in the time domain.

[0171] The value observed by the agent is determined as the differential controller output θ of the ADRC v1 (t),θ v2 (t), the output of the reference signal generator e1(t), e2(t), the command deflection angle α * (t) The converted screw precession length With feedback The error e θ (t), reference speed ω * (t) and reference current And the corresponding speed feedback ω(t) and current feedback i q (t).

[0172] In order to better optimize the training algorithm, the input value is dimensionalized and limited. At the same time, the two feedback quantities ω and i q Add a Kalman filter and add a low-pass filter to the output of the agent.

[0173] In order to comprehensively improve the performance of the controller, the tasks faced by the reinforcement learning controller should be as rich as possible to include all possible command patterns, including random step commands, low-frequency sinusoidal commands, and high-frequency sinusoidal commands.

[0174] In order to reduce the swing mechanism angle α and the command angle α ref The error is calculated by taking the square of the difference between the reference value of each ring and the feedback value of each ring and taking the opposite number as the reward value for each time step, and multiplying it by a fixed coefficient to balance the weights between the three rings:

[0175]

[0176] Status Using the optimization strategy function Enables the agent to output tuning values To this end, it is necessary to establish two Q networks to evaluate the value of the output of the policy network and use the evaluation value to perform gradient optimization on the actor network.

[0177] At the same time, the agent sampling time is set to 0.01s.

[0178] Specifically, the present invention includes the following considerations:

[0179] 1) The present invention installs the agent to the voltage input terminal of the permanent magnet synchronous motor, and the agent driven by reinforcement learning directly controls the input voltage. Since the input voltage is the q-axis voltage, the output limit of the reinforcement learning agent is 220*2 / 3V.

[0180] 2) The state space variables of reinforcement learning should be values ​​that can be collected or calculated by the controller. d , q-axis current i q Already managed by the decoupling current controller, i d Close to 0.

[0181] 3) In order to optimize the training algorithm better, the input value is dimensioned so that the input value is maintained at 10 in various working states. 0 ~10 2 At the same time, for sudden position instructions (such as steps), the derivative value will become a very large number, resulting in pathological experience samples, so the value of the derivative needs to be limited to a certain range.

[0182] Due to the high nonlinearity of neural networks, at the beginning of training, the output of the network often takes the boundary value and oscillates back and forth between the upper and lower boundaries. In the motor model, the oscillation of the speed command will cause the motor feedback value to oscillate, which in turn causes the reinforcement learning controller to oscillate, and finally put the motor in a highly unstable state, as shown in Figure 10. Using this pathological data as the experience value of the intelligent agent often does not allow us to learn anything from it. In many cases, even if the intelligent agent has gone through many rounds of training, regardless of whether it can achieve the effect of the controller, the final output still oscillates repeatedly, which is not allowed in actual point motor control and may even damage the motor. In order to alleviate this situation, the two feedback quantities ω and i of the motor can be adjusted. q Add a Kalman filter. After such an operation, even if the system oscillates at the beginning of training, the presence of the Kalman filter will filter the input state into a low-frequency signal, and the output will also be a low-frequency signal during the gradual training, which ensures the stability of the training to a certain extent.

[0183] 4) In order to comprehensively improve the performance of the controller, the tasks faced by the reinforcement learning controller should be as rich as possible to include all possible command modes. According to the principle of automatic control, the commonly used performance analysis methods for linear systems are: examining the response ability to step commands in the time domain or examining the open-loop amplitude-frequency characteristics and open-loop logarithmic phase-frequency characteristics in the frequency domain. Combined with the final performance evaluation scheme, the training tasks are divided into the following three types:

[0184] a. Random step command: The zero-position steady state (all derivatives are 0) is used as the initial state, and a new random step target is generated every 2.5 seconds to ensure that the step range d∈[-3.5°,3.5°].

[0185] b. Low-frequency sine command: zero-position steady state is used as the initial state, generating amplitude δ cm ∈[1.5°,4°], low-frequency sinusoidal command δ with circular frequency ω∈[2π×0.05,2π×0.1](rad / s) c =δ cm ·sin(ωt).

[0186] c. High-frequency sine command: zero-position steady state is used as the initial state, generating amplitude δ cm ∈[0.3°,1.2°], high-frequency sinusoidal command δ with circular frequency ω∈[1,20](rad / s) c =δ cm ·sin(ωt).

[0187] During training, each Episode selects a task as an instruction with equal probability and lasts for 10 seconds.

[0188] 5) The setting of the reinforcement learning reward function directly affects the optimization of the algorithm. Considering that the target to be optimized in the present invention is the swing mechanism angle α and the command angle α ref Therefore, the inverse of the square of the difference between the reference value and the feedback value of each link in the three-loop control can be used as the reward value for each time step. Such rewards are not sparse rewards. At the same time, the range of rewards can be stabilized within [-200, 200] by adjusting the coefficients, which may be beneficial to training.

[0189]

[0190] 6) During training, for the input agent state value:

[0191]

[0192] We hope to train an optimal policy function π so that the optimal compensation value can be output for s:

[0193]

[0194] The PPO training algorithm improves the policy gradient method by limiting the amplitude of policy updates, making the training process more stable and efficient. The specific training process is as follows:

[0195] For each step i∈[1,2,3…T / Ts] with duration T and agent time step Ts, the agent samples the state s of Eq. 4.4. i , and use the policy network to generate an action a:

[0196]

[0197] The next state s caused by this action i+1 Together with the state s of this step i 、Action a i And the reward value r calculated by the environment for this step i As training samples [a i ,s i ,r i ,s i+1 ]Experience Replay Cache In. When The number of samples in reaches a mini-batch of size N, and each step starts from A batch of samples is sampled from , and the advantage function is calculated by calculating the time series difference error or the generalized advantage estimate.

[0198] Next, PPO optimizes the objective function. First, PPO defines the probability ratio of the old strategy to the new strategy. And define the clipping objective function:

[0199]

[0200] Here, s is a hyperparameter for the clipping range.

[0201] Then, PPO uses the stochastic gradient ascent method (Adam optimizer) to update the policy network parameters to maximize the clipping objective function L CLIP (θ).

[0202] At the same time, PPO uses the stochastic gradient descent method (Adam optimizer) to update the value network parameters to minimize the loss function of the value network:

[0203]

[0204] Finally, update the parameters of the target network with a smoothing ratio τ, and repeat the above steps until the termination condition of the training is reached:

[0205] θ'←τθ+(1-τ)θ'

[0206]

[0207] 7) Considering the deployment requirements on embedded chips, the actor network should not be set too large. The present invention sets it as an MLP with 2 hidden layers; and the critic network is set as an MLP with 4 hidden layers. At the same time, considering that the sinusoidal command frequency input by the controller can reach 20rad / s and the training speed should not be too slow, it is more appropriate to set the agent sampling time to 0.01s. Other training parameters are set as follows:

[0208]

[0209] In order to comprehensively evaluate the performance improvement of the position loop tuning scheme based on reinforcement learning compared with the traditional PID method, the following three indicators are used to evaluate the performance.

[0210] 1) Load position characteristics

[0211] Under the maximum load condition, the angle command sequence α ref As the command input, the actual swing angle sequence α of the swing mechanism within the time range t∈(11,31] is collected. fb .in:

[0212]

[0213] α ref As the horizontal axis, α fbThe position loop curve is drawn as the ordinate. The nominal position curve is the midpoint line of the position loop curve on the horizontal axis, and the nominal position baseline is the first-order linear fit of the nominal position curve. The tracking accuracy in the time domain of the algorithm is analyzed by loop width and zero deviation. The maximum swing angle in the positive and negative directions represents the tracking capability of the approaching command limit position; Δδ max The maximum loop width characterizes the maximum value of the tracking error; the δ0 zero deviation measures the symmetry of the control algorithm in the positive and negative directions when facing a symmetrical signal such as a sinusoidal low frequency.

[0214] Combining Figure 11(a) and the table below, the DRL method is superior to the PID and ADRC methods in terms of the maximum swing angle, maximum loop width, and zero-position deviation during the control process. This proves that the DRL control method has better tracking accuracy through data-driven experience learning.

[0215]

[0216] 2) Speed ​​characteristics experiment

[0217] Under the maximum load condition, a step command with an amplitude of 3° is used as the system input α ref , collect the actual load position sequence in the time period when the swing angle is in the range of (0.5°, 1.5°), and use the average swing angular velocity of the swing angle The response speed of the algorithm in the time domain is analyzed, and the experimental results are shown in the table below. Combining Figure 11(b) and the table below, it can be seen that the PID control method has the fastest rise speed, but its overshoot is also the highest, proving that although the PID control method has a fast response, it is also the most unstable; the ADRC control method is the opposite, although it has the slowest response, it is the most stable; the DRL method is between the two, with a faster response speed and a more stable performance.

[0218]

[0219] 3) Frequency characteristics experiment

[0220] Under the maximum load condition, the sinusoidal command α ref =δ·sin(ωt) as α ref , where δ=0.5°, 0.8°, 1.1°, ω=2, 4, 8, 16 (rad / s). Six instruction cycles are simulated at each frequency and amplitude. The phase attenuation of the swing angle output compared to the input instruction is measured. and amplitude attenuation L, analyze the response characteristics of the algorithm in the frequency domain. ref and α fb In with α ref Perform orthogonal decomposition on the sine and cosine bases of the same frequency to obtain αref and α fb The sine and cosine components a ref ,b ref and a fb , b fb Calculate the relative standard excitation amplitude and phase angle

[0221]

[0222] The gain L (dB) and phase lag of the actual angle signal with respect to the command signal are calculated by the above formula: The results are recorded below.

[0223] Combining Figure 11(c), Figure 11(d) and the table below, it can be seen that the controller under the DRL method is generally better than the PID method in terms of phase attenuation index.

[0224]

[0225] The above three test methods are used to quantitatively measure the control performance of PID method, ADRC method and DRL method in various indicators. After comparison, it can be seen that the DRL method has better tracking accuracy and response speed than the PID method and ADRC method in most indicators.

Claims

1. An intelligent control method for a permanent magnet synchronous motor servo system in a semi-closed loop scenario, characterized in that: The method includes constructing a permanent magnet synchronous motor operation model based on a FOC control framework and a mathematical model of a load and its transmission mechanism, and then based on a three-loop structure, using a reinforcement learning agent to determine a state space and action settings, using an anti-disturbance algorithm in a position loop to improve the anti-disturbance capability of the control system, using PI control in a speed loop, and using a reinforcement learning algorithm in a current loop to train a strategy network, and by observing the feedback position input by the position loop and the reference speed output, the feedback speed input by the speed loop and the reference current and feedback current output, the decision output voltage applied to the permanent magnet synchronous motor is made, and a PPO reinforcement learning algorithm with continuous state and action spaces is used for agent optimization, and an ADRC controller is used to receive an external input reference value of a controlled object, and a reference value and a rate of change of the position control are output through a differential controller; the value observed by the agent is determined as the differential controller output θ of the ADRC v1 (t),θ v2 (t), the output of the reference signal generator e1(t), e2(t), the command deflection angle α * (t) The converted screw precession length With feedback The error e θ (t), reference speed ω * (t) and reference current And the corresponding feedback speed ω(t) and feedback current i q (t); In order to reduce the swing mechanism angle α and the command deflection angle α * The error of (t) is obtained by taking the square of the difference between the reference value and the feedback value of each of the three rings and taking the inverse of the square as the reward value for each time step, and multiplying it by a fixed coefficient to balance the weights among the three rings: Status Using the optimization strategy function Enables the agent to output tuning values 2. The intelligent control method of a permanent magnet synchronous motor servo system in a semi-closed loop scenario according to claim 1 is characterized in that: The implementation steps of this method include: S1. Construct a permanent magnet synchronous motor operation model based on the FOC control framework. The FOC method of the permanent magnet synchronous motor decomposes the phase variable of the motor into a magnetic field component and a torque component, and controls them independently to achieve precise control of the motor magnetic field and torque; S2. Construct a mathematical model of the load and its transmission mechanism. The motor load model takes into account the elastic deformation of the transmission mechanism, the dynamic equation of the swing mechanism, and also considers nonlinear factors such as Coulomb friction and torque transmission in the triangular linkage mechanism; S3. Considering the permanent magnet synchronous motor operation model and high-order nonlinear load model in the above modeling, a deep reinforcement learning-driven edge intelligent control method is constructed on the FOC control framework; In the control method, the ADRC controller receives the external input reference value of the controlled object, and outputs the reference value of the position control and its rate of change through the differential controller; at the same time, the ADRC controller receives the feedback of the measurement element of the controlled object. Since there is a certain amount of noise and oscillation in the time domain during the measurement of the feedback, these will affect the correct output of the subsequent deep reinforcement learning agent. Therefore, the ADRC controller first inputs the feedback signal into the state expansion observer to predict the noise contained therein, and uses the corresponding feedforward mechanism to remove it; finally, the ADRC controller will make a difference between the outputs of the differential controller and the state expansion observer to output the reference value of the next loop; The PI controller makes a difference between the reference value of the ADRC controller and the feedback value of the measuring element, and linearly superimposes the difference and the integral value over time as the output control quantity to the next actuator; S4, is the feedback quantity ω and i q A Kalman filter is added and a low-pass filter is added to the agent output to smooth the control voltage of the permanent magnet synchronous motor.

3. The intelligent control method of a permanent magnet synchronous motor servo system in a semi-closed loop scenario according to claim 2 is characterized in that: The method comprises the following optimization process: The spatial state of the agent is dimensionalized and limited, and the two feedback quantities ω and i q Add a Kalman filter and add a low-pass filter to the output of the agent; The controller command modes include random step commands, low-frequency sine commands, and high-frequency sine commands. Two Q networks are established to evaluate the value of the output of the policy network, and the evaluation values ​​are used to perform gradient optimization on the actor network.

4. The intelligent control method of a permanent magnet synchronous motor servo system in a semi-closed loop scenario according to claim 2 is characterized in that: Step S2 specifically includes: For the high-order load mechanism with nonlinear factors, the motor shaft torque is transmitted to the screw by gears, and the triangular connecting rod mechanism is pushed and pulled by the screw stroke to form a force arm, which pushes the swing mechanism to deflect the angle. In the process of modeling, in order to fit the mechanical properties of the actual load, the elastic motion of the screw, the motion equation of the swing mechanism and the nonlinear Coulomb friction during the motion are considered; The dynamic equation of this load can be described as: Among them, θ m is the rotor mechanical angle, g r is the gear reduction coefficient, n r is the screw reduction coefficient, L is the screw retraction / extension, K s is the combined rotation stiffness, F is the force exerted on the screw by the torque; M e is the mass of the screw, B e is elastic damping, ΔL is the screw compression stroke; T L is the motor shaft load torque, effi is the transmission efficiency; M is the swing torque, K p is the combined precession stiffness, r is the length of the lever arm; α is the swing mechanism angle, J b is the swing inertia, B b is the swing damping, K delta is the position resistance torque, M f is the friction torque, which is modeled as Coulomb friction and is expressed as follows: The above load model takes into account the elastic deformation of the screw. If we only consider the geometric relationship of the triangular linkage mechanism, the equivalent triangular structure formed between the screw side, the swing mechanism and the fixed fulcrum; without considering the elastic deformation of the screw, the relationship between the retraction / extension amount L of the screw and the angle α of the swing mechanism can be approximately expressed as: Where a and b are the lengths of the two adjacent sides OA and OB of the swing angle in the triangular linkage mechanism; α0 and L0 are the swing center angle ∠AOB and the length of the side AB when the load deflection angle is 0; and, since the servo system and the linkage mechanism are located on the same straight line, the feedback θ of the servo system loc Equal to the angle α with ∠AOB.

5. The intelligent control method of a permanent magnet synchronous motor servo system in a semi-closed loop scenario according to claim 1 is characterized in that: The specific steps to alleviate the problem of insufficient control accuracy caused by position feedback error include: In order to improve the overall performance of the controller, random step instructions, low-frequency sine instructions, and high-frequency sine instructions are mixed for training. Each Episode selects a task as an instruction with equal probability and lasts for 10 seconds. Considering the deployment requirements on embedded chips, the actor is set to an MLP with 2 hidden layers. The critic network is set to an MLP with 4 hidden layers. During training, the agent sampling time is set to 0.01s.

Images