Servo motor control method based on reinforcement learning

Abstract

The application is suitable for the technical field of motor control, and provides a servo motor control method based on reinforcement learning, which comprises the following steps: obtaining three-phase stator currents during the operation of a target motor, and converting the three-phase stator currents to obtain a first current component and a second current component, wherein the first current component is used to represent a control reference of the rotor flux of the target motor, and the second current component is used to represent the torque output capability of the target motor; comparing the first current component and the second current component with preset reference values respectively to obtain error data; obtaining compensation data for the target motor, and inputting the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage instruction and a second voltage instruction; determining a three-phase voltage waveform according to the first voltage instruction and the second voltage instruction through a space vector pulse width modulation algorithm, and then controlling the target motor according to the three-phase voltage waveform.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of motor control, and particularly relates to a servo motor control method based on reinforcement learning. BACKGROUND

[0002] As a core component of modern mechatronic systems, the servo motor controller plays a key role in accurately regulating the speed, position and torque of the motor. Its performance is directly related to the accuracy, response speed and stability of the entire system. Permanent magnet synchronous motor servo systems have been widely used in modern industrial systems. At the same time, the permanent magnet synchronous motor drive system has the characteristics of high nonlinearity, strong coupling and multi-variable, and the motor will appear external load disturbance in the running process, which makes the traditional PI control have obvious shortcomings in ensuring the dynamic response and anti-interference ability of the drive system. SUMMARY

[0003] The embodiment of the application provides a servo motor control method based on reinforcement learning, which can solve the problems of insufficient progress tracking degree and poor anti-interference ability in the prior art.

[0004] In a first aspect, the embodiment of the application provides a servo motor control method based on reinforcement learning, comprising:

[0005] Obtaining three-phase stator currents when a target motor is running, and converting the three-phase stator currents to obtain a first current component and a second current component, the first current component being used to represent a control reference of a rotor flux of the target motor, and the second current component being used to represent a torque output capability of the target motor;

[0006] Comparing the first current component and the second current component with preset reference values respectively to obtain error data;

[0007] Obtaining compensation data for the target motor, and inputting the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage instruction and a second voltage instruction;

[0008] Determining a three-phase voltage waveform according to the first voltage instruction and the second voltage instruction through a space vector pulse width modulation algorithm, and then controlling the target motor according to the three-phase voltage waveform.

[0009] Optionally, the step of obtaining the three-phase stator currents when the target motor is running comprises:

[0010] Establishing an initial PMSM model of the target motor, and the initial PMSM model is represented as:

[0011]

[0012] According to the mathematical operation rule, the initial PMSM model is converted to obtain a converted initial PMSM model:

[0013]

[0014] The converted initial PMSM model is discretely processed by using a first-order Euler formula to obtain a target PMSM model:

[0015]

[0016] The load data with the friction term is input into the target PMSM model to obtain three-phase stator currents of the target motor in operation.

[0017] Optionally, before the step of inputting the load data with the friction term into the target PMSM model, the method further comprises:

[0018] A friction force model for calculating the friction term is established, and the friction force model is represented as:

[0019]

[0020] wherein, is a Coulomb friction torque, is a maximum static friction torque; is a rotational angular velocity, is a Stribeck velocity, is a bristle stiffness coefficient, is a bristle damping coefficient, is a viscous friction coefficient;

[0021] The speed feedback data of the target motor in operation is fed into the friction force model to obtain the friction term.

[0022] Optionally, the step of converting the three-phase stator currents to obtain the first current component and the second current component comprises:

[0023] The three-phase stator currents are converted to a two-phase orthogonal coordinate system by using a Clark transformation technology to obtain a first current signal and a second current signal;

[0024] The first current signal and the second current signal are mapped to a rotating coordinate system by using a Park transformation to obtain the first current component and the second current component.

[0025] Optionally, before the step of comparing the first current component and the second current component with preset reference values respectively to obtain error data, the method further comprises:

[0026] acquiring an angular velocity of the target motor through a position sensor, and calculating an initial rotating speed of the target motor based on the angular velocity;

[0027] filtering the initial rotating speed to obtain a target rotating speed of the target motor;

[0028] inputting the target rotating speed and a reference rotating speed into a preset PI controller to obtain a second threshold corresponding to the second current component in the preset reference value.

[0029] Optionally, the step of acquiring compensation data of the target motor comprises:

[0030] inputting the target rotating speed and the second current component into a preset disturbance observation model to obtain the compensation data of the target motor, wherein the preset disturbance observation model comprises a low-pass filter.

[0031] Optionally, the step of comparing the first current component and the second current component with a preset reference value respectively to obtain error data comprises:

[0032] comparing the first current component with a first threshold in the preset reference value to obtain first error data in the error data;

[0033] comparing the second current component with a second threshold in the preset reference value to obtain second error data in the error data.

[0034] Optionally, the step of inputting the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage instruction and a second voltage instruction comprises:

[0035] correcting the second error data in the error data based on the compensation data, inputting the corrected second error data and the first error data into the preset reinforcement learning network to obtain a first voltage signal and a second voltage signal, wherein the preset reinforcement learning network comprises an Actor sub-neural network and a Critic sub-neural network, the Actor sub-neural network is used for updating an action policy; the Critic sub-neural network is used for evaluating an action and a state and outputting evaluation data; the Actor sub-neural network updates the action policy according to the evaluation data output by the Critic sub-neural network; the preset reinforcement learning network further comprises a reward function, and the reward function is expressed as:

[0036]

[0037] is a reward gain, is a past control gain, is a penalty term, gk For the target function, the target function is expressed as:

[0038] ;

[0039] The first voltage signal and the second voltage signal are inversely Park transformed to obtain a first voltage command and a second voltage command.

[0040] Optionally, the step of determining a three-phase voltage waveform according to the first voltage command and the second voltage command by the space vector pulse width modulation algorithm comprises:

[0041] The first voltage command and the second voltage command are input into the space vector pulse width modulation algorithm to determine a sector to which a target voltage vector belongs, the target voltage vector being a voltage vector synthesized from the first voltage command and the second voltage command;

[0042] The on-time of each bridge arm switch tube of a three-phase inverter for supplying power to the target motor is calculated based on the sector to which the target voltage vector belongs;

[0043] The three-phase voltage waveform is determined in an optimized manner according to the on-time.

[0044] Optionally, the process of inputting the error data and the compensation data into a preset reinforcement learning network is a Markov decision process defined by a tuple , wherein X is a state space, A is an action space, a transition function, a reward function, a discount factor, the preset reinforcement learning network being used to determine a target action, the target action being defined by a policy to maximize the total expected discounted reward of the agent , expressed as:

[0045] .

[0046] Compared with the prior art, the embodiments of the present application have the beneficial effects that:

[0047] The reinforcement learning algorithm is ingeniously introduced into the permanent magnet synchronous motor control, which not only gives the motor controller the ability of self-learning and optimization, but also enables it to accurately learn the optimal control strategy through continuous interaction with the environment, realizes more efficient and accurate motor control, effectively filters out noise to obtain a clearer and smoother disturbance torque signal, improves the overall performance of the system, and ensures its stable operation. The combination of reinforcement learning and friction compensation technology not only solves the problem of torque ripple generated by the permanent magnet synchronous motor when outputting torque, but also optimizes the control strategy through intelligent learning, improving the overall performance of the control system. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0049] Figure 1 is a flowchart of a servo motor control method based on reinforcement learning provided by an embodiment of the present application;

[0050] Figure 2 is a structural schematic diagram of a servo motor control system based on reinforcement learning provided by another embodiment of the present application;

[0051] Figure 3 is a structural schematic diagram of a disturbance observer according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] In the following description, specific details such as specific system structures, techniques, etc. are presented in order to thoroughly understand the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits and methods are omitted to avoid unnecessary details that hinder the description of the present application.

[0053] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or sets thereof.

[0054] It should also be understood that the term "and / or" used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0055] As used in the description of the application and the appended claims, the term "if' can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]," depending on the context.

[0056] In addition, the description in the specification of the application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0057] In the description of the application, the reference "one embodiment" or "some embodiments" and the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in other some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0058] Figure 1 The schematic flowchart of the servo motor control method based on reinforcement learning provided by the application is shown, and the embodiment of the application provides a servo motor control method based on reinforcement learning, comprising:

[0059] S101, acquiring three-phase stator currents of a target motor running, and converting the three-phase stator currents to obtain a first current component and a second current component, the first current component being used to represent a control reference of a rotor flux of the target motor, and the second current component being used to represent a torque output capability of the target motor;

[0060] S102, comparing the first current component and the second current component with preset reference values respectively to obtain error data;

[0061] S103, acquiring compensation data for the target motor, and inputting the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage instruction and a second voltage instruction;

[0062] S104, determining a three-phase voltage waveform according to the first voltage instruction and the second voltage instruction through a space vector pulse width modulation algorithm, and then controlling the target motor according to the three-phase voltage waveform.

[0063] The reinforcement learning algorithm is ingeniously introduced into the permanent magnet synchronous motor control, which not only gives the motor controller the ability of self-learning and optimization, but also enables it to accurately learn the optimal control strategy through continuous interaction with the environment, realizes more efficient and accurate motor control, effectively filters out noise to obtain a clearer and smoother disturbance torque signal, improves the overall performance of the system, and ensures its stable operation. The combination of reinforcement learning and friction compensation technology not only solves the problem of torque ripple generated by the permanent magnet synchronous motor when outputting torque, but also optimizes the control strategy through intelligent learning, improving the overall performance of the control system.

[0064] In a possible implementation, the step of obtaining the three-phase stator current of the target motor during operation includes:

[0065] An initial PMSM model of the target motor is established, and the initial PMSM model is represented as:

[0066]

[0067] According to the mathematical operation rule, the initial PMSM model is converted to obtain a converted initial PMSM model:

[0068]

[0069] The converted initial PMSM model is discretely processed by using a first-order Euler formula to obtain a target PMSM model:

[0070]

[0071] The load data with the friction term is input into the target PMSM model to obtain the three-phase stator current of the target motor during operation.

[0072] For example, when constructing a PMSM position controller based on reinforcement learning, the state is first defined as the current position error and the current speed error These state information is obtained by measuring the actual position and speed of the motor. Then, the action is defined as the current of the motor, which is output to the motor by the controller to adjust its operating state. Through such design, the current input can be dynamically adjusted according to the current state of the motor (i.e. the error of position and speed), so as to realize accurate control of the motor position.

[0073] In a possible implementation, before the step of inputting the load data with the friction term into the target PMSM model, the method further comprises:

[0074] establishing a friction force model for calculating the friction term, the friction force model being expressed as:

[0075]

[0076] wherein, is a Coulomb friction torque, is a maximum static friction torque; is a rotational angular velocity, is a Stribeck velocity, is a bristle stiffness coefficient, is a bristle damping coefficient, is a viscous friction coefficient;

[0077] feeding the speed feedback data of the target motor in operation into the friction force model to obtain the friction term.

[0078] Exemplarily, in the PMSM control model, a friction force compensation term (friction term) is introduced to reduce the influence of the friction force on the position control accuracy.

[0079] In a possible implementation, the step of converting the three-phase stator current to obtain the first current component and the second current component comprises:

[0080] the three-phase stator current is converted to a two-phase orthogonal coordinate system by using a Clark transformation technique to obtain a first current signal and a second current signal;

[0081] the first current signal and the second current signal are mapped to a rotating coordinate system by using a Park transformation to obtain the first current component and the second current component.

[0082] Exemplarily, when the Park transformation is performed, the angular velocity of the PMSM model is obtained by using a position sensor, and the first current signal and the second current signal are mapped to the rotating coordinate system according to the angular velocity.

[0083] In a possible implementation, before the step of comparing the first current component and the second current component with preset reference values respectively to obtain error data, the method further comprises:

[0084] the angular velocity of the target motor is obtained by using a position sensor, and the initial rotational speed of the target motor is calculated based on the angular velocity;

[0085] the initial rotational speed is filtered to obtain a target rotational speed of the target motor;

[0086] inputting the target rotating speed and the reference rotating speed into a preset PI controller to obtain a second threshold corresponding to the second current component in the preset reference value.

[0087] In a possible implementation, the step of obtaining compensation data for the target motor comprises:

[0088] inputting the target rotating speed and the second current component into a preset disturbance observation model to obtain the compensation data of the target motor, wherein the preset disturbance observation model comprises a low-pass filter.

[0089] In a possible implementation, the step of comparing the first current component and the second current component with a preset reference value respectively to obtain error data comprises:

[0090] comparing the first current component with a first threshold in the preset reference value to obtain first error data in the error data;

[0091] comparing the second current component with a second threshold in the preset reference value to obtain second error data in the error data.

[0092] In a possible implementation, the step of inputting the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage instruction and a second voltage instruction comprises:

[0093] correcting the second error data in the error data based on the compensation data, inputting the corrected second error data and the first error data into the preset reinforcement learning network to obtain a first voltage signal and a second voltage signal, wherein the preset reinforcement learning network comprises an Actor sub-neural network and a Critic sub-neural network, the Actor sub-neural network is used to update an action policy; the Critic sub-neural network is used to evaluate an action and a state and output evaluation data; the Actor sub-neural network updates the action policy according to the evaluation data output by the Critic sub-neural network; the preset reinforcement learning network further comprises a reward function, and the reward function is expressed as:

[0094]

[0095] is a reward gain, is a past control gain, is a penalty term, g k is an objective function, and the objective function is expressed as:

[0096] ;

[0097] performing inverse Park transformation on the first voltage signal and the second voltage signal to obtain a first voltage command and a second voltage command.

[0098] In a possible implementation, the step of determining a three-phase voltage waveform according to the first voltage command and the second voltage command by using a space vector pulse width modulation algorithm includes:

[0099] inputting the first voltage command and the second voltage command into the space vector pulse width modulation algorithm to determine a sector to which a target voltage vector belongs, the target voltage vector being a voltage vector synthesized from the first voltage command and the second voltage command;

[0100] calculating, based on the sector to which the target voltage vector belongs, on-time of a switch tube of each bridge arm of a three-phase inverter for supplying power to the target motor;

[0101] determining the three-phase voltage waveform in an optimized manner according to the on-time.

[0102] In a possible implementation, the process of inputting the error data and the compensation data into a preset reinforcement learning network is a Markov decision process, and the Markov decision process is defined by a tuple , where X is a state space, A is an action space, a transition function, a reward function, a discount factor, the preset reinforcement learning network is used to determine a target action, and the target action is defined by a policy to maximize the total expected discounted reward of the agent , which is expressed as:

[0103] .

[0104] In a possible implementation, a servo motor control method based on reinforcement learning and friction compensation includes: building a PMSM model, which is expressed by a formula as follows:

[0105] a dq-axis voltage equation

[0106] (1)

[0107] In another expression:

[0108] (2)

[0109] The formula (2) can be discretized by using a first-order Euler formula as follows:

[0110] (3)

[0111] uabc and udq are calculated by the state of the switch tube

[0112] (4)

[0113] (5)

[0114] Designing a reinforcement learning adaptive controller (preset reinforcement learning network). First, initialize the Simulink model, open MATLAB / Simulink, and create a new Simulink model. Add the necessary components, such as signal sources, PID controllers, controlled objects, and measurement elements (position sensors). The RL agent (adaptive controller, preset reinforcement learning network) interacts with the environment (error data), representing a Markov Decision Process (MDP), represented by a tuple , where X is the state space, 𝐴 is the action space, Transition function, Reward function, Discount factor. The goal of the RL agent is to find the optimal behavior, which is defined by the policy , which maps actions to states to maximize the agent's total expected discounted reward , as follows:

[0115] (6)

[0116] The reward function is one of the key elements of the PMSM position controller based on reinforcement learning. The reward function needs to consider the weight factors of position error and speed error, as well as the stability of the controller, etc. In the current control, the goal is to minimize the current error, and the four-element objective function can be written as:

[0117] (7)

[0118] The objective function, constraint violation penalty, and past actions are used as rewards for RL, as follows:

[0119] (8)

[0120] where, is the reward gain, is the past control gain. is the penalty term, which is used to ensure safe operation and prevent the occurrence of overcurrent regions during training.

[0121] Add a PID controller module (preset PI controller) in the Simulink model. In the Simulink model, one or more custom S-Functions or MATLAB Function Blocks need to be added to implement the reinforcement learning algorithm. The reinforcement learning algorithm receives feedback (i.e. error signal) from the environment and updates the parameters of the PID controller based on this feedback.

[0122] In the Simulink model settings, configure the simulation parameters such as simulation time, step size, etc.

[0123] Analyze the friction force received by the PMSM during motion and establish the corresponding mathematical model. In the PMSM control model, introduce a friction force compensation term to reduce the impact of friction on position control accuracy. The model of the friction force compensation term is:

[0124] (9)

[0125] where, , are the Coulomb friction torque and the maximum static friction torque, respectively; , are the angular velocity and the Stribeck velocity, respectively. , , are the bristle stiffness coefficient, the bristle damping coefficient, and the viscous friction coefficient, respectively.

[0126] Design a disturbance observer (preset disturbance observation model), as shown in Figure 3 , which includes a low-pass filter Qs to eliminate high-frequency noise in the torque and speed signals. The disturbance torque is usually processed through the low-pass filter Qs. This step can effectively filter out noise and obtain a clearer and smoother disturbance torque signal for subsequent torque compensation.

[0127] According to the constructed PMSM model, PID controller, and adaptive controller, the reinforcement learning-based servo motor control system is obtained as shown in Figure 2 , and simulation is performed.

[0128] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0129] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / network device / terminal device and method can be implemented in other manners. For example, the embodiments of the apparatus / network device / terminal device described above are merely schematic; the division of the modules or units is merely logical function division; there can be another division manner in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0130] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0131] The above-described embodiments are merely used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalent replacements; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for controlling a servo motor based on reinforcement learning, characterized by, The method comprises the following steps: obtaining the three-phase stator current of the target motor in operation, and converting the three-phase stator current to obtain a first current component and a second current component, wherein the first current component is used to represent the control reference of the rotor flux of the target motor, and the second current component is used to represent the torque output capability of the target motor; before the step of comparing the first current component and the second current component with the preset reference value respectively to obtain error data, the method further comprises the following steps: obtaining the angular velocity of the target motor through a position sensor, and calculating the initial rotating speed of the target motor based on the angular velocity; filtering the initial rotating speed to obtain the target rotating speed of the target motor; inputting the target rotating speed and the reference rotating speed into a preset PI controller to obtain a second threshold value corresponding to the second current component in the preset reference value; comparing the first current component and the second current component with the preset reference value respectively to obtain error data; the step of comparing the first current component and the second current component with the preset reference value respectively to obtain error data comprises the following steps: comparing the first current component with a first threshold value in the preset reference value to obtain first error data in the error data; comparing the second current component with a second threshold value in the preset reference value to obtain second error data in the error data; obtaining compensation data for the target motor, and inputting the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage instruction and a second voltage instruction; the step of obtaining compensation data for the target motor comprises the following steps: inputting the target rotating speed and the second current component into a preset disturbance observation model to obtain the compensation data of the target motor, wherein the preset disturbance observation model comprises a low-pass filter; the step of inputting the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage instruction and a second voltage instruction comprises the following steps: based on the compensation data, correcting the second error data in the error data, inputting the corrected second error data and the first error data into the preset reinforcement learning network to obtain a first voltage signal and a second voltage signal, wherein the preset reinforcement learning network comprises an Actor sub-neural network and a Critic sub-neural network, the Actor sub-neural network is used to update the action policy; the Critic sub-neural network is used to evaluate the action and the state, and output evaluation data; the Actor sub-neural network updates the action policy according to the evaluation data output by the Critic sub-neural network; the preset reinforcement learning network further comprises a reward function, and the reward function is represented as: is a reward gain, is a past control gain, is a penalty term, g k is an objective function, expressed as: ； The process of inputting the error data and the compensation data into a preset reinforcement learning network is a Markov decision process, and the Markov decision process is represented by a tuple Definition, wherein X is a state space, A is an action space, Transition function, Reward function, Discount factor, the preset reinforcement learning network is used to determine the target behavior, and the target behavior is represented by a policy Definition, to maximize the total expected discounted reward of the agent , represented as: performing inverse Park transformation on the first voltage signal and the second voltage signal to obtain a first voltage instruction and a second voltage instruction; determining a three-phase voltage waveform according to the first voltage instruction and the second voltage instruction through a space vector pulse width modulation algorithm, and then controlling the target motor according to the three-phase voltage waveform. 2.The reinforcement learning-based servo motor control method of claim 1, wherein, The step of obtaining the three-phase stator current of the target motor in operation includes: An initial PMSM model of the target motor is established, and the initial PMSM model is represented as: The initial PMSM model is converted according to a mathematical operation rule to obtain a converted initial PMSM model: The converted initial PMSM model is discretely processed by using a first-order Euler formula to obtain a target PMSM model: The three-phase stator current of the target motor in operation is obtained by inputting load data with a friction term into the target PMSM model.

3. The reinforcement learning-based servo motor control method of claim 2, wherein, Before the step of inputting the load data with the friction term into the target PMSM model, the method further includes: A friction force model for calculating the friction term is established, and the friction force model is represented as: wherein is the Coulomb friction torque, is the maximum static friction torque; is the angular velocity of rotation, is the Stribeck velocity, is the bristle stiffness coefficient, is the bristle damping coefficient, is the viscous friction coefficient; The speed feedback data of the target motor in operation is fed into the friction force model to obtain the friction term. 4.The reinforcement learning based servo motor control method of claim 1, wherein, The step of converting the three-phase stator current to obtain the first current component and the second current component includes: The three-phase stator current is converted to a two-phase orthogonal coordinate system by using a Clark conversion technology to obtain a first current signal and a second current signal; The first current signal and the second current signal are mapped to a rotating coordinate system by using a Park conversion to obtain the first current component and the second current component. 5.The reinforcement learning based servo motor control method of claim 1, wherein, The step of determining the three-phase voltage waveform according to the first voltage instruction and the second voltage instruction by using the space vector pulse width modulation algorithm includes: The first voltage instruction and the second voltage instruction are input into the space vector pulse width modulation algorithm to determine a sector to which a target voltage vector belongs, and the target voltage vector is a voltage vector synthesized by the first voltage instruction and the second voltage instruction; The conduction time of a switch tube of each bridge arm of a three-phase inverter for supplying power to the target motor is calculated based on the sector to which the target voltage vector belongs; The three-phase voltage waveform is determined in an optimal manner according to the conduction time.

Images