Servo motor control method based on reinforcement learning

Abstract

The invention is suitable for the technical field of motor control, and provides a servo motor control method based on reinforcement learning, and the method comprises the steps: obtaining a three-phase stator current during the operation of a target motor, converting the three-phase stator current, and obtaining a first current component and a second current component, the first current component is used for representing the control reference of the magnetic flux of a target motor rotor, and the second current component is used for representing the torque output capability of the target motor; comparing the first current component and the second current component with a preset reference value to obtain error data; acquiring compensation data for the target motor, and inputting the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage instruction and a second voltage instruction; and determining a three-phase voltage waveform according to the first voltage instruction and the second voltage instruction through a space vector pulse width modulation algorithm, and further controlling the target motor according to the three-phase voltage waveform.

Description

Technical Field

[0001] The present application belongs to the field of motor control technology, and in particular to a servo motor control method based on reinforcement learning. Background Art

[0002] As the core component of modern mechatronics systems, servo motor controllers play a key role in accurately adjusting the speed, position and torque of motors. Its performance is directly related to the accuracy, response speed and stability of the entire system. Permanent magnet synchronous motor servo systems have been widely used in modern industrial systems. At the same time, permanent magnet synchronous motor drive systems have the characteristics of high nonlinearity, strong coupling and multivariables, and external load disturbances will occur during the operation of the motor, which makes traditional PI control have obvious deficiencies in ensuring the dynamic response and anti-interference ability of the drive system. Summary of the invention

[0003] The embodiment of the present application provides a servo motor control method based on reinforcement learning, which can solve the problems of insufficient progress tracking and poor anti-interference ability in the prior art.

[0004] In a first aspect, an embodiment of the present application provides a servo motor control method based on reinforcement learning, comprising:

[0005] Acquire a three-phase stator current of a target motor when it is running, and convert the three-phase stator current to obtain a first current component and a second current component, wherein the first current component is used to characterize a control reference of a rotor magnetic flux of the target motor, and the second current component is used to characterize a torque output capacity of the target motor;

[0006] Compare the first current component and the second current component with preset reference values ​​respectively to obtain error data;

[0007] Acquire compensation data for the target motor, and input the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage command and a second voltage command;

[0008] A three-phase voltage waveform is determined according to the first voltage instruction and the second voltage instruction through a space vector pulse width modulation algorithm, and then the target motor is controlled according to the three-phase voltage waveform.

[0009] Optionally, the step of obtaining the three-phase stator current of the target motor when it is running includes:

[0010] An initial PMSM model of the target motor is established, and the initial PMSM model is expressed as:

[0011]

[0012] The initial PMSM model is converted according to mathematical operation rules to obtain a converted initial PMSM model:

[0013]

[0014] The converted initial PMSM model is discretized using the first-order Euler formula to obtain the target PMSM model:

[0015]

[0016] The load data with the friction term is input into the target PMSM model to obtain the three-phase stator current when the target motor is running.

[0017] Optionally, before the step of inputting the load data with friction items into the target PMSM model, the method further includes:

[0018] A friction model for calculating the friction term is established, and the friction model is expressed as:

[0019]

[0020] Among them, T c is the Coulomb friction torque, T s is the maximum static friction torque; ω is the angular velocity of rotation, ω s is the Stribeck velocity, σ 0 is the bristle stiffness coefficient, σ 1 is the bristle damping coefficient, σ 2 is the viscous friction coefficient;

[0021] The speed feedback data of the target motor during operation is brought into the friction force model to obtain the friction term.

[0022] Optionally, the step of converting the three-phase stator current to obtain a first current component and a second current component includes:

[0023] Using Clark transformation technology, the three-phase stator current is converted into a two-phase orthogonal coordinate system to obtain a first current signal and a second current signal;

[0024] The first current signal and the second current signal are mapped to a rotating coordinate system through Park transformation to obtain a first current component and a second current component.

[0025] Optionally, before the step of comparing the first current component and the second current component with preset reference values ​​respectively to obtain error data, the method further includes:

[0026] Acquiring the angular velocity of the target motor through a position sensor, and calculating the initial rotation speed of the target motor based on the angular velocity;

[0027] Filtering the initial speed to obtain a target speed of the target motor;

[0028] The target speed and the reference speed are input into a preset PI controller to obtain a second threshold corresponding to the second current component in the preset reference value.

[0029] Optionally, the step of acquiring compensation data for the target motor includes:

[0030] The target rotation speed and the second current component are input into a preset disturbance observation model to obtain compensation data of the target motor, wherein the preset disturbance observation model includes a low-pass filter.

[0031] Optionally, the step of comparing the first current component and the second current component with preset reference values ​​respectively to obtain error data includes:

[0032] Compare the first current component with a first threshold in the preset reference value to obtain first error data in the error data;

[0033] The second current component is compared with a second threshold in the preset reference value to obtain second error data in the error data.

[0034] Optionally, the step of inputting the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage instruction and a second voltage instruction includes:

[0035] Based on the compensation data, the second error data in the error data is corrected, and the corrected second error data and the first error data are input into the preset reinforcement learning network to obtain a first voltage signal and a second voltage signal, wherein the preset reinforcement learning network includes an Actor sub-neural network and a Critic sub-neural network, the Actor sub-neural network is used to update the action strategy; the Critic sub-neural network is used to evaluate the action and state and output the evaluation data; the Actor sub-neural network updates the action strategy according to the evaluation data output by the Critic sub-neural network; the preset reinforcement learning network also includes a reward function, and the reward function is expressed as:

[0036]

[0037] ω 1 ,ω 2 ,ω 3 is the reward gain, is the past control gain, P k is the penalty term, g k is the objective function, and the objective function is expressed as:

[0038]

[0039] Perform inverse Park transformation on the first voltage signal and the second voltage signal to obtain a first voltage instruction and a second voltage instruction.

[0040] Optionally, the step of determining the three-phase voltage waveform according to the first voltage instruction and the second voltage instruction by using a space vector pulse width modulation algorithm includes:

[0041] Inputting the first voltage command and the second voltage command into the space vector pulse width modulation algorithm to determine the sector to which the target voltage vector belongs, wherein the target voltage vector is a voltage vector synthesized by the first voltage command and the second voltage command;

[0042] Calculating the conduction time of each bridge arm switch tube of the three-phase inverter that supplies power to the target motor based on the sector to which the target voltage vector belongs;

[0043] The three-phase voltage waveform is determined in an optimized manner according to the conduction time.

[0044] Optionally, the process of inputting the error data and the compensation data into the preset reinforcement learning network is a Markov decision process, and the Markov decision process is defined by a tuple (X, A, P, r, γ), where X is the state space, A is the action space, and P(x k+1 |x k ,a k ) transition function, r(x k ,a k ) reward function, γ∈[0,1] discount factor, the preset reinforcement learning network is used to determine the target behavior, which is defined by the policy π to maximize the agent's total expected discounted return J(π), expressed as:

[0045]

[0046] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0047] The ingenious introduction of reinforcement learning algorithm into permanent magnet synchronous motor control not only gives the motor controller the ability to self-learn and optimize, but also enables it to accurately learn the best control strategy through continuous interaction with the environment, achieving more efficient and precise motor control, while effectively filtering out noise, thereby obtaining a clearer and smoother disturbance torque signal, improving the overall performance of the system, and ensuring its stable operation. Combining reinforcement learning with friction compensation technology not only solves the torque pulsation problem generated by permanent magnet synchronous motors when outputting torque, but also optimizes the control strategy through intelligent learning, improving the overall performance of the control system. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 It is a flowchart of a servo motor control method based on reinforcement learning provided in one embodiment of the present application;

[0050] Figure 2 is a structural schematic diagram of a servo motor control system based on reinforcement learning provided by another embodiment of the present application;

[0051] Figure 3 It is a schematic diagram of the structure of a disturbance observer according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0053] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0054] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0055] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0056] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0057] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0058] Figure 1 A schematic flow chart of a servo motor control method based on reinforcement learning provided by the present application is shown. The embodiment of the present application provides a servo motor control method based on reinforcement learning, including:

[0059] S101, obtaining a three-phase stator current of a target motor when it is running, and converting the three-phase stator current to obtain a first current component and a second current component, wherein the first current component is used to characterize a control reference of a rotor magnetic flux of the target motor, and the second current component is used to characterize a torque output capacity of the target motor;

[0060] S102, comparing the first current component and the second current component with preset reference values ​​respectively to obtain error data;

[0061] S103, obtaining compensation data for the target motor, and inputting the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage command and a second voltage command;

[0062] S104. Determine a three-phase voltage waveform according to the first voltage instruction and the second voltage instruction through a space vector pulse width modulation algorithm, and then control the target motor according to the three-phase voltage waveform.

[0063] The ingenious introduction of reinforcement learning algorithm into permanent magnet synchronous motor control not only gives the motor controller the ability to self-learn and optimize, but also enables it to accurately learn the best control strategy through continuous interaction with the environment, achieving more efficient and precise motor control, while effectively filtering out noise, thereby obtaining a clearer and smoother disturbance torque signal, improving the overall performance of the system, and ensuring its stable operation. Combining reinforcement learning with friction compensation technology not only solves the torque pulsation problem generated by permanent magnet synchronous motors when outputting torque, but also optimizes the control strategy through intelligent learning, improving the overall performance of the control system.

[0064] In a possible implementation manner, the step of obtaining the three-phase stator current of the target motor when it is running includes:

[0065] An initial PMSM model of the target motor is established, and the initial PMSM model is expressed as:

[0066]

[0067] The initial PMSM model is converted according to mathematical operation rules to obtain a converted initial PMSM model:

[0068]

[0069] The converted initial PMSM model is discretized using the first-order Euler formula to obtain the target PMSM model:

[0070]

[0071] The load data with the friction term is input into the target PMSM model to obtain the three-phase stator current when the target motor is running.

[0072] For example, when constructing a PMSM position controller based on reinforcement learning, the state is first defined as the current position error e k and the current speed error v k This state information is obtained by measuring the actual position and speed of the motor. Then, the action is defined as the current of the motor, which is output to the motor through the controller to adjust its operating state. With this design, the current input can be dynamically adjusted according to the current state of the motor (i.e. the error in position and speed), thereby achieving precise control of the motor position.

[0073] In a possible implementation manner, before the step of inputting the load data with friction items into the target PMSM model, the step further includes:

[0074] A friction model for calculating the friction term is established, and the friction model is expressed as:

[0075]

[0076] Among them, T c is the Coulomb friction torque, T s is the maximum static friction torque; ω is the angular velocity of rotation, ω s is the Stribeck velocity, σ 0 is the bristle stiffness coefficient, σ 1 is the bristle damping coefficient, σ 2 is the viscous friction coefficient;

[0077] The speed feedback data of the target motor during operation is brought into the friction force model to obtain the friction term.

[0078] Exemplarily, in the PMSM control model, a friction compensation term (friction term) is introduced to reduce the influence of friction on position control accuracy.

[0079] In a possible implementation manner, the step of converting the three-phase stator current to obtain the first current component and the second current component includes:

[0080] Using Clark transformation technology, the three-phase stator current is converted into a two-phase orthogonal coordinate system to obtain a first current signal and a second current signal;

[0081] The first current signal and the second current signal are mapped to a rotating coordinate system through Park transformation to obtain a first current component and a second current component.

[0082] Exemplarily, when performing Park transformation, the angular velocity of the PMSM model is acquired through a position sensor, and the first current signal and the second current signal are mapped to a rotating coordinate system according to the angular velocity.

[0083] In a possible implementation manner, before the step of comparing the first current component and the second current component with preset reference values ​​respectively to obtain error data, the step further includes:

[0084] Acquiring the angular velocity of the target motor through a position sensor, and calculating the initial rotation speed of the target motor based on the angular velocity;

[0085] Filtering the initial speed to obtain a target speed of the target motor;

[0086] The target speed and the reference speed are input into a preset PI controller to obtain a second threshold corresponding to the second current component in the preset reference value.

[0087] In a possible implementation manner, the step of acquiring compensation data for the target motor includes:

[0088] The target rotation speed and the second current component are input into a preset disturbance observation model to obtain compensation data of the target motor, wherein the preset disturbance observation model includes a low-pass filter.

[0089] In a possible implementation manner, the step of comparing the first current component and the second current component with preset reference values ​​respectively to obtain error data includes:

[0090] Compare the first current component with a first threshold in the preset reference value to obtain first error data in the error data;

[0091] The second current component is compared with a second threshold in the preset reference value to obtain second error data in the error data.

[0092] In a possible implementation manner, the step of inputting the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage instruction and a second voltage instruction includes:

[0093] Based on the compensation data, the second error data in the error data is corrected, and the corrected second error data and the first error data are input into the preset reinforcement learning network to obtain a first voltage signal and a second voltage signal, wherein the preset reinforcement learning network includes an Actor sub-neural network and a Critic sub-neural network, the Actor sub-neural network is used to update the action strategy; the Critic sub-neural network is used to evaluate the action and state and output the evaluation data; the Actor sub-neural network updates the action strategy according to the evaluation data output by the Critic sub-neural network; the preset reinforcement learning network also includes a reward function, and the reward function is expressed as:

[0094]

[0095] ω 1 ,ω 2 ,ω 3 is the reward gain, is the past control gain, P k is the penalty term, g k is the objective function, and the objective function is expressed as:

[0096]

[0097] Perform inverse Park transformation on the first voltage signal and the second voltage signal to obtain a first voltage instruction and a second voltage instruction.

[0098] In a possible implementation manner, the step of determining the three-phase voltage waveform according to the first voltage instruction and the second voltage instruction by using a space vector pulse width modulation algorithm includes:

[0099] Inputting the first voltage command and the second voltage command into the space vector pulse width modulation algorithm to determine the sector to which the target voltage vector belongs, wherein the target voltage vector is a voltage vector synthesized by the first voltage command and the second voltage command;

[0100] Calculating the conduction time of each bridge arm switch tube of the three-phase inverter that supplies power to the target motor based on the sector to which the target voltage vector belongs;

[0101] The three-phase voltage waveform is determined in an optimized manner according to the conduction time.

[0102] In a possible implementation, the process of inputting the error data and the compensation data into the preset reinforcement learning network is a Markov decision process, and the Markov decision process is defined by a tuple (X, A, P, r, γ), where X is the state space, A is the action space, and P(x k+1 |x k ,a k ) transition function, r(x k ,a k ) reward function, γ∈[0,1] discount factor, the preset reinforcement learning network is used to determine the target behavior, which is defined by the policy π to maximize the agent's total expected discounted return J(π), expressed as:

[0103]

[0104] In a possible implementation, a servo motor control method based on reinforcement learning and friction compensation includes: building a PMSM model, and the formula is expressed as:

[0105] dq axis voltage equation

[0106]

[0107] Another way to express it:

[0108]

[0109] Using the first-order Euler formula to discretize formula (2), we can get:

[0110]

[0111] Calculate uabc and udq by the state of the switch tube

[0112]

[0113]

[0114] Design a reinforcement learning adaptive controller (preset reinforcement learning network). First initialize the Simulink model, open MATLAB / Simulink, and create a new Simulink model. Add necessary components such as signal source, PID controller, controlled object, and measurement element (position sensor). The RL agent (adaptive controller, preset reinforcement learning network) interacts with the environment (error data) and is represented as a Markov decision process (MDP) defined by a tuple (X, A, P, r, γ), where X is the state space, A is the action space, and P (x k+1 |x k ,a k ) transition function, r(x k ,a k ) reward function, γ∈[0,1] discount factor. The goal of the RL agent is to find the optimal behavior, which is defined by a policy π that maps actions to states to maximize the agent’s total expected discounted return J(π), as follows:

[0115]

[0116] The reward function is one of the keys to the PMSM position controller based on reinforcement learning. The reward function needs to consider the weight factors of the position error and the speed error, as well as the stability of the controller. In the current control, the goal is to minimize the current error. The quaternary objective function can be written as:

[0117]

[0118] The objective function, constraint violation penalty, and past actions are used as rewards for RL as follows:

[0119]

[0120] Among them, ω 1 ,ω 2 ,ω 3 is the reward gain, is the past control gain. k is a penalty term used to ensure safe operation and prevent the occurrence of overcurrent regions during training.

[0121] Add a PID controller block (preset PI controller) to the Simulink model. In the Simulink model, you need to add one or more custom S-Functions or MATLAB Function Blocks to implement the reinforcement learning algorithm. The reinforcement learning algorithm receives feedback (i.e., error signal) from the environment and updates the parameters of the PID controller based on this feedback.

[0122] In the Simulink model settings, configure simulation parameters such as simulation time, step size, etc.

[0123] The friction force that PMSM is subjected to during movement is analyzed and the corresponding mathematical model is established. In the PMSM control model, a friction compensation term is introduced to reduce the influence of friction on position control accuracy. The model of the friction compensation term is:

[0124]

[0125] Among them, T c 、T s are the Coulomb friction torque and the maximum static friction torque respectively; ω, ω s are the angular velocity and the Stribeck velocity respectively. 0 , σ 1 , σ 2 They are the bristle stiffness coefficient, bristle damping coefficient, and viscous friction coefficient respectively.

[0126] Design a disturbance observer (preset disturbance observation model), such as Figure 3 As shown, a low-pass filter Qs is included to eliminate high-frequency noise in the torque and speed signals. The disturbance torque is usually processed through a low-pass filter Qs. This step can effectively filter out the noise, thereby obtaining a clearer and smoother disturbance torque signal for subsequent torque compensation.

[0127] According to the constructed PMSM model, PID controller and adaptive controller, the following is obtained: Figure 2 The reinforcement learning-based servo motor control system shown in the figure is simulated and run.

[0128] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0129] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment / terminal equipment and methods can be implemented in other ways. For example, the device / network equipment / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0130] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0131] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A servo motor control method based on reinforcement learning, It is characterized in that include: Acquire a three-phase stator current of a target motor when it is running, and convert the three-phase stator current to obtain a first current component and a second current component, wherein the first current component is used to characterize a control reference of a rotor magnetic flux of the target motor, and the second current component is used to characterize a torque output capacity of the target motor; Compare the first current component and the second current component with preset reference values ​​respectively to obtain error data; Acquire compensation data for the target motor, and input the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage command and a second voltage command; A three-phase voltage waveform is determined according to the first voltage instruction and the second voltage instruction through a space vector pulse width modulation algorithm, and then the target motor is controlled according to the three-phase voltage waveform.

2. The servo motor control method based on reinforcement learning as claimed in claim 1, It is characterized in that The step of obtaining the three-phase stator current of the target motor when it is running includes: An initial PMSM model of the target motor is established, and the initial PMSM model is expressed as: The initial PMSM model is converted according to mathematical operation rules to obtain a converted initial PMSM model: The converted initial PMSM model is discretized using the first-order Euler formula to obtain the target PMSM model: The load data with the friction term is input into the target PMSM model to obtain the three-phase stator current when the target motor is running.

3. The servo motor control method based on reinforcement learning as claimed in claim 2, It is characterized in that Before the step of inputting the load data with friction items into the target PMSM model, the method further includes: A friction model for calculating the friction term is established, and the friction model is expressed as: Among them, T c is the Coulomb friction torque, T s is the maximum static friction torque; ω is the angular velocity of rotation, ω s is the Stribeck velocity, σ 0 is the bristle stiffness coefficient, σ 1 is the bristle damping coefficient, σ 2 is the viscous friction coefficient; The speed feedback data of the target motor during operation is brought into the friction force model to obtain the friction term.

4. The servo motor control method based on reinforcement learning as claimed in claim 1, It is characterized in that The step of converting the three-phase stator current to obtain a first current component and a second current component comprises: Using Clark transformation technology, the three-phase stator current is converted into a two-phase orthogonal coordinate system to obtain a first current signal and a second current signal; The first current signal and the second current signal are mapped to a rotating coordinate system through Park transformation to obtain a first current component and a second current component.

5. The servo motor control method based on reinforcement learning as claimed in claim 1, It is characterized in that Before the step of comparing the first current component and the second current component with preset reference values ​​respectively to obtain error data, the method further includes: Acquiring the angular velocity of the target motor through a position sensor, and calculating the initial rotation speed of the target motor based on the angular velocity; Filtering the initial speed to obtain a target speed of the target motor; The target speed and the reference speed are input into a preset PI controller to obtain a second threshold corresponding to the second current component in the preset reference value.

6. The servo motor control method based on reinforcement learning as claimed in claim 5, It is characterized in that The step of obtaining compensation data for the target motor includes: The target rotation speed and the second current component are input into a preset disturbance observation model to obtain compensation data of the target motor, wherein the preset disturbance observation model includes a low-pass filter.

7. The servo motor control method based on reinforcement learning as claimed in claim 5, It is characterized in that The step of comparing the first current component and the second current component with preset reference values ​​respectively to obtain error data comprises: Compare the first current component with a first threshold in the preset reference value to obtain first error data in the error data; The second current component is compared with a second threshold in the preset reference value to obtain second error data in the error data.

8. The servo motor control method based on reinforcement learning as claimed in claim 7, It is characterized in that The step of inputting the error data and the compensation data into a preset reinforcement learning network to obtain a first voltage instruction and a second voltage instruction includes: Based on the compensation data, the second error data in the error data is corrected, and the corrected second error data and the first error data are input into the preset reinforcement learning network to obtain a first voltage signal and a second voltage signal, wherein the preset reinforcement learning network includes an Actor sub-neural network and a Critic sub-neural network, the Actor sub-neural network is used to update the action strategy; the Critic sub-neural network is used to evaluate the action and state and output the evaluation data; the Actor sub-neural network updates the action strategy according to the evaluation data output by the Critic sub-neural network; the preset reinforcement learning network also includes a reward function, and the reward function is expressed as: ω 1 ,ω 2 ,ω 3 is the reward gain, is the past control gain, P k is the penalty term, g k is the objective function, and the objective function is expressed as: Perform inverse Park transformation on the first voltage signal and the second voltage signal to obtain a first voltage instruction and a second voltage instruction.

9. The servo motor control method based on reinforcement learning as claimed in claim 1, It is characterized in that The step of determining the three-phase voltage waveform according to the first voltage instruction and the second voltage instruction by using a space vector pulse width modulation algorithm comprises: Inputting the first voltage command and the second voltage command into the space vector pulse width modulation algorithm to determine the sector to which the target voltage vector belongs, wherein the target voltage vector is a voltage vector synthesized by the first voltage command and the second voltage command; Calculating the conduction time of each bridge arm switch tube of the three-phase inverter that supplies power to the target motor based on the sector to which the target voltage vector belongs; The three-phase voltage waveform is determined in an optimized manner according to the conduction time.

10. The servo motor control method based on reinforcement learning according to claim 1, It is characterized in that The process of inputting the error data and the compensation data into the preset reinforcement learning network is a Markov decision process, and the Markov decision process is defined by a tuple (X, A, P, r, γ), where X is the state space, A is the action space, and P(x k+1 |x k ,a k ) transition function, r(x k ,a k ) reward function, γ∈[0,1] discount factor, the preset reinforcement learning network is used to determine the target behavior, which is defined by the policy π to maximize the agent's total expected discounted return J(π), expressed as:

Images