Training method, control method and system of permanent magnet synchronous motor control model

Abstract

The application discloses a permanent magnet synchronous motor control method, a model training method and a system. By continuously learning and updating the parameters of the main strategy learning network, an optimized nonlinear control strategy is obtained. The trained main strategy learning network replaces the existing flux linkage PI regulator and torque PI regulator, avoids the problem of low control accuracy caused by the design limitation of the PI regulator parameters, improves the control accuracy of the permanent magnet synchronous motor, reduces torque ripple, avoids bearing wear caused by motor vibration, and improves the service life of the bearing. In addition, the application also takes the speed variable (i.e. the speed measurement value) of the outer ring as the observation value of reinforcement learning, realizes the deep cooperation of the outer ring (speed control ring) and the inner ring (torque and flux linkage control ring), integrates the two-stage independent control into an overall optimized intelligent control, brings more superior system-level dynamic performance, and further improves the control accuracy of the permanent magnet synchronous motor.

Description

Technical Field

[0001] This invention relates to motor control technology, and more particularly to a training method, control method, and system for a permanent magnet synchronous motor control model. Background Technology

[0002] A permanent magnet synchronous motor is a synchronous motor that uses permanent magnets to generate a magnetic field. The rotor speed is synchronized with the current frequency of the stator windings. Permanent magnet synchronous motors have wide applications in industrial production and daily life.

[0003] In space vector modulation direct torque control of a permanent magnet synchronous motor, the outer loop employs a speed PI controller, whose output is the torque reference value; the inner loop employs a flux linkage PI controller and a torque PI controller, whose outputs are the reference values ​​for the d-axis and q-axis components of the stator voltage, respectively. The design of the flux linkage PI controller and the torque PI controller directly affects the system's dynamic response, steady-state accuracy, and robustness.

[0004] However, the design of PI controller parameters is usually limited by: actual permanent magnet synchronous motor parameters, such as stator resistance and inductance; the accuracy of the mathematical model of the permanent magnet synchronous motor; the operating point of the permanent magnet synchronous motor; the experience of the designers, etc., which leads to low control accuracy, large torque pulsation, large motor vibration, causing bearing wear and reducing bearing life. Summary of the Invention

[0005] This invention provides a training method, control method, and system for a permanent magnet synchronous motor control model, in order to improve the control accuracy of the permanent magnet synchronous motor, reduce torque pulsation, avoid bearing wear caused by motor vibration, and improve the service life of the bearing.

[0006] In a first aspect, the present invention provides a training method for a control model of a permanent magnet synchronous motor, comprising: The training requires experience samples from the experience pool. The experience samples include a first state vector, an action vector inferred by the main policy learning network of the TD3 algorithm model based on the first state vector, a second state vector at the next time step, and a reward for taking the action vector under the first state vector. The state vector includes flux linkage measurement value, torque measurement value, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement value, and speed reference value. The action vector includes reference values ​​for the d-axis component and the q-axis component of the stator voltage. The TD3 algorithm model is trained using the empirical samples, and the parameters of the TD3 algorithm model are updated until the reward of the TD3 algorithm model is greater than the reward threshold or the number of training rounds reaches the maximum number of training rounds. The master policy learning network after training is used to predict the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component during the operation of the permanent magnet synchronous motor.

[0007] Optionally, the TD3 algorithm model includes a main policy learning network, a first value evaluation network, a second value evaluation network, a target policy learning network, a first target value evaluation network, and a second target value evaluation network. The TD3 algorithm model is trained using the empirical samples, and its parameters are updated until the reward of the TD3 algorithm model exceeds a reward threshold or the number of training epochs reaches the maximum number of training epochs. This includes: Samples are taken from the experience pool to obtain batch sample data including multiple experience samples; The batch sample data is input into the TD3 algorithm model for training, and the parameters of the first value evaluation network and the second value evaluation network are updated. The parameters of the main policy learning network are updated every preset number of times the parameters of the first value evaluation network and the second value evaluation network are updated. The parameters of the target policy learning network, the first target value evaluation network, and the second target value evaluation network are updated using a soft update method. Repeat the above steps until the reward of the TD3 algorithm model is greater than the reward threshold or the number of training rounds reaches the maximum number of training rounds. Each round includes multiple time steps, and each time step includes the entire process of the TD3 algorithm model from receiving experience samples, inferring action vectors, returning rewards, and the state vector of the next time step.

[0008] Optionally, the batch sample data is input into the TD3 algorithm model for training, updating the parameters of the first value evaluation network and the second value evaluation network, including: The second state vector is input into the target policy learning network for inference to obtain the first action vector; Add truncated noise to the first action vector to obtain the target action vector; The target action vector is input into the first target value evaluation network to calculate the first action value of the first action vector; The target action vector is input into the second target value evaluation network to calculate the second action value of the first action vector; The smaller of the first action value and the second action value is taken as the target action value; Calculate the sum of the discounts between the reward and the value of the target action to obtain the target evaluation value; Input the action vector corresponding to the first state vector into the first value evaluation network to calculate the third action value of the action vector corresponding to the first state vector. Input the action vector corresponding to the first state vector into the second value evaluation network to calculate the fourth action value of the action vector corresponding to the first state vector. The parameters of the first value evaluation network are calculated using the gradient descent algorithm to minimize the error between the target evaluation value and the value of the third action, and the parameters of the first value evaluation network are updated. The parameters of the second value evaluation network are calculated using the gradient descent algorithm to minimize the error between the target evaluation value and the value of the fourth action, and the parameters of the second value evaluation network are updated.

[0009] Optionally, the parameters of the main policy learning network are updated every preset number of updates to the parameters of the first and second value evaluation networks, including: When the parameters of the first value evaluation network and the second value evaluation network are updated a preset number of times, the second state vector is input into the main policy learning network for inference to obtain the second action vector. The second action vector is input into the first value evaluation network to calculate the fifth action value of the second action vector; The parameters of the master policy learning network that maximize the value of the fifth action are calculated using the gradient ascent algorithm, and the parameters of the master policy learning network are updated.

[0010] Optionally, the above steps are repeated until the reward of the TD3 algorithm model is greater than the reward threshold or the number of rounds in this round has reached the maximum number of training rounds, including: After updating the parameters of the target policy learning network, the first target value evaluation network, and the second target value evaluation network using a soft update method, it is determined whether the time step of this round has reached the maximum time step of a single round. If the time step of this round reaches the maximum time step of a single round, then end the training for this round and determine whether the number of training rounds has reached the evaluation interval. If the time step of this round does not reach the maximum time step of a single round, then return to the step of sampling from the experience pool to obtain batch sample data including multiple experience samples; If the number of training rounds has not reached the evaluation interval, then proceed to determine whether the number of rounds in this round is greater than the maximum number of training rounds. If the number of training rounds reaches the evaluation interval, then a test sample is taken from the experience pool, and the TD3 algorithm model is run for a preset number of rounds. Calculate the cumulative discount reward generated at all time steps in each round within the preset round as the return; Calculate the average of all rewards for the preset round; Determine if the average of all returns is greater than the return threshold; If so, then end the training process; If not, then determine whether the number of rounds in this round is greater than the maximum number of training rounds; If so, then end the training process; If not, return to the step of sampling from the experience pool to obtain batch sample data including multiple experience samples.

[0011] Optionally, sampling is performed from the experience pool to obtain batch sample data including multiple experience samples, including: Initialize the three-phase current, rotor speed measurement value, and rotor position measurement value of the permanent magnet synchronous motor; Calculate the stator flux linkage and torque measurements based on the three-phase current and rotor position measurements; Input the measured speed value and the speed reference value into the speed PI controller to calculate the flux linkage reference value and torque reference value; The first state vector, composed of the flux linkage measurement value, torque measurement value, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement value, and speed reference value, is input into the main policy learning network for inference to obtain the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component. Among them, the torque deviation is the difference between the torque reference value and the torque measurement value, and the flux linkage deviation is the difference between the flux linkage reference value and the flux linkage measurement value. By performing the inverse Parker transform on the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component, the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component are obtained. Space vector pulse width modulation is performed based on the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component to obtain a pulse width modulation signal, which is used to control the operation of the inverter of the permanent magnet synchronous motor. The reward for taking the action vector under the first state vector in this time step is calculated based on the reference values ​​of torque deviation, flux deviation, stator voltage d-axis component and stator voltage q-axis component. The three-phase current, rotor speed measurement value, and rotor position measurement value of the permanent magnet synchronous motor are collected, and the process returns to the step of calculating the stator flux linkage measurement value and torque measurement value based on the three-phase current and rotor position measurement value to obtain the second state vector of the next time step; The action vector composed of the state vector, the reference value of the stator voltage d-axis component, and the reference value of the stator voltage q-axis component, the reward, and the second state vector of the next time step are combined into an experience sample and stored in the experience pool. Determine whether the number of experience samples in the experience pool has reached the preset number required for training. If so, sample from the experience pool to obtain batch sample data including multiple experience samples; If not, return to the steps of initializing the three-phase current, rotor speed measurement, and rotor position measurement of the permanent magnet synchronous motor until the number of experience samples in the experience pool reaches the preset number required for training.

[0012] Optionally, the reward for taking the action vector under the first state vector in this time step is calculated based on the reference values ​​of torque deviation, flux deviation, stator voltage d-axis component, and stator voltage q-axis component. The calculation formula is as follows: in, The reward for taking the action vector under the first state vector at the k-th time step, where Q1, Q2, and R are hyperparameters. For torque deviation, For magnetic flux deviation, This is the reference value for the d-axis component of the stator voltage. This is the reference value for the q-axis component of the stator voltage.

[0013] Secondly, the present invention also provides a permanent magnet synchronous motor control method, which includes a master policy learning network trained based on the training method described in the first aspect of the present invention, comprising: During the operation of the permanent magnet synchronous motor, the three-phase current, rotor speed measurement value, and rotor position measurement value of the permanent magnet synchronous motor are determined; Calculate the stator flux linkage and torque measurements based on the three-phase current and rotor position measurements; Input the measured speed value and the speed reference value into the speed PI controller to calculate the flux linkage reference value and torque reference value; The state vector composed of flux linkage measurement, torque measurement, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement, and speed reference value is input into the main policy learning network of the TD3 algorithm model for inference, so as to obtain the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component. Among them, the torque deviation is the difference between the torque reference value and the torque measurement value, and the flux linkage deviation is the difference between the flux linkage reference value and the flux linkage measurement value. By performing the inverse Parker transform on the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component, the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component are obtained. Space vector pulse width modulation is performed based on the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component to obtain a pulse width modulation signal, which is used to control the operation of the inverter of the permanent magnet synchronous motor. Repeat the above steps until a stop signal from the permanent magnet synchronous motor is received.

[0014] Thirdly, the present invention also provides a training system for a permanent magnet synchronous motor control model, comprising: A heterogeneous system-on-a-chip, the heterogeneous system-on-a-chip including a processor unit and a programmable logic unit, the processor unit including a playback buffer, the playback buffer being used to store an experience pool; A cloud server is configured to execute the training method as described in the first aspect of the present invention and deploy the trained master policy learning network to the programmable logic unit.

[0015] Optionally, the processor unit further includes a speed PI regulator and a reward calculation module, and the programmable logic unit further includes a Parker inverse transform module, a space vector pulse width modulation module, an analog-to-digital conversion module, a Clarke transform module, a torque flux linkage calculation module, and a speed and position calculation module. The speed and position calculation module is used to calculate the rotor's position measurement value and rotor speed measurement value based on the sensor's acquired signals; The analog-to-digital converter module is used to acquire the three-phase current of the permanent magnet synchronous motor and convert the three-phase current into digital signals; The Clarke transform module is used to perform Clarke transform on the digital signal of the three-phase current to obtain the measured values ​​of the α-axis component and the β-axis component of the stator current. The torque flux linkage calculation module is used to calculate the stator flux linkage measurement value and torque measurement value based on the measured values ​​of the stator current α-axis component and the stator current β-axis component. The speed PI regulator is used to calculate the flux linkage reference value and torque reference value based on the measured speed value and the speed reference value; The reward calculation module is used to calculate the reward for taking the action vector under the first state vector in the current time step based on the reference values ​​of torque deviation, flux deviation, stator voltage d-axis component and stator voltage q-axis component, and to calculate the cumulative discount reward generated by all time steps in each round within the preset round as a reward.

[0016] The training method for the permanent magnet synchronous motor control model provided by this invention obtains the experience samples required for training from an experience pool. The experience samples include a first state vector, an action vector inferred by the main policy learning network of the TD3 algorithm model based on the first state vector, a second state vector for the next time step, and a reward for taking the action vector under the first state vector. The state vector includes flux linkage measurement value, torque measurement value, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement value, and speed reference value. The action vector includes reference values ​​for the d-axis component and the q-axis component of the stator voltage. The TD3 algorithm model is trained using the experience samples, and the parameters of the TD3 algorithm model are updated until the reward of the TD3 algorithm model is greater than the reward threshold or the number of training rounds reaches the maximum number of training rounds. The trained main policy learning network is used to predict the reference values ​​of the d-axis component and the q-axis component of the stator voltage during the operation of the permanent magnet synchronous motor. By continuously learning and updating the parameters of the master policy learning network, an optimized nonlinear control strategy is obtained. After training, the master policy learning network replaces the existing flux linkage PI regulator and torque PI regulator, avoiding the problem of low control accuracy caused by the design limitations of the PI regulator parameters. This improves the control accuracy of the permanent magnet synchronous motor, reduces torque ripple, avoids bearing wear caused by motor vibration, and extends bearing life. Furthermore, this invention also uses the speed variable of the outer loop (i.e., the measured speed value) as an observation value for reinforcement learning, achieving deep collaboration between the outer loop (speed control loop) and the inner loop (torque and flux linkage control loop). This integrates two independent control levels into a holistic optimized intelligent control, resulting in superior system-level dynamic performance and further improving the control accuracy of the permanent magnet synchronous motor.

[0017] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a training method for a permanent magnet synchronous motor control model provided by this invention; Figure 2 A flowchart of another training method for a permanent magnet synchronous motor control model provided by the present invention; Figure 3A flowchart of a permanent magnet synchronous motor control method provided by the present invention; Figure 4 A schematic diagram of the structure of a training system for a permanent magnet synchronous motor control model provided by the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device provided by the present invention.

[0020] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] Figure 1 This is a flowchart illustrating a training method for a permanent magnet synchronous motor (PMSM) control model provided by the present invention. This embodiment is applicable to model training based on the TD3 (Twin Delayed Deep Deterministic Policy Gradient) algorithm. During PMSM operation, the master policy learning network in the trained TD3 algorithm model predicts the reference values ​​of the stator voltage d-axis and q-axis components. This method can be executed by the PMSM control model training device provided by the present invention. This device can be implemented in software and / or hardware, and is typically configured in electronic devices, such as… Figure 1 As shown, the training method for the permanent magnet synchronous motor control model includes the following steps: S101. Obtain the experience samples required for training from the experience pool. The experience samples include the first state vector, the action vector obtained by the main policy learning network of the TD3 algorithm model based on the first state vector, the second state vector of the next time step, and the reward for taking the action vector under the first state vector. The state vector includes the flux linkage measurement value, torque measurement value, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement value, and speed reference value. The action vector includes the reference value of the d-axis component of the stator voltage and the reference value of the q-axis component of the stator voltage.

[0024] In this embodiment of the invention, in the TD3 algorithm, the experience pool (Replay Buffer) is a limited-capacity cache used to store the history of interactions between the agent and the environment (i.e., experience). It is the core carrier of the "experience replay" technique in deep reinforcement learning. The data unit stored in the experience pool is a complete experience, that is, the transformation generated at each time step. In this embodiment of the invention, the experience in the experience pool is: in, Let the state vector be a state vector at a certain time step. The main policy learning network (Actor) for the TD3 algorithm model is based on state vectors. The action vector obtained through reasoning This is the state vector for the next time step. State vector Take action vector The reward. A time step includes the entire process of the TD3 algorithm model from receiving experience samples, inferring action vectors, returning rewards, and the state vector for the next time step.

[0025] The state vector includes flux linkage measurement value, torque measurement value, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement value, and speed reference value. The action vector includes reference values ​​for the d-axis component of the stator voltage and reference values ​​for the q-axis component of the stator voltage.

[0026] In this embodiment of the invention, the experience in the experience pool can be generated in real time during the training process. The agent continuously generates new experiences by interacting with the environment and stores them in the experience pool. Subsequently, it samples from the experience pool to update the network.

[0027] Experience samples are sampled from the experience pool when the amount of experience in the pool meets the requirements for training. Experience samples include the first state vector at a given time step, the action vector inferred by the main policy learning network of the TD3 algorithm model based on the first state vector, the second state vector at the next time step, and the reward for taking the action vector under the first state vector. The state vector includes flux linkage measurement, torque measurement, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement, and speed reference value. The action vector includes reference values ​​for the d-axis component and q-axis component of the stator voltage.

[0028] Here, the reward refers to the feedback signal obtained after the permanent magnet synchronous motor operates according to the reference values ​​of the stator voltage d-axis component and stator voltage q-axis component shown in the action vector under the condition of the state vector. It is used to measure the immediate quality of taking the action vector at a certain state vector. The flux linkage measurement value refers to the amplitude of the flux linkage calculated based on the collected operating data of the permanent magnet synchronous motor (e.g., three-phase current). The torque measurement value refers to the electromagnetic torque calculated based on the collected operating data of the permanent magnet synchronous motor (e.g., three-phase current). The torque deviation refers to the difference between the torque reference value and the torque measurement value. The flux linkage deviation refers to the difference between the flux linkage reference value and the flux linkage measurement value. The torque reference value and the flux linkage reference value are calculated based on the motor speed. The torque deviation integral is the cumulative value of the torque deviation, and the flux linkage deviation integral is the cumulative value of the flux linkage deviation. The rotor position measurement value can be calculated based on the pulse signal output by the sensor (e.g., photoelectric encoder). The speed measurement value is obtained by calculating the change in position per unit time (i.e., the derivative).

[0029] S102. Train the TD3 algorithm model using empirical samples, and update the parameters of the TD3 algorithm model until the reward of the TD3 algorithm model is greater than the reward threshold or the number of training rounds reaches the maximum number of training rounds. The master policy learning network after training is used to predict the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component during the operation of the permanent magnet synchronous motor.

[0030] In this embodiment of the invention, the TD3 algorithm is sampled, and the TD3 algorithm model is trained using empirical samples. The parameters of the TD3 algorithm model are continuously learned and updated until the reward of the TD3 algorithm model exceeds the reward threshold or the number of training rounds reaches the maximum, thereby obtaining an optimized nonlinear control strategy. The trained master policy learning network is used to predict the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component during the operation of the permanent magnet synchronous motor. These reference values ​​are used for the space vector modulation of the permanent magnet synchronous motor. The reward of the TD3 algorithm model refers to the long-term cumulative benefit; in this embodiment, it refers to the cumulative discounted reward generated over multiple time steps. A training round is a sequence of multiple consecutive time steps.

[0031] Furthermore, this invention also uses the speed variable of the outer loop (i.e., the speed measurement value) as the observation value for reinforcement learning, realizing deep collaboration between the outer loop (speed control loop) and the inner loop (torque and flux control loop), integrating the two independent control levels into a whole optimized intelligent control, resulting in superior system-level dynamic performance and further improving the control accuracy of the permanent magnet synchronous motor.

[0032] The training method for the permanent magnet synchronous motor control model provided by this invention obtains the experience samples required for training from an experience pool. The experience samples include a first state vector, an action vector inferred by the main policy learning network of the TD3 algorithm model based on the first state vector, a second state vector for the next time step, and a reward for taking the action vector under the first state vector. The state vector includes flux linkage measurement value, torque measurement value, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement value, and speed reference value. The action vector includes reference values ​​for the d-axis component and the q-axis component of the stator voltage. The TD3 algorithm model is trained using the experience samples, and the parameters of the TD3 algorithm model are updated until the reward of the TD3 algorithm model is greater than the reward threshold or the number of training rounds reaches the maximum number of training rounds. The trained main policy learning network is used to predict the reference values ​​of the d-axis component and the q-axis component of the stator voltage during the operation of the permanent magnet synchronous motor. By continuously learning and updating the parameters of the master policy learning network, an optimized nonlinear control strategy is obtained. After training, the master policy learning network replaces the existing flux linkage PI regulator and torque PI regulator, avoiding the problem of low control accuracy caused by the design limitations of the PI regulator parameters. This improves the control accuracy of the permanent magnet synchronous motor, reduces torque ripple, avoids bearing wear caused by motor vibration, and extends bearing life. Furthermore, this invention also uses the speed variable of the outer loop (i.e., the measured speed value) as an observation value for reinforcement learning, achieving deep collaboration between the outer loop (speed control loop) and the inner loop (torque and flux linkage control loop). This integrates two independent control levels into a holistic optimized intelligent control, resulting in superior system-level dynamic performance and further improving the control accuracy of the permanent magnet synchronous motor.

[0033] Figure 2 This is a flowchart illustrating another training method for a permanent magnet synchronous motor control model provided by the present invention. This training process is offline and performed in a simulation environment. The simulation environment establishes the necessary environment for reinforcement learning, including a permanent magnet synchronous motor simulation model, a three-phase inverter simulation model, an SVPWM (Space Vector Pulse Width Modulation) module, a rotor position calculation module, a speed calculation module, a Clarke transform and per-unitization module, a Park inverse transform module, a torque and flux linkage estimation module, and a speed PI regulator module. The reinforcement learning TD3-Agent interacts with the environment to train a suitable principal policy learning network (Actor). The TD3 algorithm model includes a principal policy learning network (Actor), a first value evaluation network (Critic1), a second value evaluation network (Critic2), a target policy learning network (Target Actor), a first target value evaluation network (Target Critic1), and a second target value evaluation network (Target Critic2). (Reference) Figure 2The training method for the permanent magnet synchronous motor control model includes: 1. Set training parameters.

[0034] The training parameters include: maximum number of training epochs MaxEpisode = 1000; maximum number of iterations per epoch MaxStep = 10000; replay buffer size = 100000; mini-batch size = 512; discount factor γ = 0.9; and learning rate τ = 0.001.

[0035] 2. Initialize the environment and model.

[0036] The initialization environment includes permanent magnet synchronous motor parameters, three-phase inverter parameters, rotor position, speed PI regulator parameters, and speed reference values. The Actor network in TD3-Agent is also initialized. Critic1 network Critic2 network Target Actor Network Target Critic1 network Target Critic2 network Initialize the playback buffer.

[0037] 3. Reset the environment.

[0038] Environment reset ensures the continuity of training. Without a reset, the agent would be stuck in a terminated state after a round, unable to continue exploring. The reset mechanism allows the agent to try repeatedly and continuously collect experience, which is the foundation of round-based task training.

[0039] 4. Generate experience samples and store them in the experience pool.

[0040] (1) Calculate the stator flux linkage and torque measurements based on the three-phase current and rotor position measurements.

[0041] For example, the pulse signal output by the sensor (e.g., photoelectric encoder) is input into the rotor position calculation module to calculate the rotor position measurement value, and the position measurement value is input into the speed calculation module. The speed measurement value is obtained by calculating the change in position per unit time (i.e., the derivative).

[0042] In this embodiment of the invention, the measured speed value and the reference speed value are input into the torque and flux estimation module for calculation to obtain the flux measurement value and the torque measurement value.

[0043] In this embodiment of the invention, the stator flux linkage measurement value and torque measurement value are calculated using the following formula: ; ; ; ; in, , The collected three-phase current of the motor is input to the Clarke transform and per-unit transformation module to perform Clarke transform and per-unit transformation to obtain the α-axis and β-axis components. Indicates the per-unit value. , For the stator flux linkage measurement value , Per-unit components on the axis This is the per-unit value of the stator inductance. This is the per-unit value of the permanent magnet flux linkage, representing the amplitude of the flux linkage generated by the rotor permanent magnet in the stator winding. The position measurement of the rotor (i.e., the rotor) shaft and (angle between axes) The fundamental angular frequency (rated angular frequency) is used for dimensional conversion in per-unit systems. The measured value of the stator flux linkage. This is the torque measurement value.

[0044] (2) Input the measured speed value and the reference speed value into the speed PI controller, and calculate the reference flux value and the reference torque value.

[0045] For example, the measured speed value and the reference speed value are compared, the difference between the reference speed value and the measured speed value is calculated, and this difference is input into the speed PI controller. The speed PI controller calculates the torque reference value through proportional-integral adjustment. For example, the calculation formula is as follows: in, This is a torque reference value. , These are the parameters of the speed PI controller. This is a reference value for rotational speed. This is the measured rotational speed value.

[0046] Magnetic flux reference value The optimal operating range of the motor is crucial, directly impacting its efficiency and control stability. When the motor operates below its rated speed, maintaining a constant air gap flux is desirable to maximize the utilization of the motor's core. Therefore, the flux linkage reference value is typically set to a constant rated value. This value can be easily obtained by looking up a table or directly assigned. When the motor needs to operate above its rated speed, the back electromotive force increases with speed, limiting the maximum voltage that the inverter can provide. In this case, field weakening control is needed to actively reduce the flux linkage reference value to maintain voltage balance, allowing the motor to continue accelerating. The field weakening control algorithm dynamically calculates a suitable flux linkage setpoint based on the current speed and voltage margin.

[0047] (3) Input the first state vector composed of flux linkage measurement value, torque measurement value, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement value and speed reference value into the main strategy learning network for inference to obtain the reference value of the stator voltage d-axis component and the reference value of the stator voltage q-axis component.

[0048] In this embodiment of the invention, the magnetic flux measurement value Torque measurement value Torque deviation Magnetic flux deviation Torque deviation integral Magnetic flux deviation integral Speed ​​measurement value and speed reference value The first state vector s is input into the main policy learning network (Actor) for inference to obtain the action vector. Action vector Reference values ​​including the d-axis component of the stator voltage Reference values ​​for the q-axis component of the stator voltage .Right now .

[0049] (4) Perform the Park inverse transformation on the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component to obtain the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component.

[0050] In this embodiment of the invention, the reference value of the d-axis component of the stator voltage is... Reference values ​​for the q-axis component of the stator voltage Input the Parker inverse transform module to obtain the reference value of the d-axis component of the stator voltage. Reference values ​​for the q-axis component of the stator voltage Perform the inverse Parker transform to obtain the reference value of the α-axis component of the stator voltage. Reference values ​​for the stator voltage β-axis component .

[0051] (5) Based on the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component, space vector pulse width modulation is performed to obtain a pulse width modulation signal, which is used to control the operation of the inverter of the permanent magnet synchronous motor.

[0052] In this embodiment of the invention, the reference value of the α-axis component of the stator voltage is... Reference values ​​for the stator voltage β-axis component The input SVPWM module performs space vector pulse width modulation to obtain a pulse width modulation signal. This pulse width modulation signal is then input to the three-phase inverter simulation model, which inverts the input DC power into AC power to drive the permanent magnet synchronous motor simulation model.

[0053] (6) Calculate the reward for the action vector taken under the first state vector in this time step based on the reference values ​​of torque deviation, flux deviation, stator voltage d-axis component and stator voltage q-axis component.

[0054] In this embodiment of the invention, the reference values ​​of torque deviation, flux linkage deviation, stator voltage d-axis component, and stator voltage q-axis component are input into the reward function to calculate the reward for taking the action vector under the first state vector in the current time step. The reward function is as follows: in, The action vector to be taken under the first state vector s in the k-th time step. The reward, Q1, Q2, and R are hyperparameters. For torque deviation, For magnetic flux deviation, This is the reference value for the d-axis component of the stator voltage. This is the reference value for the q-axis component of the stator voltage.

[0055] (7) Collect the three-phase current, rotor speed measurement value and rotor position measurement value of the permanent magnet synchronous motor, and return to the step of calculating the stator flux measurement value and torque measurement value based on the three-phase current and rotor position measurement value to obtain the second state vector of the next time step.

[0056] In this embodiment of the invention, the three-phase current, rotor speed measurement value and rotor position measurement value of the permanent magnet synchronous motor are collected in the next time step, and the step of calculating the stator flux linkage measurement value and torque measurement value based on the three-phase current and rotor position measurement value is returned to obtain the second state vector s' of the next time step.

[0057] (8) Combine the action vector composed of the state vector, the reference value of the stator voltage d-axis component and the reference value of the stator voltage q-axis component, the reward and the second state vector of the next time step into an experience sample and store it in the experience pool.

[0058] The reference values ​​of the state vector s and the stator voltage d-axis components are used. Reference values ​​for the q-axis component of the stator voltage Composed action vectors The reward and the second state vector s' of the next time step are combined to form an experience sample. Store it in the experience pool.

[0059] 5. Determine whether the number of experience samples in the experience pool meets the preset number required for training.

[0060] In this embodiment of the invention, each time an experience sample is stored, it is determined whether the number of experience samples in the experience pool has reached the preset number required for training. The preset number can be the number of experience samples required for one training round. If yes, step 6 is executed; otherwise, the process returns to the steps of initializing the three-phase current, rotor speed measurement, and rotor position measurement of the permanent magnet synchronous motor, until the number of experience samples in the experience pool has reached the preset number required for training. In a specific embodiment of the invention, refer to... Figure 2 If not, proceed to sub-step (1) of step 10.

[0061] 6. Sample from the experience pool to obtain batch sample data including multiple experience samples.

[0062] If the number of experience samples in the experience pool reaches the preset number required for training, then sample data is obtained from the experience pool, which includes multiple experience samples.

[0063] 7. Input the batch sample data into the TD3 algorithm model for training, and update the parameters of the first value evaluation network and the second value evaluation network.

[0064] In this embodiment of the invention, batch sample data is input into the TD3 algorithm model for training, and the parameters of the first value evaluation network (Critic1) and the second value evaluation network (Critic2) are updated.

[0065] For example, the update process for the first value assessment network (Critic1) and the second value assessment network (Critic2) is as follows: (1) Input the second state vector into the target policy learning network for inference to obtain the first action vector.

[0066] In this embodiment of the invention, the second state vector s' is input into the target policy learning network (target Actor) for inference to obtain the first action vector. , .

[0067] (2) Add truncation noise to the first action vector to obtain the target action vector.

[0068] The target action vector is obtained by adding truncated noise to the first action vector. For example, in the first action vector... Adding normally distributed noise yields the target action vector. , The noise is normally distributed. By adding truncated noise to the first action vector, it is prevented from deviating too much from the original action.

[0069] (3) Input the target action vector into the first target value evaluation network and calculate the first action value of the first action vector.

[0070] target action vector Input the first target value evaluation network (target Critic1) to calculate the first action vector. The value of the first action .

[0071] (4) Input the target action vector into the second target value evaluation network and calculate the second action value of the first action vector.

[0072] target action vector Input the first action vector into the second objective value evaluation network (objective Critic2). The second action value .

[0073] (5) Take the smaller of the first action value and the second action value as the target action value.

[0074] In this embodiment of the invention, the first action value Second action value The smaller of the values ​​is used as the target action value to reduce the risk of overestimation.

[0075] (6) Calculate the discount sum of the reward and the target action value to obtain the target evaluation value.

[0076] Calculate the product of the target action value and the discount factor γ, and sum this product with the current time step as the reward and the discounted sum of the target action value to obtain the target evaluation value. Specifically, the calculation formula is as follows: in, The target evaluation value, The reward for the current time step. As a discount factor, Value of the first action Second action value The smaller of the two, namely the target action value.

[0077] (7) Input the action vector corresponding to the first state vector into the first value evaluation network and calculate the third action value of the action vector corresponding to the first state vector.

[0078] The action vector corresponding to the first state vector s Input into the first value evaluation network (Critic1), calculate the third action value of the action vector corresponding to the first state vector s. .

[0079] (8) Input the action vector corresponding to the first state vector into the second value evaluation network and calculate the fourth action value of the action vector corresponding to the first state vector.

[0080] The action vector corresponding to the first state vector s Input into the second value evaluation network (Critic2) to calculate the fourth action value of the action vector corresponding to the first state vector s. .

[0081] (9) Calculate the parameters of the first value evaluation network that minimizes the error between the target evaluation value and the third action value using the gradient descent algorithm, and update the parameters of the first value evaluation network.

[0082] In this embodiment of the invention, the gradient descent algorithm is used to calculate the target evaluation value. Value of the third action The parameters of the first value evaluation network (Critic1) that minimizes the error. θ 1. And update the parameters of the first value assessment network (Critic1).

[0083] (10) Calculate the parameters of the second value evaluation network that minimizes the error between the target evaluation value and the fourth action value using the gradient descent algorithm, and update the parameters of the second value evaluation network.

[0084] In this embodiment of the invention, the gradient descent algorithm is used to calculate the target evaluation value. Value of the fourth action The parameters of the second value evaluation network (Critic2) that minimizes the error. θ 2. And update the parameters of the second value assessment network (Critic2).

[0085] For example, in an embodiment of the present invention, updates are made by minimizing the mean square error. and ,Right now Where N is the number of empirical samples in the batch sample data.

[0086] 8. Update the parameters of the main policy learning network every preset number of times the parameters of the first value evaluation network and the second value evaluation network are updated.

[0087] In this embodiment of the invention, the parameters of the main policy learning network are updated once every preset number of times (e.g., twice) when the parameters of the first value evaluation network (Critic1) and the second value evaluation network (Critic2) are updated. If the number of updates of the first value evaluation network and the second value evaluation network does not meet the update timing of the main policy learning network, sub-step (1) of step 10 is executed.

[0088] For example, the process of updating the parameters of the master policy learning network is as follows: (1) When the parameters of the first value evaluation network and the second value evaluation network are updated a preset number of times, the second state vector is input into the main policy learning network for inference to obtain the second action vector.

[0089] Every time the parameters of the first value evaluation network (Critic1) and the second value evaluation network (Critic2) are updated a preset number of times (e.g., twice), the second state vector s' is input into the main policy learning network (Actor) for inference to obtain the second action vector. .

[0090] (2) Input the second action vector into the first value evaluation network and calculate the fifth action value of the second action vector.

[0091] The second action vector Input the data into the first value assessment network (Critic1) to calculate the second action vector. The value of the fifth action .

[0092] (3) Use the gradient ascent algorithm to calculate the parameters of the main policy learning network that maximizes the value of the fifth action, and update the parameters of the main policy learning network.

[0093] The gradient ascent algorithm is used to obtain the parameter ϕ that maximizes the value Q of the fifth action, and the main policy learning network (Actor) is then updated. That is: in, The objective function of the main policy learning network (Actor) Regarding the gradient of the network parameter ϕ, This represents the number of empirical samples in the batch sample data. The value Q of the fifth action is relative to the second action vector. gradient, For the second action vector The gradient with respect to the network parameter ϕ.

[0094] 9. Update the parameters of the target policy learning network, the first target value evaluation network, and the second target value evaluation network using a soft update method.

[0095] In this embodiment of the invention, the parameters of the target policy learning network (target Actor), the first target value evaluation network (target Critic1), and the second target value evaluation network (target Critic1) are updated using a soft update method. That is: ; ; in, It is a constant much smaller than 1, for example, 0.005.

[0096] During the training process of this invention, the update frequency of the main policy learning network and the target network (including the target policy learning network, the first target value evaluation network, and the second target value evaluation network) is lower than that of the first value evaluation network and the first value evaluation network. This allows the first value evaluation network and the first value evaluation network to converge faster, provide more accurate action evaluation values, avoid drastic policy oscillations, and improve the stability and final performance of the training.

[0097] 10. Repeat the above steps until the reward of the TD3 algorithm model is greater than the reward threshold or the number of training rounds reaches the maximum number of training rounds. Each round includes multiple time steps, and each time step includes the entire process of the TD3 algorithm model from receiving experience samples, inferring action vectors, returning rewards, and the state vector of the next time step.

[0098] Repeat steps 1-9 above until the reward of the TD3 algorithm model is greater than the reward threshold or the number of training rounds reaches the maximum number of training rounds. Each round includes multiple time steps, and each time step includes the entire process of the TD3 algorithm model from receiving experience samples, inferring action vectors, returning rewards, and the state vector of the next time step.

[0099] Specifically, such as Figure 2 As shown, after updating the parameters of the target policy learning network, the first target value evaluation network, and the second target value evaluation network using a soft update method, the process also includes: (1) Determine whether the time step of this round has reached the maximum time step of a single round.

[0100] After updating the parameters of the target policy learning network, the first target value evaluation network and the second target value evaluation network each time using a soft update method, it is determined whether the time step of this round has reached the maximum time step MaxStep of a single round. If yes, the following step (2) is executed. If no, the step of sampling from the experience pool to obtain batch sample data including multiple experience samples is returned. In a specific embodiment of the present invention, if no, the step of step 4 is returned to generate new experience.

[0101] (2) End the training for this round and determine whether the number of training rounds has reached the evaluation interval.

[0102] If the time step of this round reaches the maximum time step of a single round (MaxStep), it means that the training of this round has been completed. Then, the training of this round ends, and it is determined whether the number of training rounds has reached the evaluation interval. If yes, then the following step (3) is executed; if no, then the step of determining whether the number of rounds of this round is greater than the maximum number of training rounds is executed.

[0103] (3) Conduct model performance evaluation to determine whether the model performance meets the training stopping criteria.

[0104] If the number of training rounds reaches the evaluation interval, model performance is evaluated to determine whether the model's performance meets the training stopping criteria. If yes, the training process ends; otherwise, step (4) is executed. Specifically, the model performance evaluation process is as follows: (3.1) Take test samples from the experience pool and run the TD3 algorithm model for a preset number of rounds.

[0105] In this embodiment of the invention, if the number of training rounds reaches the evaluation interval, test samples are taken from the experience pool and the TD3 algorithm model is run continuously for a preset number of rounds (e.g., 3 rounds).

[0106] (3.2) Calculate the cumulative discount reward generated by all time steps in each round within the preset round as a reward.

[0107] In this embodiment of the invention, the cumulative discount reward generated at all time steps in each round within a preset round is calculated as the return. For example, the cumulative discount reward generated at all time steps in the i-th round is calculated as follows: ; in, The cumulative discount reward generated over all time steps in round i. The total number of time steps in one round. The reward for the k-th time step. This is the discount factor.

[0108] (3.3) Calculate the average of all rewards for the preset round.

[0109] In this embodiment of the invention, the average value of all rewards in a preset round is calculated.

[0110] (3.4) Determine whether the average of all returns is greater than the return threshold.

[0111] In this embodiment of the invention, it is determined whether the average value of all returns is greater than the return threshold. If yes, it means that the model's performance has reached the training stopping criterion, and the training process ends. If no, it means that the model's performance has not yet reached the training stopping criterion, and the following step (4) is executed.

[0112] (4) Determine whether the number of rounds in this round is greater than the maximum number of training rounds.

[0113] Determine if the current training round number is greater than the maximum training round number MaxEpisode. If yes, end the training process; otherwise, return to the step of sampling from the experience pool to obtain batch sample data including multiple experience samples. In a specific embodiment of the invention, if no, return to step 3, environment reset. That is, the environment needs to be reset after each training round.

[0114] This invention also provides a control method for a permanent magnet synchronous motor, based on the master policy learning network trained according to any of the foregoing embodiments of this invention. Figure 3 A flowchart of a permanent magnet synchronous motor control method provided by the present invention is shown below. Figure 3 As shown, the control method for permanent magnet synchronous motors includes: S201. During the operation of the permanent magnet synchronous motor, determine the three-phase current, rotor speed measurement value, and rotor position measurement value of the permanent magnet synchronous motor.

[0115] During the operation of a permanent magnet synchronous motor, the three-phase current, rotor speed measurement, and rotor position measurement are determined in real time. For example, as mentioned above, the three-phase current can be acquired through an analog-to-digital converter module, the rotor position measurement can be calculated based on the pulse signal output by a sensor (e.g., a photoelectric encoder), and the speed measurement is obtained by calculating the change in position per unit time (i.e., the derivative).

[0116] S202. Calculate the stator flux linkage and torque measurements based on the three-phase current and rotor position measurements.

[0117] In this embodiment of the invention, the stator flux linkage measurement value and torque measurement value are calculated using the following formula: ; ; ; ; in, , The collected three-phase motor currents were subjected to Clarke transform and per-unit scaling to obtain the α-axis and β-axis components. Indicates the per-unit value. , For the stator flux linkage measurement value , Per-unit components on the axis This is the per-unit value of the stator inductance. This is the per-unit value of the permanent magnet flux linkage, representing the amplitude of the flux linkage generated by the rotor permanent magnet in the stator winding. The position measurement of the rotor (i.e., the rotor) shaft and (angle between axes) The fundamental angular frequency (rated angular frequency) is used for dimensional conversion in per-unit systems. The measured value of the stator flux linkage. This is the torque measurement value.

[0118] S203. Input the measured speed value and the speed reference value into the speed PI controller to calculate the flux linkage reference value and the torque reference value.

[0119] For example, the measured speed value and the reference speed value are compared, the difference between the reference speed value and the measured speed value is calculated, and this difference is input into the speed PI controller. The speed PI controller calculates the torque reference value through proportional-integral adjustment. For example, the calculation formula is as follows: in, This is a torque reference value. , These are the parameters of the speed PI controller. This is a reference value for rotational speed. This is the measured rotational speed value.

[0120] Magnetic flux reference value The optimal operating range of the motor is crucial, directly impacting its efficiency and control stability. When the motor operates below its rated speed, maintaining a constant air gap flux is desirable to maximize the utilization of the motor's core. Therefore, the flux linkage reference value is typically set to a constant rated value. This value can be easily obtained by looking up a table or directly assigned. When the motor needs to operate above its rated speed, the back electromotive force increases with speed, limiting the maximum voltage that the inverter can provide. In this case, field weakening control is needed to actively reduce the flux linkage reference value to maintain voltage balance, allowing the motor to continue accelerating. The field weakening control algorithm dynamically calculates a suitable flux linkage setpoint based on the current speed and voltage margin.

[0121] S204. Input the state vector composed of the flux linkage measurement value, torque measurement value, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement value, and speed reference value into the main policy learning network of the TD3 algorithm model for inference to obtain the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component. Among them, the torque deviation is the difference between the torque reference value and the torque measurement value, and the flux linkage deviation is the difference between the flux linkage reference value and the flux linkage measurement value.

[0122] In this embodiment of the invention, the magnetic flux measurement value Torque measurement value Torque deviation Magnetic flux deviation Torque deviation integral Magnetic flux deviation integral Speed ​​measurement value and speed reference value The first state vector s is input into the main policy learning network (Actor) for inference to obtain the action vector. Action vector Reference values ​​including the d-axis component of the stator voltage Reference values ​​for the q-axis component of the stator voltage .Right now .

[0123] S205. Perform the Parker inverse transform on the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component to obtain the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component.

[0124] In this embodiment of the invention, the reference value for the d-axis component of the stator voltage is... Reference values ​​for the q-axis component of the stator voltage Perform the inverse Parker transform to obtain the reference value of the α-axis component of the stator voltage. Reference values ​​for the stator voltage β-axis component .

[0125] SS06. Based on the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component, space vector pulse width modulation is performed to obtain a pulse width modulation signal, which is used to control the operation of the inverter of the permanent magnet synchronous motor.

[0126] In this embodiment of the invention, the reference value of the α-axis component of the stator voltage is... Reference values ​​for the stator voltage β-axis component The input SVPWM module performs space vector pulse width modulation to obtain a pulse width modulation signal. This pulse width modulation signal is input to a three-phase inverter, which converts the input DC power into AC power to drive the permanent magnet synchronous motor.

[0127] S207. Repeat the above steps until a stop signal from the permanent magnet synchronous motor is received.

[0128] The permanent magnet synchronous motor control method provided by this invention is based on the trained master strategy learning network described in the first aspect of this invention. After training, the master strategy learning network replaces the existing flux linkage PI regulator and torque PI regulator, avoiding the problem of low control accuracy caused by the design limitations of PI regulator parameters. This improves the control accuracy of the permanent magnet synchronous motor, reduces torque ripple, avoids bearing wear caused by motor vibration, and extends bearing life. Furthermore, this invention also uses the speed variable of the outer loop (i.e., the measured speed value) as an observation value, achieving deep coordination between the outer loop (speed control loop) and the inner loop (torque and flux linkage control loop). This integrates two independent control levels into a holistic optimized intelligent control, resulting in superior system-level dynamic performance and further improving the control accuracy of the permanent magnet synchronous motor.

[0129] This invention also provides a training system for a control model of a permanent magnet synchronous motor. Figure 4 This is a schematic diagram of the structure of a training system for a permanent magnet synchronous motor control model provided by the present invention, as shown below. Figure 4 As shown, the training system for the permanent magnet synchronous motor control model includes: Heterogeneous system-on-a-chip (Heterogeneous SOC) includes a processor unit and a programmable logic unit. The processor unit's memory is equipped with a playback buffer, which is used to store the experience pool.

[0130] A cloud server is used to execute the training method provided in any of the foregoing embodiments of the present invention and to deploy the trained master policy learning network (Actor) into a programmable logic unit.

[0131] The cloud server communicates with the processor unit, enabling functions such as data collection, remote monitoring, model training, and collaborative optimization. Cloud communication transforms the permanent magnet synchronous motor (PMSM) control system from an isolated intelligent device into a continuously learning and collaboratively optimizing intelligent network. This not only solves the problem of limited edge computing capabilities but also opens up new possibilities such as adaptive control, predictive maintenance, and global optimization.

[0132] In some embodiments of the present invention, such as Figure 4 As shown, the processor unit also includes a speed PI regulator and a reward calculation module, and the programmable logic unit also includes an inverse Park transform module, a space vector pulse width modulation module (SVPWM), an analog-to-digital converter (ADC), a Clarke transform module (Clarke transform and per-unit), and a torque flux calculation module (…). Figure 4 The module includes torque and flux estimation and velocity-position calculation.

[0133] The speed and position calculation module is used to calculate the rotor's position measurement value and rotor speed measurement value based on the sensor's acquired signals. The specific calculation process can be referred to in the aforementioned embodiments, and will not be repeated here.

[0134] The analog-to-digital converter module is used to acquire the three-phase current of the permanent magnet synchronous motor and convert the three-phase current into digital signals.

[0135] The Clarke transform module is used to perform Clarke transform on the digital signals of three-phase currents to obtain measured values ​​of the stator current α-axis component and the stator current β-axis component. In one specific embodiment, the Clarke transform module can be used to normalize the measured values ​​of the stator current α-axis component and the stator current β-axis component.

[0136] The torque flux linkage calculation module is used to calculate the stator flux linkage measurement value and torque measurement value based on the measured values ​​of the stator current α-axis component and the stator current β-axis component. The specific calculation process can be referred to the aforementioned embodiments, and will not be repeated here.

[0137] The speed PI regulator is used to calculate the flux linkage reference value and torque reference value based on the speed measurement value and the speed reference value. The specific calculation process can be referred to the aforementioned embodiments, and will not be repeated here.

[0138] The reward calculation module is used to calculate the reward for the action vector taken under the first state vector in the current time step based on the reference values ​​of torque deviation, flux deviation, stator voltage d-axis component and stator voltage q-axis component, and to calculate the cumulative discount reward generated by all time steps in each round within the preset round as the reward. The specific calculation process can be referred to the aforementioned embodiments, and will not be repeated here.

[0139] In some embodiments of the invention, since the trained master policy learning network has a large number of parameters and high precision, it is time-consuming to perform calculations using high-precision parameters during inference in the programmable logic unit. The parameters of the master policy learning network can be converted into bit unsigned numbers to simplify the parameters and transmit them to the processor unit. The processor unit transmits the simplified parameters to the master policy learning network of the programmable logic unit through a high-speed bus to complete the deployment and speed up the inference speed of the master policy learning network.

[0140] The present invention also provides a training device for a permanent magnet synchronous motor control model, comprising: An experience sample acquisition module is used to acquire experience samples required for training from an experience pool. The experience samples include a first state vector, an action vector inferred by the main policy learning network of the TD3 algorithm model based on the first state vector, a second state vector at the next time step, and a reward for taking the action vector under the first state vector. The state vector includes flux linkage measurement value, torque measurement value, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement value, and speed reference value. The action vector includes reference values ​​for the d-axis component and the q-axis component of the stator voltage. The model training module is used to train the TD3 algorithm model using the empirical samples and update the parameters of the TD3 algorithm model until the reward of the TD3 algorithm model is greater than the reward threshold or the number of training rounds reaches the maximum number of training rounds. The master policy learning network after training is used to predict the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component during the operation of the permanent magnet synchronous motor.

[0141] In some embodiments of the present invention, the TD3 algorithm model includes a main policy learning network, a first value evaluation network, a second value evaluation network, a target policy learning network, a first target value evaluation network, and a second target value evaluation network. The model training module includes: The sampling submodule is used to sample from the experience pool to obtain batch sample data including multiple experience samples; The first parameter update submodule is used to input the batch sample data into the TD3 algorithm model for training and update the parameters of the first value evaluation network and the second value evaluation network. The second parameter update submodule is used to update the parameters of the main policy learning network every preset number of times the parameters of the first value evaluation network and the second value evaluation network are updated. The third parameter update submodule is used to update the parameters of the target policy learning network, the first target value evaluation network, and the second target value evaluation network using a soft update method. The repeat execution submodule is used to repeatedly execute the above steps until the reward of the TD3 algorithm model is greater than the reward threshold or the number of training rounds reaches the maximum number of training rounds. Each round includes multiple time steps, and each time step includes the entire process of the TD3 algorithm model from receiving experience samples, inferring action vectors, returning rewards, and the state vector of the next time step.

[0142] In some embodiments of the present invention, the first parameter update submodule includes: The first action vector calculation unit is used to input the second state vector into the target policy learning network for inference to obtain the first action vector. A noise-adding unit is used to add truncation noise to the first action vector to obtain the target action vector; The first action value calculation unit is used to input the target action vector into the first target value evaluation network and calculate the first action value of the first action vector. The second action value calculation unit is used to input the target action vector into the second target value evaluation network and calculate the second action value of the first action vector; The value-taking unit is used to take the smaller of the first action value and the second action value as the target action value; The target evaluation value calculation unit is used to calculate the discount sum of the reward and the target action value to obtain the target evaluation value; The third action value calculation unit is used to input the action vector corresponding to the first state vector into the first value evaluation network and calculate the third action value of the action vector corresponding to the first state vector. The fourth action value calculation unit is used to input the action vector corresponding to the first state vector into the second value evaluation network and calculate the fourth action value of the action vector corresponding to the first state vector. The first parameter update unit is used to calculate the parameters of the first value evaluation network that minimize the error between the target evaluation value and the third action value using the gradient descent algorithm, and update the parameters of the first value evaluation network. The second parameter update unit is used to calculate the parameters of the second value evaluation network that minimize the error between the target evaluation value and the value of the fourth action using the gradient descent algorithm, and to update the parameters of the second value evaluation network.

[0143] In some embodiments of the present invention, the second parameter update submodule includes: The second action vector calculation unit is used to input the second state vector into the main policy learning network for inference every preset number of parameter updates of the first value evaluation network and the second value evaluation network to obtain the second action vector. The fifth action value calculation unit is used to input the second action vector into the first value evaluation network and calculate the fifth action value of the second action vector; The third parameter update unit is used to calculate the parameters of the main policy learning network that maximizes the value of the fifth action using the gradient ascent algorithm, and to update the parameters of the main policy learning network.

[0144] In some embodiments of the present invention, the repeat execution submodule includes: The first judgment unit is used to determine whether the time step of the current round has reached the maximum time step of a single round after updating the parameters of the target policy learning network, the first target value evaluation network and the second target value evaluation network using a soft update method. The second judgment unit is used to end the training of the current round if the time step of the current round reaches the maximum time step of a single round, and to determine whether the number of training rounds has reached the evaluation interval. The first execution unit is used to return to the step of sampling from the experience pool to obtain batch sample data including multiple experience samples if the time step of the current round does not reach the maximum time step of a single round. The second execution unit is used to perform the step of determining whether the number of training rounds is greater than the maximum number of training rounds if the number of training rounds has not reached the evaluation interval. The model running unit is used to retrieve test samples from the experience pool and run the TD3 algorithm model for a preset number of rounds if the number of training rounds reaches the evaluation interval. The reward calculation unit is used to calculate the cumulative discount reward generated at all time steps in each round within the preset round as the reward; The mean calculation unit is used to calculate the average value of all returns in the preset round; The third judgment unit is used to determine whether the average of all returns is greater than the return threshold; The first termination unit is used to terminate the training process if the condition is met. The fourth judgment unit is used to determine whether the number of rounds in the current round is greater than the maximum number of training rounds if the condition is not met. The second termination unit is used to terminate the training process if the condition is met. The third execution unit is used to, if not, return to the step of sampling from the experience pool to obtain batch sample data including multiple experience samples.

[0145] In some embodiments of the present invention, the sampling submodule includes: An initialization unit is used to initialize the three-phase current, rotor speed measurement value, and rotor position measurement value of the permanent magnet synchronous motor. The first calculation unit is used to calculate the stator flux linkage measurement and torque measurement based on the three-phase current and the rotor position measurement; The second calculation unit is used to input the measured speed value and the speed reference value into the speed PI regulator to calculate the flux reference value and the torque reference value; The third calculation unit is used to input the first state vector, composed of the flux linkage measurement value, torque measurement value, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement value, and speed reference value, into the main policy learning network for inference, so as to obtain the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component. Among them, the torque deviation is the difference between the torque reference value and the torque measurement value, and the flux linkage deviation is the difference between the flux linkage reference value and the flux linkage measurement value. The fourth calculation unit is used to perform the Parker inverse transformation on the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component to obtain the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component. The modulation unit is used to perform space vector pulse width modulation based on the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component to obtain a pulse width modulation signal, which is used to control the operation of the inverter of the permanent magnet synchronous motor. The reward calculation unit is used to calculate the reward for taking the action vector under the first state vector in the current time step based on the reference values ​​of torque deviation, flux deviation, stator voltage d-axis component and stator voltage q-axis component. The fourth execution unit is used to collect the three-phase current, rotor speed measurement value and rotor position measurement value of the permanent magnet synchronous motor, and return to execute the step of calculating the stator flux linkage measurement value and torque measurement value based on the three-phase current and rotor position measurement value to obtain the second state vector of the next time step; The experience storage unit is used to combine the action vector composed of the state vector, the reference value of the stator voltage d-axis component, and the reference value of the stator voltage q-axis component, the reward, and the second state vector of the next time step into an experience sample and store it in the experience pool. The fifth judgment unit is used to determine whether the number of experience samples in the experience pool has reached the preset number required for training. A sampling unit is used to sample from the experience pool if the condition is met, to obtain batch sample data including multiple experience samples. The fifth execution unit is used to, if not, return to the steps of initializing the three-phase current, rotor speed measurement value, and rotor position measurement value of the permanent magnet synchronous motor, until the experience samples in the experience pool have reached the preset number required for training.

[0146] In some embodiments of the present invention, the reward for taking the action vector under the first state vector in the current time step is calculated based on the torque deviation, flux linkage deviation, reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component, and the calculation formula is as follows: in, The reward for taking the action vector under the first state vector at the k-th time step, where Q1, Q2, and R are hyperparameters. For torque deviation, For magnetic flux deviation, This is the reference value for the d-axis component of the stator voltage. This is the reference value for the q-axis component of the stator voltage.

[0147] The training device for the permanent magnet synchronous motor control model described above can execute the training method for the permanent magnet synchronous motor control model provided in the foregoing embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the training method for the permanent magnet synchronous motor control model.

[0148] The present invention also provides a permanent magnet synchronous motor control device, which includes a master policy learning network trained based on any of the foregoing embodiments of the present invention, comprising: The parameter determination module is used to determine the three-phase current, rotor speed measurement value, and rotor position measurement value of the permanent magnet synchronous motor during operation. The measurement value calculation module is used to calculate the stator flux linkage measurement value and torque measurement value based on the three-phase current and rotor position measurement value; The reference value calculation module is used to input the speed measurement value and speed reference value into the speed PI regulator to calculate the flux reference value and torque reference value; The inference module is used to input the state vector composed of flux linkage measurement value, torque measurement value, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement value and speed reference value into the target policy learning network of the TD3 algorithm model for inference, so as to obtain the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component. Among them, the torque deviation is the difference between the torque reference value and the torque measurement value, and the flux linkage deviation is the difference between the flux linkage reference value and the flux linkage measurement value. The Parker inverse transform module is used to perform Parker inverse transform on the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component to obtain the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component. The space vector modulation module is used to perform space vector pulse width modulation based on the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component to obtain a pulse width modulation signal. The pulse width modulation signal is used to control the operation of the inverter of the permanent magnet synchronous motor. The repeat execution module is used to repeatedly execute the above steps until a stop signal from the permanent magnet synchronous motor is received.

[0149] The aforementioned permanent magnet synchronous motor control device can execute the permanent magnet synchronous motor control method provided in the foregoing embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the permanent magnet synchronous motor control method.

[0150] Figure 5 This is a schematic diagram of an electronic device provided by the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0151] like Figure 5 As shown, the electronic device includes at least one processor and a memory, such as a read-only memory (ROM) or a random access memory (RAM), communicatively connected to the at least one processor. The memory stores computer programs executable by the at least one processor. The processor can perform various appropriate actions and processes based on the computer programs stored in the ROM or loaded from memory cells into the RAM. The RAM may also store various programs and data required for the operation of the electronic device. The processor, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0152] Multiple components in an electronic device are connected to an I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0153] A processor can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processors include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor performs the various methods and processes described above, such as training methods for permanent magnet synchronous motor control models or methods for controlling permanent magnet synchronous motors.

[0154] In some embodiments, the training method for the permanent magnet synchronous motor control model or the permanent magnet synchronous motor control method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via a ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the training method for the permanent magnet synchronous motor control model or the permanent magnet synchronous motor control method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the training method for the permanent magnet synchronous motor control model or the permanent magnet synchronous motor control method by any other suitable means (e.g., by means of firmware).

[0155] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0156] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0157] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0158] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0159] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0160] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0161] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements a training method for a permanent magnet synchronous motor control model or a permanent magnet synchronous motor control method as provided in any embodiment of this application.

[0162] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0163] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0164] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A training method for a control model of a permanent magnet synchronous motor, characterized in that, include: The training requires experience samples from the experience pool. The experience samples include a first state vector, an action vector inferred by the main policy learning network of the TD3 algorithm model based on the first state vector, a second state vector at the next time step, and a reward for taking the action vector under the first state vector. The state vector includes flux linkage measurement value, torque measurement value, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement value, and speed reference value. The action vector includes reference values ​​for the d-axis component and the q-axis component of the stator voltage. The TD3 algorithm model is trained using the empirical samples, and the parameters of the TD3 algorithm model are updated until the reward of the TD3 algorithm model is greater than the reward threshold or the number of training rounds reaches the maximum number of training rounds. The master policy learning network after training is used to predict the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component during the operation of the permanent magnet synchronous motor.

2. The training method for the permanent magnet synchronous motor control model according to claim 1, characterized in that, The TD3 algorithm model includes a main policy learning network, a first value evaluation network, a second value evaluation network, a target policy learning network, a first target value evaluation network, and a second target value evaluation network. The TD3 algorithm model is trained using the empirical samples, and its parameters are updated until the reward of the TD3 algorithm model exceeds a reward threshold or the maximum number of training epochs is reached. This includes: Samples are taken from the experience pool to obtain batch sample data including multiple experience samples; The batch sample data is input into the TD3 algorithm model for training, and the parameters of the first value evaluation network and the second value evaluation network are updated. The parameters of the main policy learning network are updated every preset number of times the parameters of the first value evaluation network and the second value evaluation network are updated. The parameters of the target policy learning network, the first target value evaluation network, and the second target value evaluation network are updated using a soft update method. Repeat the above steps until the reward of the TD3 algorithm model is greater than the reward threshold or the number of training rounds reaches the maximum number of training rounds. Each round includes multiple time steps, and each time step includes the entire process of the TD3 algorithm model from receiving experience samples, inferring action vectors, returning rewards, and the state vector of the next time step.

3. The training method for the permanent magnet synchronous motor control model according to claim 2, characterized in that, The batch sample data is input into the TD3 algorithm model for training, updating the parameters of the first value evaluation network and the second value evaluation network, including: The second state vector is input into the target policy learning network for inference to obtain the first action vector; Add truncated noise to the first action vector to obtain the target action vector; The target action vector is input into the first target value evaluation network to calculate the first action value of the first action vector; The target action vector is input into the second target value evaluation network to calculate the second action value of the first action vector; The smaller of the first action value and the second action value is taken as the target action value; Calculate the sum of the discounts between the reward and the value of the target action to obtain the target evaluation value; Input the action vector corresponding to the first state vector into the first value evaluation network to calculate the third action value of the action vector corresponding to the first state vector. Input the action vector corresponding to the first state vector into the second value evaluation network to calculate the fourth action value of the action vector corresponding to the first state vector. The parameters of the first value evaluation network are calculated using the gradient descent algorithm to minimize the error between the target evaluation value and the value of the third action, and the parameters of the first value evaluation network are updated. The parameters of the second value evaluation network are calculated using the gradient descent algorithm to minimize the error between the target evaluation value and the value of the fourth action, and the parameters of the second value evaluation network are updated.

4. The training method for the permanent magnet synchronous motor control model according to claim 2, characterized in that, Every preset number of parameter updates to the first and second value evaluation networks, the parameters of the main policy learning network are updated, including: When the parameters of the first value evaluation network and the second value evaluation network are updated a preset number of times, the second state vector is input into the main policy learning network for inference to obtain the second action vector. The second action vector is input into the first value evaluation network to calculate the fifth action value of the second action vector; The parameters of the master policy learning network that maximize the value of the fifth action are calculated using the gradient ascent algorithm, and the parameters of the master policy learning network are updated.

5. The training method for the permanent magnet synchronous motor control model according to claim 2, characterized in that, Repeat the above steps until the reward of the TD3 algorithm model is greater than the reward threshold or the number of rounds in this round has reached the maximum number of training rounds, including: After updating the parameters of the target policy learning network, the first target value evaluation network, and the second target value evaluation network using a soft update method, it is determined whether the time step of this round has reached the maximum time step of a single round. If the time step of this round reaches the maximum time step of a single round, then end the training for this round and determine whether the number of training rounds has reached the evaluation interval. If the time step of this round does not reach the maximum time step of a single round, then return to the step of sampling from the experience pool to obtain batch sample data including multiple experience samples; If the number of training rounds has not reached the evaluation interval, then proceed to determine whether the number of rounds in this round is greater than the maximum number of training rounds. If the number of training rounds reaches the evaluation interval, then a test sample is taken from the experience pool, and the TD3 algorithm model is run for a preset number of rounds. Calculate the cumulative discount reward generated at all time steps in each round within the preset round as the return; Calculate the average of all rewards for the preset round; Determine if the average of all returns is greater than the return threshold; If so, then end the training process; If not, then determine whether the number of rounds in this round is greater than the maximum number of training rounds; If so, then end the training process; If not, return to the step of sampling from the experience pool to obtain batch sample data including multiple experience samples.

6. The training method for the permanent magnet synchronous motor control model according to claim 2, characterized in that, Samples are drawn from the experience pool to obtain batch sample data comprising multiple experience samples, including: Initialize the three-phase current, rotor speed measurement value, and rotor position measurement value of the permanent magnet synchronous motor; Calculate the stator flux linkage and torque measurements based on the three-phase current and rotor position measurements; Input the measured speed value and the speed reference value into the speed PI controller to calculate the flux linkage reference value and torque reference value; The first state vector, composed of the flux linkage measurement value, torque measurement value, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement value, and speed reference value, is input into the main policy learning network for inference to obtain the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component. Among them, the torque deviation is the difference between the torque reference value and the torque measurement value, and the flux linkage deviation is the difference between the flux linkage reference value and the flux linkage measurement value. By performing the inverse Parker transform on the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component, the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component are obtained. Space vector pulse width modulation is performed based on the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component to obtain a pulse width modulation signal, which is used to control the operation of the inverter of the permanent magnet synchronous motor. The reward for taking the action vector under the first state vector in this time step is calculated based on the reference values ​​of torque deviation, flux deviation, stator voltage d-axis component and stator voltage q-axis component. The three-phase current, rotor speed measurement value, and rotor position measurement value of the permanent magnet synchronous motor are collected, and the process returns to the step of calculating the stator flux linkage measurement value and torque measurement value based on the three-phase current and rotor position measurement value to obtain the second state vector of the next time step; The action vector composed of the state vector, the reference value of the stator voltage d-axis component, and the reference value of the stator voltage q-axis component, the reward, and the second state vector of the next time step are combined into an experience sample and stored in the experience pool. Determine whether the number of experience samples in the experience pool has reached the preset number required for training. If so, sample from the experience pool to obtain batch sample data including multiple experience samples; If not, return to the steps of initializing the three-phase current, rotor speed measurement, and rotor position measurement of the permanent magnet synchronous motor until the number of experience samples in the experience pool reaches the preset number required for training.

7. The training method for the permanent magnet synchronous motor control model according to claim 6, characterized in that, The reward for taking the action vector under the first state vector in this time step is calculated based on the reference values ​​of torque deviation, flux linkage deviation, stator voltage d-axis component, and stator voltage q-axis component. The calculation formula is as follows: in, The reward for taking the action vector under the first state vector at the k-th time step, where Q1, Q2, and R are hyperparameters. For torque deviation, For magnetic flux deviation, This is the reference value for the d-axis component of the stator voltage. This is the reference value for the q-axis component of the stator voltage.

8. A control method for a permanent magnet synchronous motor, characterized in that, The master policy learning network trained based on the training method according to any one of claims 1-7 includes: During the operation of the permanent magnet synchronous motor, the three-phase current, rotor speed measurement value, and rotor position measurement value of the permanent magnet synchronous motor are determined; Calculate the stator flux linkage and torque measurements based on the three-phase current and rotor position measurements; Input the measured speed value and the speed reference value into the speed PI controller to calculate the flux linkage reference value and torque reference value; The state vector composed of flux linkage measurement, torque measurement, torque deviation, flux linkage deviation, torque deviation integral, flux linkage deviation integral, speed measurement, and speed reference value is input into the main policy learning network of the TD3 algorithm model for inference, so as to obtain the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component. Among them, the torque deviation is the difference between the torque reference value and the torque measurement value, and the flux linkage deviation is the difference between the flux linkage reference value and the flux linkage measurement value. By performing the inverse Parker transform on the reference values ​​of the stator voltage d-axis component and the stator voltage q-axis component, the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component are obtained. Space vector pulse width modulation is performed based on the reference values ​​of the stator voltage α-axis component and the stator voltage β-axis component to obtain a pulse width modulation signal, which is used to control the operation of the inverter of the permanent magnet synchronous motor. Repeat the above steps until a stop signal from the permanent magnet synchronous motor is received.

9. A training system for a control model of a permanent magnet synchronous motor, characterized in that, include: A heterogeneous system-on-a-chip, the heterogeneous system-on-a-chip including a processor unit and a programmable logic unit, the processor unit including a playback buffer, the playback buffer being used to store an experience pool; A cloud server is configured to execute the training method as described in any one of claims 1-7 and deploy the trained master policy learning network to the programmable logic unit.

10. The training system for the permanent magnet synchronous motor control model according to claim 9, characterized in that, The processor unit also includes a speed PI regulator and a reward calculation module, and the programmable logic unit also includes a Parker inverse transform module, a space vector pulse width modulation module, an analog-to-digital converter module, a Clarke transform module, a torque flux calculation module, and a speed and position calculation module. The speed and position calculation module is used to calculate the rotor's position measurement value and rotor speed measurement value based on the sensor's acquired signals; The analog-to-digital converter module is used to acquire the three-phase current of the permanent magnet synchronous motor and convert the three-phase current into digital signals; The Clarke transform module is used to perform Clarke transform on the digital signal of the three-phase current to obtain the measured values ​​of the α-axis component and the β-axis component of the stator current. The torque flux linkage calculation module is used to calculate the stator flux linkage measurement value and torque measurement value based on the measured values ​​of the stator current α-axis component and the stator current β-axis component. The speed PI regulator is used to calculate the flux linkage reference value and torque reference value based on the measured speed value and the speed reference value; The reward calculation module is used to calculate the reward for taking the action vector under the first state vector in the current time step based on the reference values ​​of torque deviation, flux deviation, stator voltage d-axis component and stator voltage q-axis component, and to calculate the cumulative discount reward generated by all time steps in each round within the preset round as the reward.

Images