A TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor

By introducing TD3-assisted PI self-tuning vector control into the outer speed loop of the aircraft induction motor FOC, and combining the PI regulator and TD3 agent, the performance degradation caused by parameter drift and load mutation is solved, achieving control effects with fast response, low overshoot and high steady-state accuracy.

CN123122379APending Publication Date: 2026-11-17GUIZHOU UNIV +1
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202611636223.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-22
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In the existing FOC speed outer loop control of aircraft induction motors, the fixed parameters of traditional PI controllers make it difficult to adapt to parameter drift and load changes in wide speed range operation. This makes it difficult to achieve fast response, low overshoot and high steady-state accuracy at the same time. Furthermore, the direct use of TD3 reinforcement learning controller has the problem of insufficient steady-state reliability.

Method used

The TD3-assisted PI self-tuning vector control method is adopted. By retaining the PI regulator as the deterministic main closed loop in the speed outer loop, the TD3 agent is introduced as the auxiliary regulation unit. A composite reward function is constructed for training to generate the q-axis auxiliary current and online gain correction, thereby achieving parameter adaptive compensation.

Benefits of technology

Under conditions of parameter drift and load abrupt changes, the system achieves rapid response, low overshoot, and high steady-state accuracy of the aircraft motor, thereby improving the dynamic performance and disturbance rejection capability of the system and meeting the reliability requirements of airborne electric drives.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123122379A_ABST
    Figure CN123122379A_ABST
Patent Text Reader

Abstract

This application provides a TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor, relating to the field of aircraft electric drive motor control technology. This method retains the PI regulator as a deterministic main closed loop in the rotor flux-oriented (FOC) speed loop, and deploys a TD3 agent in the speed loop bypass. The agent takes speed error, error differential, actual speed, q-axis current feedback, historical auxiliary current, and PI parameter status as inputs, and provides the q-axis auxiliary current, proportional gain correction, and integral gain correction in an incremental manner. Simultaneously, it constructs a reward evaluation rule that varies with the operating phase, emphasizing rapid torque establishment during startup and compensation recovery, overshoot constraint, and oscillation suppression during the approach to the target speed phase. The technical solution provided in this application can reduce the impact of training exploration on the stability of the outer speed loop and improve response speed, tracking accuracy, and disturbance recovery capability under multiple operating conditions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of aviation motor control technology, and more specifically, to a TD3-assisted PI self-tuning vector control method for the outer speed loop of an aviation motor. Background Technology

[0002] As airborne systems in passenger aircraft evolve towards greater electrification and full electrification, the loads driven by electric motors in actuation, fuel delivery, environmental control, and auxiliary power systems are constantly increasing. Drive systems need to handle starting, speed changes, sudden load changes, and long-term continuous operation under high power density and high reliability requirements. Induction motors, with their robust structure, lack of permanent magnet demagnetization risk, strong environmental adaptability, and low maintenance costs, are suitable for use in wide-body passenger aircraft and related airborne electric drive scenarios.

[0003] In induction motor speed control, the Forward Coefficient (FOC) transforms three-phase AC quantities into a dq coordinate system and utilizes an outer speed loop and an inner current loop to achieve speed regulation and current tracking, respectively. Existing speed outer loops typically employ fixed-parameter PI controllers, which are simple to implement and have good stability. However, when motor parameters are perturbed, loads are suddenly increased, or the target speed changes rapidly, the fixed gain struggles to simultaneously achieve short rise time, low overshoot, steady-state accuracy, and rapid disturbance recovery. Deep reinforcement learning, on the other hand, can learn nonlinear compensation patterns from continuous interactive samples, providing a new path for self-adjustment of motor speed loop parameters. The TD3 algorithm, through a two-valued function network, target policy smoothing, and delayed policy updates, mitigates value overestimation and training oscillations, making it suitable for handling continuous action variables such as compensation current and control parameters. However, directly replacing the entire speed loop with a reinforcement learning controller may cause speed fluctuations, current overruns, or even closed-loop instability during the initial exploratory actions of training, failing to meet the reliability requirements of airborne electric drives.

[0004] Chinese patent CN115903457A discloses a control method for low-wind-speed permanent magnet synchronous wind turbines based on deep reinforcement learning, belonging to the wind power field. It employs the TD3 deep reinforcement learning algorithm to design a speed tracking controller: An agent based on the TD3 algorithm is constructed according to the environmental conditions of the low-wind-speed permanent magnet synchronous wind turbine; a reward function is designed and the agent is trained; the trained agent model is then ported into the main control chip of the turbine-side converter to establish an actual speed tracking control system based on the main control chip, achieving speed tracking control; a current tracking controller is designed using zero d-axis current control and current feedforward decoupling control strategies to achieve current tracking control. The proposed solution, which uses a TD3 intelligent agent to replace the PI speed controller, suffers from the following unresolved technical issues: First, relying on the black-box output control quantity of reinforcement learning lacks a backup control channel when training is insufficient or unknown operating conditions are encountered, compromising system steady-state accuracy and operational safety. Second, the lack of online correction for PI parameters and the fixed controller gain make it difficult to adapt to parameter drift caused by temperature rise, magnetic saturation, and sudden load changes during wide-speed operation of the aircraft induction motor. Third, a single reward function cannot coordinate the conflicts between multiple objectives such as speed, overshoot suppression, steady-state accuracy, and disturbance recovery in stages. To address these issues, a speed outer-loop control scheme that combines the guaranteed stability of PI with the online adaptive capability of reinforcement learning is urgently needed in the FOC speed outer loop control of aircraft induction motors. This scheme would overcome the shortcomings of traditional PI controllers, such as fixed parameters and difficulty in adapting to strong disturbances across a wide speed range, while avoiding the black-box control risks and insufficient steady-state reliability resulting from the complete replacement of PI by reinforcement learning agents like TD3. This would enable the system to simultaneously meet the comprehensive performance requirements of fast response, low overshoot, high steady-state accuracy, and fast disturbance recovery under parameter drift and sudden load changes. Summary of the Invention

[0005] This application provides a TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor, which can achieve fast response, low overshoot, high steady-state accuracy, and fast disturbance recovery of the aircraft motor under conditions of parameter drift and load abrupt change.

[0006] In a first aspect, this application provides a TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor, comprising the following steps: Step 1: Establish the FOC control object model of the aircraft motor and form the speed outer loop state sampling set: Based on the electromechanical coupling relationship of the aircraft motor, establish a dynamic model, construct a rotor flux orientation double closed loop control structure, and obtain the state variables used for TD3 auxiliary regulation. Step 2, construct the TD3-assisted PI self-tuning speed outer loop: using the speed loop PI regulator as the basic control channel, the state quantity is used as the input of the TD3 agent, and the agent is configured to generate the q-axis auxiliary current on the PI output side and simultaneously provide the online correction amount of the PI proportional gain and integral gain; Step 3: Complete policy training and embed it into the velocity outer loop using phased rewards: Establish a composite reward function that simultaneously reflects speed, overshoot, steady-state error and disturbance recovery, train the TD3 policy network through joint simulation, and deploy the trained Actor policy into the TD3-assisted PI self-tuning velocity outer loop.

[0007] Preferably, in step 1, establishing the FOC control object model of the aircraft motor and forming the speed outer loop state sampling set specifically includes: Step 1.1: Establish a dynamic model based on the stator and rotor flux linkage equations, voltage balance equations, electromagnetic torque equations, and mechanical motion equations of the induction motor. This model is used to describe the changes in speed, current, and torque of the airborne induction motor during startup, speed change, and load disturbance. Step 1.2: Construct an FOC dual closed-loop control structure according to the rotor flux orientation principle. Transform the three-phase electrical quantities into a synchronous rotating dq coordinate system through Clarke transformation and Park transformation. The q-axis current is given by the speed outer loop, and fast tracking is completed by the d-axis and q-axis current inner loops. In the FOC dual closed-loop control structure, the d-axis current is used to maintain the rotor flux, and the q-axis current is used to control the electromagnetic torque. Step 1.3: Sample the reference speed and feedback speed and calculate the speed error accordingly. Normalize the speed error, error difference, feedback speed, q-axis current, q-axis auxiliary current, and current PI proportional gain and integral gain, and combine them to form the state input of the seven-dimensional TD3 agent. The sampled and normalized physical quantities together constitute the speed outer loop state sampling set, which is the state quantity obtained for TD3 auxiliary regulation.

[0008] Preferably, in step 2, constructing the TD3-assisted PI self-tuning speed outer loop specifically includes: The TD3 agent takes the state quantity as input, uses the output of the PI regulator as the q-axis base current, and uses the output of the TD3 agent as the q-axis auxiliary current. The two are superimposed to obtain the q-axis current reference value of the joint velocity outer loop. The proportional gain correction term and integral gain correction term generated by the TD3 agent are superimposed on the PI reference gain, so that the speed loop parameters can be adjusted online according to the operating status. The TD3-assisted PI self-tuning speed outer loop is constructed by the superposition relationship between the PI basic channel and the TD3 auxiliary channel.

[0009] Preferably, step 2 further includes: setting the agent's actions as three-dimensional continuous increments, corresponding to the q-axis auxiliary current increment, proportional gain correction increment, and integral gain correction increment, respectively, and reducing the impact of sudden changes in absolute control quantity on the closed loop through recursive updates.

[0010] Preferably, saturation limits are set for the q-axis auxiliary current, the proportional gain correction term, and the integral gain correction term, respectively, so that the control compensation and online parameters are always within a preset safety range. The PI base channel and the TD3 auxiliary channel form a joint control relationship through algebraic superposition, and the joint control relationship is expressed as follows:

[0011]

[0012]

[0013] in, Indicates the first The q-axis current reference value output by the controller in each sampling period; Indicates the first The q-axis current output by the PI base channel in each sampling period; Indicates the first TD3 auxiliary current per sampling period; and They represent the first Online proportional gain and online integral gain for each sampling period, The gain symbol is used. As a proportion, For integration; and These represent the reference gains of the velocity loop proportional gain P and integral gain I, respectively. and They represent the first The proportional gain correction term and integral gain correction term are given by TD3 for each sampling period. This represents the discrete sampling period number.

[0014] Preferably, in step 3, completing policy training and embedding the velocity outer loop using staged rewards specifically includes: Step 3.1: Establish the state vector of the TD3 agent. The state vector is used to characterize the speed tracking error, error change trend, motor operating state, q-axis current response, auxiliary quantity at the previous moment, and current PI gain. The state vector, error, and error difference are represented as follows:

[0015]

[0016]

[0017] in, For the first The state vector of the TD3 agent in each sampling period; For the first Speed ​​error per sampling period; For the first Error difference for each sampling period; For the first Reference rotational speed for each sampling period; For the first The actual mechanical speed of the motor in each sampling period; For the first q-axis current feedback per sampling period; For the first The TD3 auxiliary current is obtained by recursion over each sampling period; and The first Online proportional gain and integral gain for each sampling period; Step 3.2: Set the three-dimensional continuous motion vector and use the saturation limiting function to recursively update the auxiliary current and PI gain correction term; Step 3.3: Construct a normalized operating phase according to the degree to which the actual rotational speed approaches the reference rotational speed. This operating phase then generates a rapid response weight for the initial stage and a convergence recovery weight for the later stage. This allows the reward objective to gradually transition from rapid error reduction in the startup phase to compensation recovery, overshoot suppression, and oscillation reduction in the steady-state phase. This leads to the establishment of a composite reward function that simultaneously reflects speed, overshoot, steady-state error, and disturbance recovery. The normalized operating phase, rapid response weight for the initial stage, and convergence recovery weight for the later stage are expressed as follows:

[0018]

[0019]

[0020] in, For the first Normalized running phase for each sampling period; For the first Each sampling period has a fast response weight in the early stage; For the first The weights are recovered during the later stages of each sampling period; For the first Actual mechanical rotation speed per sampling period For the first The reference speed is measured in sampling periods; sat(.) is the saturation limiting function; when the motor is in the initial starting stage... The speed is relatively large, and when the motor gradually approaches the reference speed... Gradually increase This refers to the discrete sampling period number; Step 3.4: In the joint simulation, the TD3 agent is trained using a dual-Critic network, delayed policy update, target policy smoothing, experience replay, and target network soft update mechanism. When the rotation speed or current exceeds the limit during training, the current cycle is terminated and a penalty is applied. After training is completed, the training of the TD3 policy network is finished.

[0021] Preferably, after training the TD3 policy network, the trained Actor policy network is deployed to the TD3 agent in the TD3-assisted PI self-tuning speed outer loop. The Actor policy network outputs an action vector based on the input state vector, expressed as:

[0022] in, (.) represents the Actor policy network after training. Its network parameters, For the first The input state vector for each sampling period, To output the action vector; through the above control strategy, the PI regulator provides an interpretable and stable basic speed closed loop. The TD3 agent only learns the incremental correction of the auxiliary current and PI gain, thereby reducing the continuous action search space and improving the speed, smoothness, tracking accuracy and load disturbance resistance of the speed outer loop of the airborne induction motor of the wide-body passenger aircraft.

[0023] Preferably, the aircraft motor is an airborne induction motor, and the rotor flux orientation is obtained by a rotor flux observer to determine the rotor flux position.

[0024] Secondly, this application provides a computer device, the computer device including a memory and a processor, the memory storing code, the processor being configured to acquire the code and execute the above-described TD3-assisted PI self-tuning vector control method for the outer loop of aircraft motor speed.

[0025] Thirdly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned TD3-assisted PI self-tuning vector control method for the outer loop of aircraft motor speed.

[0026] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: The TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor provided in this application can achieve rapid response, low overshoot, high steady-state accuracy, and rapid disturbance recovery of the aircraft motor under conditions of parameter drift and load abrupt changes. Firstly, the traditional PI regulator is retained as the deterministic main closed loop in the speed outer loop, while the TD3 agent is set as a bypass auxiliary regulating unit, enabling deterministic regulation and data-driven optimization to proceed in parallel and collaboratively. Reinforcement learning only undertakes auxiliary compensation and parameter fine-tuning, without directly replacing the PI closed loop, thereby reducing the impact of training exploration on the stability of the speed outer loop and providing a safety net control channel in scenarios where aircraft-borne electric drives have stringent reliability requirements. Secondly, the TD3 agent's actions simultaneously cover the q-axis auxiliary current, proportional gain correction term, and integral gain correction term, enabling control quantity compensation and PI parameter tuning to be completed collaboratively within the same strategy framework. This allows it to adapt to various operating scenarios such as start-up, step speed changes, and load abrupt changes, overcoming the performance degradation problem of fixed-gain PI under parameter perturbation conditions. Then, the agent's actions are updated using a three-dimensional continuous incremental recursive approach combined with saturation limiting. Each sampling period outputs only incremental rather than absolute control quantities, effectively reducing the impact of sudden changes in neural network output on the closed loop and constraining compensation and parameter correction within a safe range, thus improving engineering deployability. Finally, the phased composite reward function is optimized by adjusting the normalized operating phase. During the start-up acceleration phase, it enhances rapid error convergence and torque establishment, while during the approach to the reference speed phase, it encourages compensation recovery, overshoot suppression, and steady-state error reduction. This coordinates multi-objective conflicts within a single strategy, improving the overall dynamic quality of the speed loop. In summary, the technical solution provided in this application can achieve rapid response, low overshoot, high steady-state accuracy, and rapid disturbance recovery for aero-engines under parameter drift and load abrupt changes. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the motor FOC dual closed-loop reference structure in an embodiment of the present invention; Figure 2 This is a schematic diagram of the TD3 agent learning structure in an embodiment of the present invention; Figure 3 This is a schematic diagram of the TD3-assisted PI speed outer loop self-tuning architecture in an embodiment of the present invention; Figure 4 This is a schematic diagram of the TD3-assisted PI training process in an embodiment of the present invention; Figure 5 This is a schematic diagram of the computer device for implementing the TD3-assisted PI self-tuning vector control method for the outer loop of aircraft motor speed, as shown in this application. Figure 6 This is a verification diagram of the no-load starting dynamic performance index in an embodiment of the present invention; Figure 7 This is a verification diagram of the velocity step dynamic performance index in an embodiment of the present invention; Figure 8 This is a verification diagram of the performance index of sudden load disturbance recovery in an embodiment of the present invention. Detailed Implementation

[0028] like Figure 1-8 As shown, this embodiment takes high dynamic speed control of aircraft motors as the application scenario, and an induction motor as the controlled object, establishing a rotor flux-oriented dual closed-loop vector control reference structure. Figure 1 Based on the traditional speed outer-loop PI regulator, a TD3 intelligent agent is introduced as an auxiliary control channel. The motor operating state is used as the input of the intelligent agent, and a q-axis auxiliary current is generated by superimposing it on the PI output side, and the online correction of the PI proportional gain and integral gain is given simultaneously. Figure 2 , Figure 3 The Actor-Critic network is jointly simulated and trained using a staged composite reward function. Figure 4 The trained Actor strategy is embedded into the speed outer loop to achieve online self-tuning control; and dynamic performance verification results are given under three typical operating conditions: no-load start-up, speed step jump, and sudden load increase. Figures 6 to 8 ).

[0029] This application provides a TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor, comprising the following steps: Step 1, establishing an FOC control object model of the aircraft motor and forming a speed outer loop state sampling set: establishing a dynamic model based on the electromechanical coupling relationship of the aircraft motor, constructing a rotor flux-oriented dual closed-loop control structure, and obtaining state variables for TD3 auxiliary regulation; Step 2, constructing a TD3-assisted PI self-tuning speed outer loop: using the speed loop PI regulator as the basic control channel, using the state variables as the input of the TD3 agent, configuring the agent to superimpose a q-axis auxiliary current on the PI output side and simultaneously providing online correction values ​​for the PI proportional gain and integral gain; using staged rewards to complete strategy training and embedding it into the speed outer loop; Step 3, establishing a staged composite reward function that simultaneously reflects speed, overshoot, steady-state error, and disturbance recovery, training the TD3 policy network through co-simulation, and deploying the trained Actor policy into the TD3-assisted PI self-tuning speed outer loop.

[0030] In specific implementation, the aircraft motor is an airborne induction motor, i.e., an asynchronous motor. Its rotor does not require permanent magnets, eliminating the risk of high-temperature demagnetization and making it suitable for the airborne environment. The FOC (Field-Oriented Control) uses Clarke and Park transforms to decouple the three-phase stator current into excitation current components (d-axis) and torque current components (q-axis), achieving decoupled control similar to a DC motor. The speed outer loop refers to the outer control loop with speed as the controlled variable and q-axis current reference value as the control output; its output serves as the setpoint for the current inner loop. The TD3 (Dual-Delay Deep Deterministic Policy Gradient Algorithm) is a deep reinforcement learning algorithm suitable for continuous action spaces. It includes an Actor network and two Critic networks. It suppresses overestimation by minimizing the value of the dual Q-network, improves stability by delaying Actor network updates, and reduces value estimation variance through target policy smoothing regularization. The online correction of the PI proportional gain and integral gain refers to the real-time output of the TD3 agent to the PI controller proportional gain based on the current operating state. and integral gain The adjustment amount is not used to directly replace the PI controller.

[0031] By adopting the above technical solution, the PI controller is retained as the basic control channel and a TD3 auxiliary channel is superimposed. This not only takes advantage of the PI controller's clear structure, steady-state reliability, and ease of engineering certification, but also leverages the nonlinear mapping capability of the TD3 agent to achieve parameter adaptation and dynamic compensation. This solves the problems of training exploration endangering closed-loop safety when pure reinforcement learning replaces PI, and the difficulty of fixed-parameter PI to adapt to parameter drift in multiple operating conditions of aero-engines.

[0032] In some preferred embodiments of this application, establishing the FOC control object model of the aircraft motor and forming a speed outer loop state sampling set specifically includes: establishing a dynamic model based on the stator and rotor flux linkage equations, voltage balance equations, electromagnetic torque equations, and mechanical motion equations of the induction motor, used to describe the speed, current, and torque changes of the airborne induction motor during start-up, speed change, and load disturbance processes; constructing an FOC dual closed-loop control structure according to the rotor flux linkage orientation principle, converting the three-phase electrical quantities to a synchronous rotating dq coordinate system via Clarke and Park transformations, and generating the q-axis current command from the speed outer loop. The system performs rapid tracking using the inner loop of d-axis and q-axis currents. It samples the reference speed and feedback speed and calculates the speed error accordingly. It normalizes the speed error, error difference, feedback speed, q-axis current, q-axis auxiliary current, and the current PI proportional gain and integral gain, and combines them to form the state input of the seven-dimensional TD3 agent. The sampled and normalized physical quantities together constitute the outer loop state sampling set of the speed, which is the state quantity obtained for TD3 auxiliary regulation. In the FOC dual closed-loop control structure, the d-axis current is used to maintain the rotor flux linkage, and the q-axis current is used to control the electromagnetic torque.

[0033] It should be noted that the FOC dual-loop control structure refers to a control architecture based on field-oriented control, consisting of two closed loops connected in series: a speed outer loop and a current inner loop. Its meaning can be understood from three levels: First, field-oriented control refers to decoupling the three-phase stator current from the stationary three-phase coordinate system to the dq coordinate system, which rotates synchronously with the rotor flux linkage, through Clarke and Park transformations. This gives the AC motor decoupled control characteristics similar to those of a DC motor, where the d-axis current corresponds to the excitation component and the q-axis current corresponds to the torque component. Second, the dual-loop refers to a cascaded structure with a speed loop as the outer loop and a current loop as the inner loop. The speed outer loop takes the difference between the reference speed and the actual mechanical speed as input, which is then processed by a PI controller. The output q-axis current setpoint is used as input by the inner current loop, which takes the difference between the setpoint and the actual q-axis current feedback as input. This difference is then processed by a PI regulator with feedforward decoupling compensation to output d-axis and q-axis voltage commands. These commands are then converted into inverter switching signals via inverse Park transform and space vector pulse width modulation. Thirdly, the structure emphasizes layered coordination between the inner and outer loops on a time scale. The bandwidth of the inner current loop is much higher than that of the outer speed loop. In this scheme, the sampling period of the inner current loop is 100 microseconds, while the outer speed loop executes at the same or an integer multiple of the sampling period. The inner loop rapidly tracks the current command given by the outer loop, while the outer loop adjusts the torque command based on the speed deviation. The d-axis current setpoint is typically generated by the rotor flux setpoint via a flux regulator to maintain a constant air gap flux. This scheme retains the PI regulator as the basic channel and bypasses and superimposes the TD3 auxiliary channel on the outer speed loop of this dual-closed-loop structure, forming a TD3-assisted PI self-tuning speed outer loop.

[0034] In the specific implementation process, firstly, based on the stator and rotor flux linkage equations, voltage balance equations, electromagnetic torque equations, and mechanical motion equations of the induction motor, a dynamic model of the motor described in state space is constructed. The stator and rotor flux linkage equations are used to establish the algebraic relationship between current and flux linkage; the voltage balance equations describe the balance between voltage drop and induced electromotive force on the stator and rotor windings; the electromagnetic torque equations map the q-axis current and air gap flux linkage to the output torque; and the mechanical motion equations integrate the difference between the electromagnetic torque and the load torque using the moment of inertia and damping coefficient to obtain the mechanical speed. The output of this dynamic model is the three-phase stator current and the rotor mechanical speed. Then, based on this, a field-oriented control double-loop structure is constructed according to the rotor flux linkage orientation principle. This structure uses a rotor flux linkage observer to perform flux linkage model calculations on the stator voltage and current to obtain the spatial phase angle of the rotor flux linkage. A phase-locked loop is used to smoothly track the phase angle, and then the Clarke transformation is used to transform the three-phase stator current from a stationary three-phase coordinate system to a two-phase stationary coordinate system. The transformation matrix of the Clarke transformation is fixed as [[1,-1 / 2,-1 / 2],[0, ,- Subsequently, the Park transform is used to rotate the current components in the two-phase stationary coordinate system to the dq coordinate system, which rotates synchronously with the rotor flux linkage. The rotation angle of the Park transform is the rotor flux linkage phase angle obtained from the previous observation. After rotation, the d-axis excitation current and q-axis torque current are obtained. The d-axis current is generated by the rotor flux linkage reference through the flux linkage regulator to maintain a constant air gap flux linkage. The q-axis current is generated by the speed outer loop to control the electromagnetic torque. The d-axis and q-axis current inner loops respectively adopt a feedforward decoupling compensation plus PI regulator structure to achieve fast, zero steady-state error tracking. The output of the current inner loop is used to generate inverter switching signals to drive the motor through inverse Park transform and space vector pulse width modulation. Finally, after forming the above control structure, in each control cycle, the reference speed command, the actual mechanical speed fed back by the photoelectric encoder, the speed error obtained by subtracting the speed error from the error of the previous cycle are calculated. The error differential, the q-axis current feedback value sampled by the current sensor and low-pass filtered, the auxiliary current value recursively output by the TD3 agent in the previous cycle, and the proportional gain and integral gain of the speed loop PI regulator at the current moment are sampled. The sampling period is set to 100 microseconds to match the update rate of the inner current loop. Then, the maximum and minimum value normalization method is used for each physical quantity. For example, when the speed is normalized to the range of [-1,1], the rated speed is used as the maximum value reference. The q-axis current is used as the motor peak current reference. The PI gain is scaled by multiples of their respective reference gains. The above seven-dimensional vectors after normalization form the state input vector of the TD3 agent. This state vector also includes the current control deviation, deviation change trend, operating state point, current response level, historical auxiliary intervention amount, and current controller parameters, providing a complete description of the operating situation for the subsequent decision-making of the TD3 agent.

[0035] See Figure 1 This figure is a schematic diagram of the motor FOC dual closed-loop reference structure in an embodiment of the present invention. Reference speed Follow the arrow to the first comparison node, and compare it with the actual mechanical speed fed back by the induction motor. (k) The difference is calculated, and the "+" and "-" next to the comparison node represent positive input and negative feedback input, respectively. The output is the speed error; the speed PI generates the q-axis base current reference based on this error. . q-axis current fed back by Park transformation The difference is calculated at the second comparison node, and the difference is fed into the q-axis PI controller to form the q-axis voltage command. Rotor flux given The system then enters the flux linkage control module, which generates the d-axis current command based on the target flux linkage and motor parameters. ; With feedback d-axis electric After subtraction, the signal is fed into the d-axis PI controller to generate the d-axis voltage command. Therefore, the q-axis channel adjusts the electromagnetic torque, the d-axis channel maintains the rotor flux linkage, and the two current inner loops together form a rapid inner loop. and Enter the "Decoupling / Cross-Coupling Compensation (Feedforward)" module. This module compensates for the cross-coupling terms in the voltage equations in the synchronously rotating dq coordinate system and outputs the decoupled terms. and The inverse Park transform utilizes electrical angles. Will , Transformed into a two-phase stationary coordinate system voltage , SVPWM calculates the three-phase bridge arm switching signals based on this. , , The three-phase voltage source inverter supplies power to the induction motor according to the aforementioned switching signal. The motor's three-phase stator current... , , Following the feedback arrow, we enter the Clarke transformation, transforming from the abc coordinate system to... , Then, it is transformed and utilized by Park. Transform into , These are fed back to the d-axis and q-axis current comparison nodes, respectively. The rotor flux linkage observer receives... , Actual speed and Estimate slip angular velocity using equal quantities Actual mechanical speed Extreme logarithm Converted to rotor electric angular velocity ,and The summation yields the synchronous electric angular velocity. The electric angle is obtained after integration. . Simultaneously, the data is sent to the Park transform, inverse Park transform, and flux observer. The solid lines with arrows in the figure represent the transmission direction of control quantities, feedback quantities, or coordinate transformation quantities, thus forming a closed loop of velocity, current, flux, and electrical angle.

[0036] By adopting the above technical solution, and by establishing an accurate electromechanical coupling dynamic model and a complete state sampling set, the TD3 agent can obtain information that fully characterizes the operating state of the velocity loop, providing a data foundation for the agent to learn effective auxiliary compensation strategies. Normalization processing ensures that the contributions of each state variable in the neural network are balanced, thereby improving training convergence.

[0037] In some preferred embodiments of this application, in step 2, constructing the TD3-assisted PI self-tuning speed outer loop specifically includes: the TD3 agent takes the state quantity as input, uses the PI regulator output as the q-axis base current, and uses the TD3 agent output as the q-axis auxiliary current, and the two are superimposed to obtain the q-axis current reference value of the joint speed outer loop; the proportional gain correction term and integral gain correction term generated by the TD3 agent are respectively superimposed on the PI reference gain, so that the speed loop parameters can be adjusted online according to the operating state; the TD3-assisted PI self-tuning speed outer loop is constructed by the above superposition relationship between the PI base channel and the TD3 auxiliary channel.

[0038] In the specific implementation of step 2, the seven-dimensional state vector obtained by normalization in the previous steps is simultaneously fed into two channels: one is a speed loop PI regulator, which adopts an incremental PI algorithm, that is, the difference between the current cycle speed error and the previous cycle speed error is used as the proportional term input, and the current cycle speed error is used as the integral accumulation term input. The proportional gain and integral gain are respectively taken as the current online values. After incremental calculation, the output is the q-axis base current setpoint, which represents the deterministic torque current command under traditional PI control; the other is the Actor policy network of the TD3 agent, which is a three-layer fully connected feedforward neural network with seven input layer nodes, consistent with the dimension of the state vector, two hidden layers with three hundred neurons each, and the activation function is the modified linear unit. The output layer has three nodes, corresponding to the q-axis auxiliary current increment, the proportional gain correction increment, and the integral gain correction increment, respectively. Each output node uses the hyperbolic tangent activation function to compress the original output to [ The values ​​are calculated in the range [-1, 1], and then multiplied by their respective maximum action step sizes. For example, the maximum step size for the auxiliary current increment is set to 5% of the rated current, the maximum step size for the proportional gain correction increment is set to 10% of the base proportional gain, and the maximum step size for the integral gain correction increment is set to 10% of the base integral gain to obtain the actual action increment value. The weight parameters of the Actor policy network are optimized and determined by the policy gradient update method in the TD3 algorithm during the training phase, and remain fixed during the deployment phase, only performing forward calculations. After obtaining the three action increments, the auxiliary current value, proportional gain correction value, and integral gain correction value saved in the previous cycle are accumulated, and then constrained by the saturation limiting function. For example, the auxiliary current is limited to the range [-0.2 times rated current, 0.2 times rated current], the proportional gain correction term is limited to the range [-0.5 times base proportional gain, 0.5 times base proportional gain], and the integral gain correction term is limited to the range [-0.5 times base integral gain, 0.5 times base integral gain].Within the range of [5 times the reference integral gain], the auxiliary compensation and parameter correction are ensured to remain within the pre-set safety boundaries. Then, the limited auxiliary current value is algebraically superimposed with the base current setpoint output by the PI regulator to obtain the final q-axis current reference value of the joint speed outer loop. Simultaneously, the limited proportional gain correction term is added to the pre-set speed loop PI reference proportional gain to obtain the current online proportional gain, and the limited integral gain correction term is added to the reference integral gain to obtain the current online integral gain. These two online gains are fed back to the gain coefficient of the incremental PI algorithm for PI calculation in the next control cycle. Thus, the PI base channel provides a stable and interpretable deterministic control reference, while the TD3 auxiliary channel outputs incremental compensation and parameter fine-tuning based on the current operating conditions. Working in parallel, they form a complete TD3-assisted PI self-tuning speed outer loop. At the end of each control cycle, this TD3-assisted PI self-tuning speed outer loop outputs the final q-axis current reference value to the current inner loop as a setpoint, while simultaneously storing the updated online proportional and integral gains in memory for use in the next cycle.

[0039] See Figure 2 The figure shows a schematic diagram of the TD3 agent learning structure in an embodiment of the present invention. The TD3 algorithm network structure mainly consists of an Actor policy network, a dual Critic evaluation network, a target Actor network, a dual target Critic network, and an experience replay buffer. Compared with the traditional DDPG algorithm, TD3 effectively reduces the overestimation problem of the value function during continuous action control by introducing a dual Q-network, target policy smoothing, and delayed policy update mechanisms, thereby improving the stability of algorithm training. During training, the agent first obtains the current state of the environment. And input it into the Actor network. Generate control actions through deterministic strategies Its expression is ,in, Let represent the Actor network parameters, and N represent the random noise used to enhance exploration capabilities. Then, actions... When applied to the environment, the environment returns the corresponding reward. and the state at the next moment Thus forming interactive samples ( , , , , The samples are randomly sampled from the experience replay pool and stored at time t. During network training, the agent randomly samples a small batch of samples from the experience replay pool and uses these samples to update the network parameters to reduce the correlation between samples and improve the convergence performance of the training process. The actions output by the Actor network are simultaneously input into two Critic evaluation networks. and The value function of the current state-action combination is estimated separately, and the results are obtained. and To avoid overestimating the Q-value due to a single evaluation network, the TD3 algorithm employs a dual-Critic structure and calculates the target Q-value using the smaller of the output values ​​from the two target Critic networks. The calculation process is as follows: ,in, γ represents the target Q value, γ represents the discount factor, and done represents the state termination flag. and These represent the outputs of the target Critic network. To further improve training stability, TD3 introduces a target network with the same structure as the online network, including the target Actor network. and the target Critic network , The target Actor network determines the state based on the next time step. Generate target action ,Right now = ,in, This indicates that target action noise is truncated and restricted, and the estimation bias caused by action perturbations during policy updates is reduced through a target policy smoothing mechanism. Subsequently, the target action... Input the target Critic network and obtain the following results: and The smaller of the two values ​​is selected for the target value calculation. The online Critic network minimizes the mean squared error loss function. Continuously optimize parameters and This allows the network output value to gradually approach the true cumulative reward. During the Actor network update process, TD3 employs a delayed update mechanism, optimizing the Actor network only after the Critic network has undergone multiple updates. The policy gradient update process is represented as follows: This method can reduce the impact of Critic estimation errors on the Actor network. Simultaneously, the target network uses a soft update method to progressively track online network parameters; the update process is as follows: , in, These are the soft update coefficients. Through the aforementioned dual-Q network, target policy smoothing, and delayed update mechanism, the TD3 algorithm can improve the stability of the training process while ensuring control accuracy.

[0040] It should be noted that the Chinese in this application Both refer to the discrete sampling period number, which is an integer index used to represent the discrete time step (i.e., the sampling time), and is used to distinguish the variable state within different control periods. The proportions are given and will not be elaborated further.

[0041] See Figure 3 The figure is a schematic diagram of the TD3-assisted PI speed outer loop self-tuning architecture in an embodiment of the present invention. Figure 3 Embed the TD3 auxiliary channel into the complete FOC speed outer loop. and Difference obtained Traditional speed PI is based on Generate q-axis base current setpoint The dashed gain feedback box provides... , ,in , As the reference gain, , This is the correction value obtained through TD3 recursion. TD3 agent reception. , , , , , , Seven inputs, Actor mapping in the diagram Output 3D motion =[ The "Motion Limit / Incremental Update" module compares the three increments with the previous cycle. , , By accumulating and saturating the constraints, the auxiliary current and gain correction for this cycle are obtained. and The final q-axis current reference value is formed by superposition at the addition nodes. , , This feedback is then sent to the traditional speed PI update online gain. Figure 3 The experience replay pool and reward function in the middle The dashed line connecting the Actor and the dual Critic represents sample storage, value evaluation, and policy updates performed only during the training phase; during online deployment, the Actor and action processing modules are retained, while the dual Critic, reward function, and experience replay pool do not participate in real-time computation. Feedback After the difference is calculated, the input enters the q-axis PI, with the flux linkage set. Obtained through flux control or excitation current calculation , Feedback The difference is then fed into the d-axis PI converter; the outputs of both axes PI converters are then processed sequentially through decoupling / cross-coupling compensation, inverse Park transform, SVPWM, and a three-phase voltage source inverter to act on the induction motor. Motor current... , , Obtained through Clarke transform and Park transform , , , This data is fed back along the solid line to the inner current loop and the TD3 state sampling terminal. The rotor flux linkage observer is based on... , , and calculate , and Adding them together gives After integration, we obtain And feed it back to the coordinate transformation. Figure 3 The boundary box of the controller contains the speed outer loop and the TD3 auxiliary module, while the boundary box contains the flux command, current inner loop, power converter, motor, and feedback observation link. Solid arrows indicate the online control signal flow, and dashed arrows indicate the training or parameter update relationship.

[0042] By adopting the above technical solution, the organic integration of PI deterministic control and TD3 intelligent compensation is achieved through the superposition of base current and auxiliary current, as well as the superposition of reference gain and correction amount. The auxiliary current channel enhances the dynamic response speed, the gain correction channel realizes parameter self-adaptation, and the two channels work together to improve the overall performance of the speed loop.

[0043] In some preferred embodiments of this application, the action of the TD3 agent is set as a three-dimensional continuous increment, corresponding to the q-axis auxiliary current increment, the proportional gain correction increment, and the integral gain correction increment, respectively. The impact of sudden changes in absolute control quantity on the closed loop is reduced by recursive updates.

[0044] In the incremental action implementation of step 2, the Actor policy network output layer of the TD3 agent does not directly generate the absolute values ​​of the auxiliary current, proportional gain correction, and integral gain correction. Instead, it generates three dimensionless increment factors, corresponding to the increments of the auxiliary current, proportional gain correction, and integral gain correction, respectively. Each increment factor is compressed to the open interval (-1, 1) by the hyperbolic tangent activation function and then multiplied by its preset maximum increment step size. For example, the increment step size of the auxiliary current is 2% of the peak value of the rated current. That is, if the peak value of the rated current is 100 amperes, the maximum increment of the auxiliary current in a single cycle is 2 amperes; proportional gain correction... The increment step size is 5% of the reference proportional gain. If the reference proportional gain is 0.5 Nm / radian / second, the maximum increment per cycle is 0.025 Nm / radian / second. The integral gain correction increment step size is 5% of the reference integral gain. If the reference integral gain is 10 Nm / radian, the maximum increment per cycle is 0.5 Nm / radian, thus obtaining the actual physical dimension of the action increment value. Subsequently, the auxiliary current increment obtained in the current cycle is algebraically summed with the auxiliary current value saved in the previous cycle, the proportional gain correction increment is summed with the proportional gain correction value saved in the previous cycle, and the integral gain correction increment is summed with the integral gain saved in the previous cycle. The correction values ​​are summed, and the three summations are processed by saturation limiting functions. For example, the auxiliary current is limited to the range of [-10 Amperes, 10 Amperes], the proportional gain correction term is limited to the range of [-0.2 times the reference proportional gain, 0.2 times the reference proportional gain], and the integral gain correction term is limited to the range of [-0.2 times the reference integral gain, 0.2 times the reference integral gain]. The three values ​​after limiting are the auxiliary current, proportional gain correction term, and integral gain correction term for the current cycle. The essence of this incremental recursive update method is that in each discrete control cycle, only a finite-small adjustment is applied to the value of the previous moment. Due to the incremental step... The length is limited to within 2% to 5% of the rated value. Even if the neural network causes output jumps due to input noise, the actual control quantity will only change slowly rather than abruptly, thus effectively avoiding large numerical oscillations that may occur when the absolute control quantity is directly output. The recursively updated auxiliary current value is superimposed on the base current setpoint output by the PI regulator. The recursively updated proportional gain correction term and integral gain correction term are superimposed on the reference gain, respectively. The three results are used together for the final control output of the current control cycle and are simultaneously saved in memory as the historical reference value for the next recursively updated cycle, forming a complete closed-loop recursively updated link.

[0045] By adopting the above technical solution, incremental recursive updates significantly reduce the instantaneous impact of neural network output uncertainty on the motor closed loop, and improve the smoothness and safety of the control system during the training and online operation phases.

[0046] In some preferred embodiments of this application, saturation limits are set for the q-axis auxiliary current, the proportional gain correction term, and the integral gain correction term, respectively, so that the control compensation amount and online parameters are always within a preset safety range. The PI base channel and the TD3 auxiliary channel form a joint control relationship through algebraic superposition, and the joint control relationship is expressed as follows:

[0047]

[0048]

[0049] in, Indicates the first The q-axis current reference value output by the controller in each sampling period; Indicates the first The q-axis current output by the PI base channel in each sampling period; Indicates the first TD3 auxiliary current per sampling period; and They represent the first Online proportional gain and online integral gain for each sampling period, The gain symbol is used. As a proportion, For integration; and These represent the reference gains of the velocity loop proportional gain P and integral gain I, respectively. and They represent the first The proportional gain correction term and integral gain correction term are given by TD3 for each sampling period. This represents the discrete sampling period number.

[0050] In the specific implementation process, the saturation limiting method for the q-axis auxiliary current, proportional gain correction term, and integral gain correction term is as follows: At the beginning of each control cycle (i.e., the execution cycle in which the digital controller triggers an interrupt service at a fixed time interval, set to 100 microseconds in this scheme), the auxiliary current value, proportional gain correction value, and integral gain correction value of the previous moment are first read from the register saved in the previous cycle. Then, these values ​​are multiplied by their respective step sizes by the three incremental factors output by the Actor network in the current cycle to obtain the incremental values, which are then accumulated. After accumulation, the values ​​are divided into... Do not call the three-stage limiting function to constrain the three accumulated results. The implementation logic of this limiting function is as follows: if the input value is less than the preset lower limit, output the lower limit value; if the input value is greater than the preset upper limit, output the upper limit value; otherwise, output the input value itself. The lower and upper limits of the auxiliary current are set according to the motor's rated current. For example, when the peak rated current is 100 amps, the lower limit is -20 amps and the upper limit is 20 amps, ensuring that the auxiliary compensation does not exceed 20% of the rated current. The lower and upper limits of the proportional gain correction term are set according to the speed loop stability margin, such as the reference proportional gain. When the gain is 0.5 Nm / radian per second, the lower limit is set to -0.25 Nm / radian per second and the upper limit to 0.25 Nm / radian per second, ensuring that the online proportional gain is always between 0.25 and 0.75 Nm / radian per second, preventing slow response due to excessively low gain or oscillation caused by excessively high gain; the lower and upper limits of the integral gain correction term are set according to the integral saturation suppression requirements, such as the baseline integral gain. For a gain of 10 Nm / radian, the lower limit is set to -5 Nm / radian and the upper limit to 5 Nm / radian to ensure that the online integral gain is always between 5 and 15 Nm / radian, avoiding steady-state error due to insufficient integral action or integral saturation due to excessive integral action. After limiting, the limited auxiliary current is algebraically summed with the q-axis base current setpoint output from the PI base channel to obtain the final q-axis current reference value for the joint velocity outer loop. This value is then written into the current inner loop's setpoint register for use by the d-axis and q-axis current regulators; simultaneously, the limited proportional gain correction term is... proportional gain with speed loop PI reference (This reference value is a fixed constant pre-tuned during the controller design phase based on the stability boundary and response requirements under typical operating conditions, for example, 0.5 Nm per radian per second.) Adding these values ​​together yields the current online proportional gain. The integral gain correction term after limiting Integral gain with reference (Similarly, these are pre-tuned fixed constants, such as 10 Nm per radian) added together to obtain the current online integral gain. These two online gains are immediately written into the gain register of the speed loop PI regulator, replacing the gain value of the previous cycle, and used for error integral accumulation and proportional control calculation in the current and subsequent cycles; the entire limiting and joint control process is executed sequentially in the interrupt service routine through the above three algebraic relationships. After execution, all updated values ​​of the current cycle (including...) , , All of these are stored in global variables, which are used both for the current inner loop given output in the current cycle and as the historical reference for the next cycle's recursive update, thus ensuring both the physical safety boundary of the control quantity and the continuous smoothness of parameter adjustment.

[0051] Using the above technical solution, saturation limiting constitutes the last barrier to system safety. Even when the TD3 strategy is not sufficiently trained or encounters unseen extreme conditions, the auxiliary compensation and parameter correction will not exceed the preset safety range, thereby ensuring that the motor will not experience dangerous states such as overcurrent, stall, or closed-loop oscillation.

[0052] In some preferred embodiments of this application, step 3, which utilizes staged rewards to complete policy training and embed the velocity outer loop, specifically includes: Step 3.1: Establish the state vector of the TD3 agent. The state vector is used to characterize the speed tracking error, error change trend, motor operating state, q-axis current response, auxiliary quantity at the previous moment, and current PI gain. The state vector, error, and error difference are represented as follows:

[0053]

[0054]

[0055] in, For the first The state vector of the TD3 agent in each sampling period; For the first Speed ​​error per sampling period; For the first Error difference for each sampling period; For the first Reference rotational speed for each sampling period; For the first The actual mechanical speed of the motor in each sampling period; For the first q-axis current feedback per sampling period; For the first The TD3 auxiliary current is obtained by recursion over each sampling period; and The first Online proportional gain and integral gain for each sampling period; Step 3.2: Set the three-dimensional continuous motion vector and use the saturation limiting function to recursively update the auxiliary current and PI gain correction term; Step 3.3: Construct a normalized operating phase according to the degree to which the actual rotational speed approaches the reference rotational speed. This operating phase then generates a rapid response weight for the initial stage and a convergence recovery weight for the later stage. This allows the reward objective to gradually transition from rapid error reduction in the startup phase to compensation recovery, overshoot suppression, and oscillation reduction in the steady-state phase. This leads to the establishment of a composite reward function that simultaneously reflects speed, overshoot, steady-state error, and disturbance recovery. The normalized operating phase, rapid response weight for the initial stage, and convergence recovery weight for the later stage are expressed as follows:

[0056]

[0057]

[0058] in, For the first Normalized running phase for each sampling period; For the first Each sampling period has a fast response weight in the early stage; For the first The weights are recovered during the later stages of each sampling period; For the first Actual mechanical rotation speed per sampling period For the first The reference speed is measured in sampling periods; sat(.) is the saturation limiting function; when the motor is in the initial starting stage... The speed is relatively large, and when the motor gradually approaches the reference speed... Gradually increase This refers to the discrete sampling period number; Step 3.4: In the joint simulation, the TD3 agent is trained using a dual-Critic network, delayed policy update, target policy smoothing, experience replay, and target network soft update mechanism. When the rotation speed or current exceeds the limit during training, the current cycle is terminated and a penalty is applied. After training is completed, the training of the TD3 policy network is finished.

[0059] In step 3.2, the action vector and update relationship are expressed as follows:

[0060]

[0061]

[0062]

[0063] in, For the TD3 agent in the first The action vector output in each sampling period; , and These are the action increments for the q-axis auxiliary current, the proportional gain correction term, and the integral gain correction term, respectively. , , , , and These are the lower and upper limits of the amplitude limiting for the variables corresponding to the q-axis auxiliary current, proportional gain correction term, and integral gain correction term, respectively; sat(.) is the saturation limiting function.

[0064] The composite reward function is expressed as follows:

[0065]

[0066]

[0067] in, For the first Total reward for each sampling period; Basic performance bonus; Rewards for rapid response in the early stages; This is for subsequent compensation and recycling rewards; For steady-state reward; to This is the reward weighting coefficient; This is an overshoot penalty item; ReLU(.) is a reward for error improvement; ReLU(.) is a linear rectified function.

[0068] In the specific implementation of step 3, firstly, within each sampling cycle of the co-simulation platform (i.e., the interactive co-simulation environment of MATLAB / Simulink and Python, where Simulink is responsible for the real-time numerical solution of the motor controlled object and the FOC dual closed-loop control structure, and Python is responsible for the forward calculation and network parameter storage of the TD3 agent), the reference speed command value, the actual mechanical speed fed back by the photoelectric encoder, the current cycle error value in the speed loop error register, the historical error value saved in the previous cycle, the q-axis current feedback value after analog-to-digital conversion of the current sensor, the auxiliary current value saved after the recursive update in the previous cycle, and the online proportional gain and online integral gain of the current speed loop PI regulator are read from the Simulink workspace according to the formula. Calculate the speed error according to the formula. Calculate the error difference and combine the above seven variables into a seven-dimensional state vector. Before being fed into the Actor policy network, each dimension is standardized by subtracting the mean obtained from the training buffer and then dividing by the standard deviation, so that the numerical range of each dimension is roughly on the same order of magnitude to facilitate the numerical stability of the network's forward computation. In addition, when the absolute value of the reference rotation speed is less than the preset threshold, the normalized running phase is set to 1.

[0069] Then, the standardized state vector is used as input to the Actor policy network for a forward propagation calculation. This Actor policy network is a fully connected network with a seven-node input layer, two 300-node hidden layers, and a three-node output layer. The hidden layer activation function uses a modified linear unit, and the output layer activation function uses a hyperbolic tangent function to compress the original output to the (-1,1) interval. After the forward calculation, three dimensionless increment factors are obtained, which are multiplied by a preset maximum increment step size. For example, the auxiliary current increment step size is 2% of the rated current, the proportional gain correction increment step size is 5% of the baseline proportional gain, and the integral gain correction increment step size is 5% of the baseline integral gain, thus obtaining the actual physical unit action increment value. , , Then, it is added to the auxiliary current value, proportional gain correction value, and integral gain correction value stored in the previous cycle, and after being constrained by the saturation limiting function, the current cycle's value is obtained. , , Then, the limited auxiliary current is summed with the PI base channel output to obtain the final q-axis current reference value, and the limited correction term is summed with the reference gain to obtain the online proportional gain and integral gain; at the same time, at the end of each cycle of the training phase, the ratio of the current actual mechanical speed to the reference speed is calculated according to the formula To calculate the normalized operating phase and prevent the denominator from becoming invalid when the reference speed is zero, the training program pre-sets a condition where the absolute value of the reference speed is less than 1 radian per second. Set to 1, then follow and Calculate the front-end fast response weight and the back-end convergence and recovery weight separately. For example, when the motor starts from a standstill, the actual speed is 200 radians per second, and the reference speed is 1000 radians per second. =0.2, =0.8, =0.2, at this point, the weight of the speed objective in the reward function dominates to encourage rapid torque build-up. When the motor accelerates to 950 radians per second, =0.95, =0.05, =0.95, at this point, the weights of compensation recovery and overshoot suppression objectives dominate in the reward function to guide the strategy to reduce auxiliary intervention and suppress overshoot; substituting the above weights into the composite reward function The instant reward value for the current period is calculated in the middle, where The basic performance bonus includes a weighted combination of the absolute value of speed error, the absolute value of error difference, the absolute value of q-axis current, the L2 norm of the motion vector, the overshoot penalty, and the error improvement bonus. This is a reward for rapid response in the initial stage, designed to encourage a rapid reduction in the absolute value of the error during startup. This is a post-compensation recovery reward, used to encourage the gradual reduction of the amplitude of the auxiliary current and correction term to reduce additional intervention at steady state. This is a steady-state reward term, used to provide additional positive incentives to reduce steady-state error after entering a steady state. After calculation, the current state, action, reward, next state, and termination flag are combined into a quintuple and stored in the experience replay buffer. This buffer adopts a circular queue structure with a capacity of one million records. When the number of records stored exceeds the capacity, the earliest record is automatically overwritten.

[0070] Subsequently, a batch of 256 quintuples was randomly sampled from the experience replay buffer for updating the network parameters of the TD3 algorithm. The update process employed a dual-Critic network structure, using the smaller of the two value network outputs as the target Q-value to suppress overestimation. A delayed update of the Actor policy network was performed only after every two Critic updates to reduce update variance. The target network used a soft update method, meaning that each update brought the target network parameters closer to the current network parameters by a small sub-quantum (e.g., a soft update coefficient τ=0.005). Furthermore, clipped Gaussian noise was added to the next action when calculating the target Q-value to achieve smooth regularization of the target policy. During training, the results of each complete round (i.e., the entire simulation process from motor start-up to reaching steady state or triggering an out-of-bounds condition) were recorded. The total reward and various performance indicators are calculated, and it is also checked whether the speed exceeds 120% of the rated speed or the current exceeds 150% of the peak current in the current round. If so, the current round is terminated immediately, and an additional large negative penalty value (e.g., -100) is applied to the reward of the last step of the round. At the same time, the remaining unexecuted simulation steps of the round are cleared, and the initial speed and load torque of the motor are randomly re-initialized to start the next round of training. The training continues until the average reward of the round converges and various performance indicators reach the preset thresholds, such as 95% rise time less than 0.2 seconds, overshoot less than 2%, and steady-state error less than 5 radians per second. At this time, all weight parameters of the Actor policy network are saved, and the network is exported in ONNX format and embedded into the MATLAB Function module in Simulink. When this module is called in each sampling period, it receives the state vector input and outputs three action increments, thereby deploying the trained policy to the TD3 assisted PI self-tuning speed outer loop for actual operation.

[0071] See Figure 4 This figure is a schematic diagram of the TD3-assisted PI training process in an embodiment of the present invention. Figure 4 The complete learning process of the TD3 agent is presented in chronological order: After starting, the Actor, two Critics, and corresponding target network parameters are initialized, and the experience replay pool is initialized; the simulation environment is created or reset, the initial state of the motor, reference speed, load torque, and parameter perturbation are set, and the current state is obtained. The Actor output diagram shows... =[ The action is applied to the FOC environment after recursion and limiting; environmental feedback. Instant rewards and the termination flag done, and will ( , , , , Write it into the experience replay pool. This is the sampling time; after the sample size reaches the batch condition, random sampling is performed and the two Critics are updated. When the policy update interval is reached, the Actor is updated along the "Yes" branch and the target network is softly updated; otherwise, the round ends directly along the "No" branch. If the current round has not ended, ... Updated to The process returns to the Actor to continue the next interaction. If the round ends, it checks whether the average reward, rise time, overshoot, and steady-state error meet the convergence criteria or whether the maximum number of training rounds has been reached. If not, the environment is reset along the "No" branch to start the next round; if so, the converged Actor policy is saved and output along the "Yes" branch, and training ends. If the rotational speed or current exceeds the safety threshold, [the process will...]. Set to 1 to terminate the loop early and apply an out-of-bounds penalty. In the diagram, rectangles represent processing steps, diamonds represent conditional judgments, ellipses or rounded terminal boxes represent start / end, and arrows indicate the execution order and the loop direction of the "yes / no" branch.

[0072] By adopting the above technical solution, firstly, the combination of recursive updates and saturation limiting ensures that the auxiliary compensation quantity grows smoothly from zero and remains bounded, achieving a safe transition during the startup phase and stable compensation during the operation phase. Secondly, the multi-dimensional state vector enables the TD3 agent to comprehensively perceive the operating state of the outer speed loop, thereby outputting auxiliary compensation and parameter corrections that match the current operating conditions, improving the adaptability and control accuracy of the strategy. Furthermore, the phased reward driven by the operating phase enables a single TD3 strategy to automatically identify the operating phase and adjust the optimization objective, pursuing speed during startup and stability during convergence, solving the problem of multi-objective reward conflict and eliminating the need to train multiple strategies for different phases. Furthermore, the calculation of normalized operating phase and weights is simple and efficient, requiring only one division and two saturation operations to achieve phase identification, making it suitable for embedded real-time deployment. Finally, the multi-component composite reward comprehensively covers all performance indicators of speed loop control, the phased weights enable automatic switching of objectives for different operating phases, and the compensation recovery mechanism effectively prevents steady-state residue of auxiliary quantities, achieving an optimal balance between speed, stability, steady-state accuracy, and disturbance rejection in the trained strategy. Finally, the dual Critic and delayed update mechanism of the TD3 algorithm ensures training stability, the safety constraint termination mechanism embeds aviation safety requirements into the training process, the co-simulation platform provides a high-fidelity training environment, and the trained strategy has good adaptability to multiple operating conditions.

[0073] Specifically, after training the TD3 policy network, the trained Actor policy network is deployed to the TD3 agent in the TD3-assisted PI self-tuning speed outer loop. The Actor policy network outputs an action vector based on the input state vector, expressed as:

[0074] in, (.) represents the Actor policy network after training. Its network parameters, The input state vector, To output the action vector; through the above control strategy, the PI regulator provides an interpretable and stable basic speed closed loop. The TD3 agent only learns the incremental correction of the auxiliary current and PI gain, thereby reducing the continuous action search space and improving the speed, smoothness, tracking accuracy and load disturbance resistance of the speed outer loop of the airborne induction motor of the wide-body passenger aircraft.

[0075] Using the above technical solution, the trained Actor network has low computational load and strong determinism, making it suitable for embedded deployment in aviation motor controllers; the PI regulator provides an interpretable and stable basic speed closed loop, and the TD3 agent only learns the incremental correction of the auxiliary current and PI gain, which reduces the search space for continuous actions and improves the system's speed, stability, tracking accuracy and resistance to load disturbances.

[0076] In some preferred embodiments of this application, the aircraft motor is an airborne induction motor. The rotor flux orientation is achieved by obtaining the rotor flux position through a rotor flux observer. In the FOC dual closed-loop control structure, the d-axis current is used to maintain the rotor flux, and the q-axis current is used to control the electromagnetic torque. Rotor flux orientation ensures the decoupling accuracy of the induction motor FOC control. The independent control of the d-axis excitation and q-axis torque provides a linearized control object for the speed outer loop, enabling the TD3-assisted PI strategy to effectively optimize the speed loop performance on a decoupled basis.

[0077] Example 1: A simulation implementation method 1. Implementation Object and Platform: Taking aerospace motors as the object, a joint simulation platform of MATLAB / Simulink and Python was established. The dynamic model of the induction motor, the FOC dual closed-loop control structure, and the inverter system were solved using Simulink, while the TD3 agent, experience playback, and network parameter updates were implemented using Python. During the training phase, the reference speed, load disturbances, and some operating conditions were randomized to enhance the policy's adaptability to multiple operating conditions.

[0078] 2. Constructing a TD3-assisted PI self-tuning controller: The PI regulator provides the basic speed closed loop; the TD3 agent outputs the q-axis auxiliary current increment, proportional gain correction increment, and integral gain correction increment based on the speed error, error differential, feedback speed, q-axis current, and current PI gain; subsequently, a staged composite reward function constraint strategy is used to enhance torque build-up during startup and reduce additional compensation and parameter gain as the speed approaches the reference speed. The self-tuning architecture is described in [reference needed]. Figure 3The training process can be found here. Figure 4 For performance indicator verification, see Figures 6 to 8 .

[0079] 3. No-load start-up comparison experiment. A no-load start-up condition was set, and the traditional PI controller, the fuzzy-PI controller, and the method of this invention were compared. Simulation results show that the method of this invention enables faster motor speed establishment, smaller peak overshoot, and a shorter time to enter the stable error band; its 95% rise time is 0.1584s, which is 12.73% shorter than that of the traditional PI controller; the overshoot is reduced from 1.562% of the traditional PI controller to 0.370%; the 2% settling time is reduced from 0.1986s to 0.1634s; and the error integral is reduced from 166.87 r / min·s to 146.27 r / min·s. Corresponding performance indicators are shown in [link to performance indicators]. Figure 6 .

[0080] 4. Speed ​​Step Comparison Experiment. A speed step operation condition ranging from 400 r / min to 1400 r / min was set. The results show that the method of the present invention maintains good variable speed tracking capability in both low-speed start-up and high-speed step phases. Taking the 0.5s step time as the timing starting point for the 1400 r / min phase, the 95% rise time of the method of the present invention is 0.0442s, which is 18.30% shorter than that of the traditional PI; the overshoot is reduced from 1.677% to 0.319%; the 2% settling time is reduced from 0.0553s to 0.0467s; and the error integral is reduced by 16.00% compared to the traditional PI. Corresponding performance indicators are shown below. Figure 7 .

[0081] 5. Comparative Experiment with Sudden Load Increase. A disturbance condition was set where the load was suddenly increased from 1 N·m to 40 N·m. The results show that the method of this invention can reduce the speed drop during the instantaneous load change and shorten the recovery process after the disturbance. The 0.5% recovery time after the load increase was shortened from 9.89 ms for the traditional PI to 2.40 ms, and the maximum speed drop was reduced from 19.97 r / min to 7.40 r / min, indicating that the control method has strong load disturbance suppression capability and dynamic robustness. Corresponding performance indicators are shown below. Figure 8 .

[0082] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, the processor being configured to acquire the code and execute the above-described TD3-assisted PI self-tuning vector control method for the outer loop of aircraft motor speed.

[0083] In some embodiments, reference Figure 5This figure is a schematic diagram of the structure of a computer device for implementing a TD3-assisted PI self-tuning vector control method for the outer loop of an aircraft motor speed, according to some embodiments of this application. The TD3-assisted PI self-tuning vector control method for the outer loop of the aircraft motor speed in the above embodiments can be implemented through... Figure 5 The computer device shown is used to implement this, and the computer device includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.

[0084] The processor 301 can be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more execution methods of TD3-assisted PI self-tuning vector control for controlling the outer loop of the speed of the aircraft motor in this application.

[0085] The communication bus 302 can be used to transmit information between the aforementioned components.

[0086] The memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 303 may exist independently and be connected to the processor 301 via the communication bus 302. The memory 303 may also be integrated with the processor 301.

[0087] The memory 303 stores program code for executing the scheme of this application, and its execution is controlled by the processor 301. The processor 301 executes the program code stored in the memory 303. The program code may include one or more software modules. In the above embodiment, the determination of the TD3-assisted PI self-tuning vector control method for the outer loop of the aircraft motor speed can be achieved by the processor 301 and one or more software modules in the program code in the memory 303.

[0088] Communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0089] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0090] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0091] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned TD3-assisted PI self-tuning vector control method for the outer loop of aircraft motor speed.

[0092] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0093] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor, characterized in that, Includes the following steps: Step 1: Establish the FOC control object model of the aircraft motor and form the speed outer loop state sampling set: Based on the electromechanical coupling relationship of the aircraft motor, establish a dynamic model, construct a rotor flux orientation double closed loop control structure, and obtain the state variables used for TD3 auxiliary regulation. Step 2, construct the TD3-assisted PI self-tuning speed outer loop: using the speed loop PI regulator as the basic control channel, the state quantity is used as the input of the TD3 agent, and the agent is configured to generate the q-axis auxiliary current on the PI output side and simultaneously provide the online correction amount of the PI proportional gain and integral gain; Step 3: Complete policy training and embed it into the velocity outer loop using phased rewards: Establish a composite reward function that simultaneously reflects speed, overshoot, steady-state error and disturbance recovery, train the TD3 policy network through joint simulation, and deploy the trained Actor policy into the TD3-assisted PI self-tuning velocity outer loop.

2. The TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor as described in claim 1, characterized in that, In step 1, establishing the FOC control object model of the aircraft motor and forming the speed outer loop state sampling set specifically includes: Step 1.1: Establish a dynamic model based on the stator and rotor flux linkage equations, voltage balance equations, electromagnetic torque equations, and mechanical motion equations of the induction motor. This model is used to describe the changes in speed, current, and torque of the airborne induction motor during startup, speed change, and load disturbance. Step 1.2: Construct an FOC dual closed-loop control structure according to the rotor flux orientation principle. Transform the three-phase electrical quantities into a synchronous rotating dq coordinate system through Clarke transformation and Park transformation. The q-axis current is given by the speed outer loop, and fast tracking is completed by the d-axis and q-axis current inner loops. In the FOC dual closed-loop control structure, the d-axis current is used to maintain the rotor flux, and the q-axis current is used to control the electromagnetic torque. Step 1.3: Sample the reference speed and feedback speed and calculate the speed error accordingly. Normalize the speed error, error difference, feedback speed, q-axis current, q-axis auxiliary current, and current PI proportional gain and integral gain, and combine them to form the state input of the seven-dimensional TD3 agent. The sampled and normalized physical quantities together constitute the speed outer loop state sampling set, which is the state quantity obtained for TD3 auxiliary regulation.

3. The TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor as described in claim 1, characterized in that, In step 2, constructing the TD3-assisted PI self-tuning speed outer loop specifically includes: The TD3 agent takes the state quantity as input, uses the output of the PI regulator as the q-axis base current, and uses the output of the TD3 agent as the q-axis auxiliary current. The two are superimposed to obtain the q-axis current reference value of the joint velocity outer loop. The proportional gain correction term and integral gain correction term generated by the TD3 agent are superimposed on the PI reference gain, so that the speed loop parameters can be adjusted online according to the operating status. The TD3-assisted PI self-tuning speed outer loop is constructed by the superposition relationship between the PI basic channel and the TD3 auxiliary channel.

4. The TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor as described in claim 1, characterized in that, Step 2 further includes setting the agent's actions as three-dimensional continuous increments, corresponding to the q-axis auxiliary current increment, proportional gain correction increment, and integral gain correction increment, respectively, and reducing the impact of sudden changes in absolute control quantity on the closed loop through recursive updates.

5. The TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor as described in claim 3, characterized in that, Saturation limits are set for the q-axis auxiliary current, the proportional gain correction term, and the integral gain correction term to ensure that the control compensation and online parameters are always within a preset safety range. The PI base channel and the TD3 auxiliary channel form a joint control relationship through algebraic superposition, and the joint control relationship is expressed as follows: in, Indicates the first The q-axis current reference value output by the controller in each sampling period; Indicates the first The q-axis current output by the PI base channel in each sampling period; Indicates the first TD3 auxiliary current per sampling period; and They represent the first Online proportional gain and online integral gain for each sampling period, The gain symbol is used. As a proportion, For integration; and These represent the reference gains of the velocity loop proportional gain P and integral gain I, respectively. and They represent the first The proportional gain correction term and integral gain correction term are given by TD3 for each sampling period. This represents the discrete sampling period number.

6. The TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor as described in claim 1, characterized in that, In step 3, the process of using phased rewards to complete policy training and embedding the velocity outer loop specifically includes: Step 3.1: Establish the state vector of the TD3 agent. The state vector is used to characterize the speed tracking error, error change trend, motor operating state, q-axis current response, auxiliary quantity at the previous moment, and current PI gain. The state vector, error, and error difference are represented as follows: in, For the first The state vector of the TD3 agent in each sampling period; For the first Speed ​​error per sampling period; For the first Error difference for each sampling period; For the first Reference rotational speed for each sampling period; For the first The actual mechanical speed of the motor in each sampling period; For the first q-axis current feedback per sampling period; For the first The TD3 auxiliary current is obtained by recursion over each sampling period; and The first Online proportional gain and integral gain for each sampling period; Step 3.2: Set the three-dimensional continuous motion vector and use the saturation limiting function to recursively update the auxiliary current and PI gain correction term; Step 3.3: Construct a normalized operating phase according to the degree to which the actual rotational speed approaches the reference rotational speed. This operating phase then generates a rapid response weight for the initial stage and a convergence recovery weight for the later stage. This allows the reward objective to gradually transition from rapid error reduction in the startup phase to compensation recovery, overshoot suppression, and oscillation reduction in the steady-state phase. This leads to the establishment of a composite reward function that simultaneously reflects speed, overshoot, steady-state error, and disturbance recovery. The normalized operating phase, rapid response weight for the initial stage, and convergence recovery weight for the later stage are expressed as follows: in, For the first Normalized running phase for each sampling period; For the first Each sampling period has a fast response weight in the early stage; For the first The weights are recovered during the later stages of each sampling period; For the first Actual mechanical rotation speed per sampling period For the first The reference speed is measured in sampling periods; sat(.) is the saturation limiting function; when the motor is in the initial starting stage... The speed is relatively large, and when the motor gradually approaches the reference speed... Gradually increase This refers to the discrete sampling period number; Step 3.4: In the joint simulation, the TD3 agent is trained using a dual-Critic network, delayed policy update, target policy smoothing, experience replay, and target network soft update mechanism. When the rotation speed or current exceeds the limit during training, the current cycle is terminated and a penalty is applied. After training is completed, the training of the TD3 policy network is finished.

7. The TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor as described in claim 1, characterized in that, After training the TD3 policy network, the trained Actor policy network is deployed to the TD3 agent in the TD3-assisted PI self-tuning speed outer loop. The Actor policy network outputs an action vector based on the input state vector, as follows: in, (.) represents the Actor policy network after training. Its network parameters, For the first The input state vector for each sampling period, To output the action vector; through the above control strategy, the PI regulator provides an interpretable and stable basic speed closed loop. The TD3 agent only learns the incremental correction of the auxiliary current and PI gain, thereby reducing the continuous action search space and improving the speed, smoothness, tracking accuracy and load disturbance resistance of the speed outer loop of the airborne induction motor of the wide-body passenger aircraft.

8. The TD3-assisted PI self-tuning vector control method for the outer speed loop of an aircraft motor as described in claim 1, characterized in that, The aircraft motor is an airborne induction motor, and the rotor flux orientation is obtained by a rotor flux observer to determine the rotor flux position.

Citation Information

Patent Citations

  • Low-wind-speed permanent magnet synchronous wind driven generator control method based on deep reinforcement learning

    CN115903457A