Reinforcement learning based dual-loop adaptive control system and method for eVTOL tilt actuators

Abstract

The present application relates to the technical field of aircraft control, and in particular to an eVTOL tilting actuator double-loop adaptive control system and method based on reinforcement learning. The position loop, speed loop and current loop of EMA are regarded as a control inner loop with fixed parameters, and the dynamic performance of the control system is improved by introducing command feedforward control in the position loop and current loop. The outer loop is an online adaptive control based on reinforcement learning agent, which adjusts the feedforward control coefficients of the position loop and speed loop according to the position command and feedback curve to achieve the amplitude and phase alignment of the position loop at the target frequency point, thereby improving the dynamic response and robustness of the control system.

Description

Technical Field

[0001] This invention relates to the field of aircraft control technology, specifically to a dual-loop adaptive control system and method for eVTOL tilt actuators based on reinforcement learning. Background Technology

[0002] Tiltrotor eVTOL aircraft, as a new type of aircraft, combines the characteristics of helicopter vertical takeoff and landing (VTOL) and fixed-wing aircraft high-speed cruise. The flight process of this type of aircraft involves vertical takeoff in helicopter mode, switching to fixed-wing mode to perform the mission, and finally landing in helicopter mode. The entire flight process is divided into six phases: takeoff, hovering at low speed, tilt acceleration transition, fixed-wing cruise, tilt deceleration transition, and landing. In addition to VTOL, takeoff and landing can also be achieved via runway. Because multiple mode switches are required throughout the flight, the operation is extremely complex, making safety paramount. The tilt actuation system is particularly crucial, requiring stable operation under wide load ranges and high-frequency dynamic response requirements. It is the core actuator for switching between VTOL and cruise modes, and its dynamic performance (such as response speed at a specified frequency, tracking accuracy, and interference immunity) directly determines the aircraft's attitude stability, mode switching smoothness, and flight safety.

[0003] The operation of tilt actuators faces multiple complex challenges: first, the load torque changes nonlinearly with the tilt angle during tilting; second, there are random influences from internal and external disturbances such as aerodynamic interference, motor torque fluctuations, and mechanical transmission clearances; and third, the actuator's own dynamic characteristics drift due to factors such as temperature and wear. Existing tilt actuator control systems primarily employ control schemes with significant technical deficiencies. Firstly, single PID control relies on fixed parameter adjustments, resulting in weak suppression of nonlinear disturbances. At designated operating frequencies, amplitude attenuation and phase lag easily occur, leading to sluggish dynamic response. Secondly, while fixed-feedforward PID composite control introduces feedforward compensation, its feedforward coefficients are calibrated based on specific operating conditions and cannot adapt to dynamic characteristic changes under varying conditions. When the load or disturbance changes abruptly, amplitude and phase attenuation further intensify, potentially even triggering system oscillations. Finally, existing solutions generally lack adaptive compensation mechanisms and do not design closed-loop optimization strategies for dynamic performance degradation at specified frequency points. Therefore, parameters need to be manually recalibrated, resulting in poor adaptability and difficulty in meeting the control requirements of tiltrotor eVTOL in all-weather, multi-condition flight.

[0004] Therefore, there is an urgent need to design an adaptive control system that can sense the amplitude / phase attenuation at a specified frequency point in real time and dynamically optimize the feedforward coefficient to solve the core problem of dynamic performance degradation under varying operating conditions in existing technologies. Summary of the Invention

[0005] Purpose of the invention In view of the technical defects of existing tilt actuator control systems, such as significant amplitude attenuation and phase lag at a specified frequency point under varying operating conditions, poor disturbance rejection, and lack of adaptive parameter adjustment mechanism, this invention can solve the core problem of dynamic performance degradation under varying operating conditions in existing technologies.

[0006] Technical solution The core technical solution of this invention is to construct a dual-loop adaptive control system of "inner loop feedforward PID tracking + outer loop reinforcement learning optimization", in which each unit works collaboratively to achieve dynamic performance optimization. The eVTOL tilt actuator dual-loop adaptive control system based on reinforcement learning consists of a flight control computer communication unit, an inner loop real-time control unit, an outer loop intelligent control unit, a signal detection and processing unit, an execution unit, and the logic between them.

[0007] The flight control computer communication unit is used to receive position commands issued by the flight control computer via the aircraft communication bus and return the real-time operating status of the dual-loop adaptive control system to the flight control computer.

[0008] The inner loop control unit is a PID position loop controller based on feedforward compensation. It receives position command signals, generates advance compensation signals through the feedforward control module, and combines them with the error adjustment signals output by the PID controller to achieve real-time position control and steady-state error suppression of the tilt actuator.

[0009] The outer loop control unit is a reinforcement learning-based intelligent controller used to collect the command signals of the inner loop control unit and the position feedback signals of the tilt actuator in real time, extract the amplitude attenuation and phase attenuation at a specified frequency point, and dynamically output the feedforward control coefficient adjustment command through the reinforcement learning algorithm according to the preset ideal phase attenuation and amplitude attenuation range, so as to adaptively correct the control coefficient of the feedforward control module.

[0010] The signal detection and processing unit is used to acquire the position feedback signal of the tilt actuator, filter and frequency analyze the command signal and the feedback signal, calculate the amplitude ratio and phase difference at a specified frequency point, output the amplitude attenuation and phase attenuation, and simultaneously send the position signal and calculation results to the flight control unit.

[0011] The actuator, i.e. the tilt actuator, receives the synthesized drive signal from the inner loop control unit and drives the tilt mechanism of the tilt rotor eVTOL to complete the tilt angle adjustment.

[0012] Furthermore, the signal detection and processing unit uses a linear variable differential transformer to detect the tilt actuator position feedback signal, a rotary transformer to detect the speed feedback signal, a current sensor to collect the three-phase current, and calculates the q-axis and d-axis feedback signals based on Clark transform and Park transform.

[0013] Furthermore, the outer loop control unit is a reinforcement learning-based intelligent controller. Its inputs are the attenuation parameters output by the signal acquisition and analysis module and the operating condition parameters output by the operating condition parameter acquisition module. It dynamically generates a feedforward control coefficient adjustment command K_spd through a preset reinforcement learning algorithm to compensate for the amplitude and phase attenuation at a specified frequency point. It consists of three parts: a signal analysis module, a reinforcement learning intelligent module, and an output limiting module.

[0014] Furthermore, the signal analysis module receives the position command signal θ_ref and the position feedback signal θ_fdb, and uses a moving average filter to filter θ_ref and θ_fdb to suppress measurement noise and electromagnetic interference. Then, it uses a fast Fourier transform to extract the command amplitude A_θ_ref, feedback amplitude A_θ_fdb, command phase φ_θ_ref, and feedback phase φ_θ_fdb at a specified frequency point. Finally, it calculates the amplitude attenuation ΔA = 100% * (A_θ_ref - A_θ_fdb) / A_θ_ref and the phase attenuation Δφ = φ_θ_ref - φ_θ_fdb.

[0015] Furthermore, the reinforcement learning intelligent module comprises a state encoding submodule, a policy network submodule, a reward calculation submodule, a policy update submodule, and a threshold judgment submodule. The state encoding submodule converts continuous physical quantities into state vectors that can be processed by the reinforcement learning algorithm, and its state space S is defined as S=[ΔA,Δφ, K_spd]. The action space A of the policy network submodule is the adjustment step size of the feedforward coefficients, which directly determines the coefficient update amplitude, and is defined as A=[ΔK_spd], where ΔK_spd is the adjustment step size of the position feedforward coefficients. The action output must satisfy K_spd∈[K_spd_min, K_spd_max], where K_spd_min, K_spd_max are limit thresholds calibrated based on EMA parameters, typically taken as [-0.02, 0.02].

[0016] Furthermore, the reward value R of the reward calculation submodule is constructed using a multi-objective optimization function, expressed as: R = R_base - ω1 ΔA_err- ω2 Δφ_err - ω3 σ% - ω4 t_r, where R_base is the base reward value, set to 50, ΔA_err=|ΔA - ΔA_target| is the amplitude attenuation deviation, Δφ_err=|Δφ - Δφ_target is the phase attenuation deviation, ω1~ω4 are the weighting coefficients, σ% is the position tracking overshoot, and t_r is the position response rise time.

[0017] Furthermore, the policy update submodule adopts the Proximal Policy Optimization (PPO) algorithm, which updates the policy network parameters through "offline training + online fine-tuning". Offline training is based on the dataset collected during the actual operation of the actuator under different working conditions, and online fine-tuning is performed on the policy network parameters based on the real-time collected state data to adapt to the deviation between the real working conditions and the simulation model.

[0018] Furthermore, the threshold judgment submodule compares the phase attenuation and amplitude attenuation with the preset allowable threshold. When all attenuation amounts are within the allowable threshold, it outputs ΔK_spd=0 to maintain the current coefficient and reduce the computational load. Conversely, when any attenuation amount exceeds the threshold, it initiates strategy updates and coefficient adjustments until the attenuation amount returns to the threshold range.

[0019] The output limiting module is responsible for coefficient updates and safety limiting, ensuring that the feedforward coefficients are within a reasonable range and avoiding system instability caused by excessive compensation. Coefficient updates are calculated using the formula K_spd = ΔK_spd + K_spd. After obtaining the updated coefficients, the safety limiting module, combined with a preset coefficient limit threshold, forcibly clamps the coefficients to the corresponding limit value when the updated coefficients exceed the threshold.

[0020] The inner-loop control unit comprises a feedforward control module, a PID control module, and an SVPWM module. The feedforward control module further consists of a differentiator, a position feedforward, and a current feedforward. The position feedforward is generated using the differential signal of the position command and the feedforward coefficient K_spd output by the outer-loop intelligent control unit, expressed as ω_ff = K_spd * (dθ_ref). dt, where (dθ_ref) dt is the first derivative of the position command, reflecting the rate of change of the command calculated by the differentiator in the inner loop. K_spd is the feedforward coefficient of the outer loop dynamic adjustment, which determines the feedforward compensation strength. ω_ff is the generated velocity feedforward command signal.

[0021] Furthermore, the current feedforward unit consists of a back EMF feedforward compensation section and a cross-coupling feedforward compensation section. The back EMF feedforward compensation section reduces the influence of the back EMF on the q-axis current. This is achieved by calculating the back EMF compensation amount Uq_ffb based on the speed feedback ω_fdb and the motor permanent magnet flux linkage φf, expressed as Uq_ffb = K_p * ω_fdb * φf, where K_p is the number of pole pairs of the motor. The cross-coupling feedforward compensation section eliminates the coupling effect between the d and q axes, expressed as Uq_ffc = K_p * ω_fdb * Ld * Id_fdb, Ud_ffc = -K_p * ω_fdb * Lq * Iq_fdb, where Uq_ffc and Ud_ffc are the q-axis voltage compensation amount and d-axis voltage compensation amount, respectively, and Lq and Ld are the q-axis inductance and d-axis inductance, respectively.

[0022] The PID control module includes a position loop proportional controller, a speed loop proportional-integral controller, a q-axis current loop proportional-integral controller, and a d-axis current loop proportional-integral controller. The position loop proportional controller takes the position command θ_ref and position feedback θ_fdb as inputs, which are proportionally amplified to obtain the speed loop position command ω_pid. The speed loop proportional-integral controller takes ω_pid, the speed command feedforward signal ω_ff, and the speed feedback signal ω_fdb as inputs, and outputs Iq_pid. The q-axis current loop proportional-integral controller takes Iq_pid and Iq_fdb as inputs, and outputs the q-axis voltage Uq_pid. The d-axis current loop proportional-integral controller takes Id_ref and Id_fdb as inputs, and outputs the d-axis voltage Ud_pid, where Id_ref is a pre-specified reference, typically set to 0.

[0023] The inputs of the SVPWM module are Uq_cmd, which integrates the output of the q-axis current loop proportional-integral controller Uq_pid, the back EMF feedforward compensation part Uq_ffb, and the cross-coupled q-axis feedforward compensation part Uq_ffc; and Ud_cmd, which integrates the output of the d-axis current loop proportional-integral controller Ud_pid and the cross-coupled d-axis feedforward compensation part Ud_ffc. The expressions are: Uq_cmd = Uq_pid + Uq_ffb + Uq_ffc, Ud_cmd = Ud_pid + Uq_ffc. In these formulas, K_iq and K_id are the feedforward compensation coefficients of the outer loop intelligent control unit. Its outputs are the duty cycles of the upper transistors in phases A, B, and C of the motor, namely PWM_A, PWM_B, and PWM_C.

[0024] The execution unit drives the permanent magnet synchronous motor to rotate in a specified direction, at a specified speed and with a specified torque according to PWM_A, PWM_B and PWM_C. The rotational motion of the motor is converted into the linear motion of the tilting actuator through the ball screw, so as to achieve high-precision tracking of θ_fdb to θ_ref and form a complete closed-loop control.

[0025] Furthermore, the reinforcement learning-based dual-loop adaptive control method for eVTOL tilt actuators treats the position, speed, and current loops of the EMA as a unified control inner loop with fixed parameters. It improves the dynamic performance of the control system by introducing command feedforward control in the position and current loops. The outer loop is an online adaptive control based on a reinforcement learning agent, which adjusts the feedforward control coefficients of the position and speed loops online according to the position command and feedback curves. This achieves amplitude and phase alignment of the position loop at the target frequency, thereby improving the dynamic response and robustness of the control system.

[0026] Furthermore, the specific steps are as follows: 1. Receive control commands from the flight control computer via the flight control computer communication unit and output tilt position command signal θ_ref to the dual-loop control system; 2. The signal detection and processing unit acquires the actual position feedback signal θ_fdb, speed feedback signal ω_fdb, q-axis current feedback signal Iq_fdb, and d-axis current feedback signal Id_fdb of the tilt actuator. It uses the Kalman filter algorithm to filter and analyze the frequency of θ_fdb, ω_fdb, Iq_fdb, and Id_fdb, and calculates the amplitude attenuation ΔA and phase attenuation Δφ at a specified frequency point. 3. The outer loop control unit receives ΔA and Δφ, and outputs the position feedforward control coefficient K_spd through a reinforcement learning algorithm; 4. The inner loop control unit receives K_spd and first generates a speed feedforward signal ω_ff, an Iq current loop feedforward signal Uq_ff, and an Id current loop feedforward signal Ud_ff through the position feedforward module. Then, it combines the outputs of the speed loop PI controller (ω_pid), the Iq current loop PI controller (Uq_pid), and the Id current loop PI controller (Ud_pid) to obtain the q-axis and d-axis composite drive signals Uq_cmd and Ud_cmd. Finally, it generates the duty cycles PWM_A, PWM_B, and PWM_C of the permanent magnet synchronous motor for phases A, B, and C through the SVPWM module.

[0027] 5. The execution unit drives the permanent magnet synchronous motor according to PWM_A, PWM_B and PWM_C, and then drives the tilt actuator to achieve high-precision tracking of θ_fdb to θ_ref, forming a complete closed-loop control.

[0028] Technical effect This invention can monitor amplitude and phase decay in real time and dynamically adjust the inner loop feedforward coefficient through outer loop reinforcement learning, thereby improving the dynamic performance degradation problem under varying operating conditions. The inner loop feedforward module provides advance compensation to reduce dynamic lag, while the PID module corrects errors and suppresses disturbances. Kalman filtering further reduces noise, achieving the characteristics of "fast response + high precision + strong anti-disturbance". The optimized tilt actuator dynamic performance can ensure smooth rotor mode switching, reduce flight attitude fluctuations and reduce flight risks caused by slow actuator response, thereby improving overall flight safety. Attached Figure Description

[0029] Figure 1 This is a block diagram of the dual-loop adaptive control system proposed in this invention; Figure 2 This invention relates to the inner loop control unit based on feedforward PID control. Figure 3 This invention presents an outer-loop adaptive control system based on reinforcement learning. Figure 4 This demonstrates the control effect of the dual-loop adaptive control system proposed in this invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be described in more detail below. In the examples, the same or similar reference numerals denote the same or similar components or elements having the same or similar functions throughout. The described embodiments are some, but not all, of the embodiments of this invention. The embodiments described below with reference to reference are exemplary and intended to explain this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The embodiments of this invention will be described in detail below.

[0031] To more intuitively and clearly illustrate the structural principles and working methods in the embodiments of the present invention, the following description, in conjunction with the relevant accompanying drawings, will be provided. Figure 1 As shown, this embodiment illustrates the reinforcement learning-based dual-loop adaptive control system and method for eVTOL tilt actuators, specifically including: Step 1: Based on the dynamic response performance of the tilt actuator required by the aircraft design, set the actuator position frequency response test conditions to a sinusoidal position command signal with a frequency of 3Hz and an amplitude of 6mm. Step 2: Set the control period of the position loop proportional control to 2ms, the control period of the speed loop PI control to 200us, and the control period of the Id current loop and Iq current loop to 100us in the inner loop control unit. Step 3: Set the adjustment cycle of the outer loop control unit to 1s, set the amplitude limiting thresholds K_spd_min and K_spd_max of the position feedforward coefficient adjustment step size ΔK_spd to -0.02 and 0.02 respectively, set the base reward value R_base to 50, set the ideal amplitude attenuation deviation ΔA_target=0, set the phase attenuation deviation Δφ_target=0°, set the amplitude attenuation allowable threshold ΔA_max=30%, set the phase attenuation allowable threshold Δφ_max=45°, set the position tracking overshoot σ% to 3%, set the position response rise time t_r to 2ms, and set the weight coefficients ω1~ω4 to 50%, 20%, 15% and 15% respectively.

[0032] Step 4: Set the sampling period of the system status sampling module to 100us, the motor q-axis inductance and d-axis inductance Lq and Ld to 2.85mH and 2.61mH respectively, and the permanent magnet flux linkage φf to 29.2mVs.

[0033] Step 5, as follows Figure 2 As shown, within the inner position loop control cycle, the feedforward speed command ω_ff is obtained by receiving the tilt position command θ_ref sent by the flight control computer and using the feedforward control module. Simultaneously, the position loop proportional controller obtains the speed loop PID command ω_pid based on the position feedback θ_fdb obtained from the system state sampling module. Then, within each speed loop control cycle, the Iq_pid command is output based on the ω_pid and ω_ff speed feedback signal ω_fdb.

[0034] Step Six: Within each Id and Iq current loop control cycle, calculate the q-axis voltage Uq_pid based on the Iq_pid command and Iq_fdb, and calculate the d-axis voltage Ud_pid based on Id_ref and Id_fdb. Since maximum torque control is used, Id_ref is set to 0. Simultaneously calculate the d-axis and q-axis back EMF feedforward compensation and the cross-decoupling feedforward compensation Ud_ff and Uq_ff, and synthesize them to obtain the d-axis and q-axis voltage commands Ud_cmd and Uq_cmd.

[0035] Step 7: The SVPWM module uses Ud_cmd and Uq_cmd to obtain the duty cycles of the upper transistors of phases A, B, and C of the motor, namely PWM_A, PWM_B, and PWM_C, based on the seven-segment chopper algorithm.

[0036] Step 8: The execution unit drives the permanent magnet synchronous motor to rotate in the specified direction, speed and torque according to PWM_A, PWM_B and PWM_C. The rotational motion of the motor is converted into the linear motion of the tilting actuator through the ball screw, so as to realize the high-precision tracking of θ_fdb to θ_ref and form a complete closed loop control.

[0037] Step 9: To verify the performance of the proposed dual-loop adaptive control method, the control performance of traditional PID control and the proposed method on the tilt controller was compared. Results Figure 4 As shown, under the input conditions of a sinusoidal position command signal with a frequency of 3Hz and an amplitude of 6mm, the amplitude attenuation of traditional PID control is 30.9%, and the phase attenuation is approximately 95°. Using the method proposed in this invention, the amplitude attenuation can be controlled at 19.8%, and the phase attenuation at only 36°, meeting the preset dynamic performance response indicators. Therefore, it can be concluded that the proposed tilt actuator control system and method based on reinforcement learning can monitor amplitude and phase attenuation in real time and dynamically adjust the inner loop feedforward coefficient, improving the dynamic performance attenuation problem under varying operating conditions. It can be used for online adjustment of the feedforward factor in actuator control.

[0038] Furthermore, unless otherwise defined, the technical or scientific terms used in this application description shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "upper," "lower," "left," "right," "center," "vertical," "horizontal," "inner," and "outer," etc., used in this application description to indicate relative direction or positional relationship are used only to indicate relative orientation or positional relationship, and do not imply that the device or component must have a specific orientation, or be constructed and operated in a specific orientation. When the absolute position of the described object changes, its relative positional relationship may also change accordingly, and therefore should not be construed as a limitation on this application. The terms "first," "second," "third," and similar terms used in this application description are used only for descriptive purposes to distinguish different components, and should not be construed as indicating or implying relative importance. The terms "a," "one," or "the," etc., used in this application description should not be construed as an absolute limitation on quantity, but should be construed as indicating the existence of at least one. The terms "including," "comprising," etc., used in this application description mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, without excluding other elements or objects.

[0039] Furthermore, it should be noted that, unless otherwise explicitly specified and limited, terms such as “installation,” “connection,” and “linkage” used in the description of this application should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; or it can be a connection within two components. Those skilled in the art can understand its specific meaning in this application according to the specific circumstances.

[0040] The above description is merely a specific embodiment of the present invention and is not intended to limit the present invention. Within the spirit and principles of the present invention, any person skilled in the art may use the above-disclosed technical content to make changes or modifications to equivalent embodiments and apply them to other fields. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention, as well as any modifications, equivalent substitutions, improvements, etc., should be included within the protection scope of the present invention.

Claims

1. A dual-loop adaptive control system for eVTOL tilt actuators based on reinforcement learning, characterized in that, It includes a flight control computer communication unit, an inner loop real-time control unit, an outer loop intelligent control unit, a signal detection and processing unit, and an execution unit; The flight control computer communication unit receives position commands from the flight control computer via the aircraft communication bus and returns the real-time operating status of the dual-loop adaptive control system to the flight control computer. The inner loop control unit, a PID position loop controller based on feedforward compensation, receives position command signals, generates a lead compensation signal through the feedforward control module, and combines it with the error adjustment signal output by the PID controller to achieve real-time position control and steady-state error suppression of the tilt actuator. The outer loop control unit, a reinforcement learning-based intelligent controller, collects the command signals from the inner loop control unit and the position feedback signals from the tilt actuator in real time, extracts the amplitude attenuation and phase attenuation at specified frequency points, and dynamically outputs feedforward control coefficient adjustment commands based on preset ideal phase attenuation and amplitude attenuation ranges through a reinforcement learning algorithm to adaptively correct the control coefficients of the feedforward control module. The signal detection and processing unit collects the position feedback signals from the tilt actuator, filters and performs frequency analysis on the command and feedback signals, calculates the amplitude ratio and phase difference at specified frequency points, outputs the amplitude attenuation and phase attenuation, and simultaneously sends the position signal and calculation results to the flight control computer communication unit. The actuator, i.e. the tilt actuator, receives the synthesized drive signal from the inner loop control unit and drives the tilt mechanism of the tilt rotor eVTOL to complete the tilt angle adjustment.

2. The system as described in claim 1, characterized in that, The signal detection and processing unit uses a linear variable differential transformer to detect the tilt actuator position feedback signal, a rotary transformer to detect the speed feedback signal, a current sensor to collect three-phase current, and calculates the q-axis and d-axis feedback signals based on Clark and Park transforms.

3. The system as described in claim 2, characterized in that, The outer loop control unit is a reinforcement learning-based intelligent controller. Its inputs are the attenuation parameters output by the signal acquisition and analysis module and the operating condition parameters output by the operating condition parameter acquisition module. It dynamically generates a feedforward control coefficient adjustment instruction K_spd through a preset reinforcement learning algorithm to compensate for the amplitude and phase attenuation at a specified frequency point. It consists of three parts: a signal analysis module, a reinforcement learning intelligent module, and an output limiting module.

4. The system as described in claim 3, characterized in that, The signal analysis module receives the position command signal θ_ref and the position feedback signal θ_fdb, and uses a moving average filter to filter θ_ref and θ_fdb to suppress measurement noise and electromagnetic interference. Then, it uses a fast Fourier transform to extract the command amplitude A_θ_ref, feedback amplitude A_θ_fdb, command phase φ_θ_ref, and feedback phase φ_θ_fdb at a specified frequency point. Finally, it calculates the amplitude attenuation ΔA = 100% * (A_θ_ref - A_θ_fdb) / A_θ_ref and the phase attenuation Δφ = φ_θ_ref - φ_θ_fdb.

5. The system as described in claim 4, characterized in that, The reinforcement learning intelligent module includes a state encoding submodule, a policy network submodule, a reward calculation submodule, a policy update submodule, and a threshold judgment submodule. The state encoding submodule is used to convert continuous physical quantities into state vectors that can be processed by reinforcement learning algorithms. Its state space S is defined as S=[ΔA,Δφ, K_spd]. The action space A of the policy network submodule is the adjustment step size of the feedforward coefficients, which directly determines the coefficient update amplitude. It is defined as A=[ΔK_spd], where ΔK_spd is the adjustment step size of the position feedforward coefficients. The action output must satisfy K_spd∈[K_spd_min, K_spd_max], where K_spd_min, K_spd_max are the limit thresholds calibrated based on EMA parameters, usually taken as [-0.02, 0.02].

6. The system as described in claim 5, characterized in that, The reward value R of the reward calculation submodule is constructed using a multi-objective optimization function, expressed as: R = R_base - ω1 ΔA_err- ω2 Δφ_err - ω3 σ% - ω4 t_r, where R_base is the base reward value, set to 50, ΔA_err=|ΔA - ΔA_target| is the amplitude attenuation deviation, Δφ_err=|Δφ - Δφ_target is the phase attenuation deviation, ω1~ω4 are the weighting coefficients, σ% is the position tracking overshoot, and t_r is the position response rise time.

7. The system as described in claim 6, characterized in that, The strategy update submodule adopts a near-end strategy optimization algorithm, which updates the strategy network parameters through "offline training + online fine-tuning". Offline training is based on the dataset collected during the actual operation of the actuator under different working conditions, and online fine-tuning is performed on the strategy network parameters based on the real-time collected state data to adapt to the deviation between the real working conditions and the simulation model. The threshold judgment submodule compares the phase attenuation and amplitude attenuation with the preset allowable threshold. When all attenuation is within the allowable threshold, it outputs ΔK_spd=0 to maintain the current coefficient and reduce the amount of calculation. Conversely, when any attenuation exceeds the threshold, it initiates strategy update and coefficient adjustment until the attenuation returns to the threshold range. The output limiting module is responsible for coefficient updates and safety limiting to ensure that the feedforward coefficients are within a reasonable range. The coefficient update is calculated using the formula K_spd=ΔK_spd+K_spd; after obtaining the updated coefficient, the safety limiting module combines the preset coefficient limit threshold, and when the updated coefficient exceeds the threshold, it forcibly clamps to the corresponding limit value; The inner loop control unit comprises a feedforward control module, a PID control module, and an SVPWM module. The feedforward control module consists of a differentiator, a position feedforward, and a current feedforward. The position feedforward is generated using the differential signal of the position command and the feedforward coefficient K_spd output by the outer loop intelligent control unit, with the expression ω_ff=K_spd*(dθ_ref). dt, where (dθ_ref) dt is the first derivative of the position command, reflecting the rate of change of the command calculated by the differentiator in the inner loop. K_spd is the feedforward coefficient of the outer loop dynamic adjustment, which determines the feedforward compensation strength. ω_ff is the generated velocity feedforward command signal.

8. The system as described in claim 7, characterized in that, The current feedforward unit consists of a back EMF feedforward compensation section and a cross-coupling feedforward compensation section. The back EMF feedforward compensation section is used to reduce the influence of back EMF on the q-axis current. It is implemented by calculating the back EMF compensation amount Uq_ffb based on the speed feedback ω_fdb and the motor permanent magnet flux linkage φf, with the expression Uq_ffb=K_p*ω_fdb*φf, where K_p is the number of pole pairs of the motor. The cross-coupling feedforward compensation section is used to eliminate the coupling effect between the d and q axes, with the expression Uq_ffc=K_p*ω_fdb*Ld*Id_fdb, Ud_ffc=-K_p*ω_fdb*Lq*Iq_fdb, where Uq_ffc and Ud_ffc are the q-axis voltage compensation amount and d-axis voltage compensation amount, respectively, and Lq and Ld are the q-axis inductance and d-axis inductance, respectively. The PID control module includes a position loop proportional controller, a speed loop proportional-integral controller, a q-axis current loop proportional-integral controller, and a d-axis current loop proportional-integral controller. The inputs to the position loop proportional controller are the position command θ_ref and the position feedback θ_fdb, which are proportionally amplified to obtain the speed loop position command ω_pid. The inputs to the speed loop proportional-integral controller are ω_pid, the speed command feedforward signal ω_ff, and the speed feedback signal ω_fdb, and the output is Iq_pid. The inputs to the q-axis current loop proportional-integral controller are Iq_pid and Iq_fdb, and the output is the q-axis voltage Uq_pid. The inputs to the d-axis current loop proportional-integral controller are Id_ref and Id_fdb, and the output is the d-axis voltage Ud_pid. Id_ref is a pre-specified reference, which is usually set to 0. The input of the SVPWM module is Uq_cmd, which integrates the output of the q-axis current loop proportional-integral controller Uq_pid, the back EMF feedforward compensation part Uq_ffb, and the cross-coupled q feedforward compensation part Uq_ffc, and Ud_cmd, which integrates the output of the d-axis current loop proportional-integral controller Ud_pid and the cross-coupled d feedforward compensation part Ud_ffc. Its expression is Uq_cmd=Uq_pid+Uq_ffb+Uq_ffc, Ud_cmd=Ud_pid+Uq_ffc; where K_iq and K_id are the feedforward compensation coefficients of the outer loop intelligent control unit; its output is the duty cycle of the upper transistors of phases A, B and C of the motor, PWM_A, PWM_B and PWM_C. The execution unit drives the permanent magnet synchronous motor to rotate in a specified direction, at a specified speed and with a specified torque according to PWM_A, PWM_B and PWM_C. The rotational motion of the motor is converted into the linear motion of the tilting actuator through the ball screw, so as to realize the tracking of θ_fdb to θ_ref and form a complete closed loop control.

9. A reinforcement learning-based dual-loop adaptive control method for an eVTOL tilt actuator based on the system described in any one of claims 1-8, characterized in that, The position loop, speed loop, and current loop of the EMA are treated as a single control inner loop with fixed parameters. The dynamic performance of the control system is improved by introducing command feedforward control in the position loop and current loop. The outer loop is an online adaptive control based on reinforcement learning agent, which realizes the online adjustment of the feedforward control coefficients of the position loop and speed loop according to the curves of position command and feedback, so as to achieve the amplitude and phase alignment of the position loop at the target frequency.

10. The method as described in claim 9, characterized in that, The specific steps are as follows: 1) Receives control commands from the flight control computer via the flight control computer communication unit and outputs tilt position command signal θ_ref to the dual-loop control system; 2) The signal detection and processing unit acquires the actual position feedback signal θ_fdb, speed feedback signal ω_fdb, q-axis current feedback signal Iq_fdb, and d-axis current feedback signal Id_fdb of the tilt actuator. It uses the Kalman filter algorithm to filter and analyze the frequency of θ_fdb, ω_fdb, Iq_fdb, and Id_fdb, and calculates the amplitude attenuation ΔA and phase attenuation Δφ at a specified frequency point. 3) The outer loop control unit receives ΔA and Δφ, and outputs the position feedforward control coefficient K_spd through a reinforcement learning algorithm; 4) The inner loop control unit receives K_spd and first generates a speed feedforward signal ω_ff, an Iq current loop feedforward signal Uq_ff, and an Id current loop feedforward signal Ud_ff through the position feedforward module. Then, it combines the outputs of the speed loop PI controller (ω_pid), the Iq current loop PI controller (Uq_pid), and the Id current loop PI controller (Ud_pid) to obtain the q-axis and d-axis composite drive signals Uq_cmd and Ud_cmd. Finally, it generates the duty cycles PWM_A, PWM_B, and PWM_C of the permanent magnet synchronous motor for phases A, B, and C through the SVPWM module. 5) The execution unit drives the permanent magnet synchronous motor according to PWM_A, PWM_B and PWM_C, and then drives the tilt actuator to achieve high-precision tracking of θ_fdb to θ_ref, forming a complete closed-loop control.

Images