End effector-centric aerial manipulation and learning method

Abstract

The application discloses an end effector-centered aerial manipulation and learning method, constructs a reference coordinate system, and performs system dynamics modeling according to the reference coordinate system, including: unmanned aerial vehicle dynamics modeling, mechanical arm kinematics modeling, motor dynamics modeling, designing a model predictive control strategy MPC, using L1 adaptive compensation to compensate for external disturbances and unknown disturbances suffered during unmanned aerial vehicle and mechanical arm motor movement, realizing accurate motion control, finally constructing a teleoperation system based on vision and pose fusion and developing an action block strategy network ACT based on a Transformer structure, realizing strategy learning. The method achieves the decoupling of tasks and platforms, improves the task execution efficiency control precision and robustness, effectively deals with system modeling errors and external disturbances, can learn aerial manipulation skills from human demonstrations, has good task generalization ability, and can adapt to diversified aerial operation task requirements.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial control, and particularly relates to an aerial control and learning method with an end effector as a center. BACKGROUND

[0002] In recent years, unmanned aerial manipulators (UAM) have gradually attracted widespread attention from the industry and academia, and are widely used in high-altitude complex tasks such as high-tower bulb replacement, aircraft inspection, bridge maintenance and the like. These tasks are complex and usually consist of multiple basic movement units such as interaction, grasping, insertion and rotation. However, most of the current aerial control systems are designed for specific tasks, and the hardware design and control framework are tightly coupled, lacking universality across tasks and platforms. The end-effector-centered method is widely used in the field of robot control, and by decoupling high-level strategy and low-level control, platform and task-independent strategy development is achieved. However, in the field of aerial control, the dynamics of the unmanned aerial base and the coupling problem between the manipulator and the unmanned aerial vehicle pose great challenges to the implementation of this method. Therefore, it is urgent to develop an unmanned aerial control teleoperation and strategy learning method that can solve the above problems and has universality and high precision. The present application takes the end effector with a full-body model predictive controller MPC receiving high-level strategy instructions as a center interface, and the DH method is Denavit-Hartenberg. SUMMARY

[0003] The purpose of the present application is to solve the problem in the prior art that most of the current aerial control systems are designed for specific tasks, and the hardware design and control framework are tightly coupled, lacking universality across tasks and platforms.

[0004] To solve the above problems, the present application provides an aerial control and learning method with an end effector as a center, comprising:

[0005] S1: constructing a reference coordinate system, including: an inertial world coordinate system , a UAV body coordinate system , a manipulator base coordinate system , and an end effector coordinate system ;

[0006] S2: system dynamics modeling, including: UAV dynamics modeling, manipulator kinematics modeling, and motor dynamics modeling;

[0007] S3: designing a model predictive control strategy MPC, constructing an error function, and setting system constraints;

[0008] The system constraints include: aerial manipulator dynamics constraints, initial state constraints, running state constraints, and effector output constraints;

[0009] The operating state constraints include: self-collision avoidance constraints, environment collision avoidance constraints, and safe operation constraints.

[0010] S4: Using L1 adaptive compensation to compensate for external disturbances received by the unmanned aerial vehicle and the robot arm motor during movement With unknown disturbances , precise motion control;

[0011] S5: Constructing a teleoperation system based on vision and pose fusion;

[0012] S6: Constructing an action block policy network ACT based on the Transformer structure.

[0013] In the preferred mode, step S1 of constructing the reference coordinate system comprises:

[0014] The unmanned aerial vehicle body coordinate system The origin is located at the center of mass of the unmanned aerial vehicle, and the coordinate axes point to the front, right, and top of the unmanned aerial vehicle, respectively. The origin is located at the connection between the robot arm base and the unmanned aerial vehicle, and the coordinate axes are in the same direction as the unmanned aerial vehicle body coordinate system. The origin is located at the end of the robot arm, and the coordinates are , , point to the roll axis direction of the fourth joint of the robot arm, the horizontal direction, = * The inertia world coordinate system The axis direction is opposite to the direction of gravity, , The axis direction is the same as

[0015] In the preferred mode, step S2 of system dynamics modeling comprises:

[0016] The unmanned aerial vehicle dynamics modeling adopts the Newton-Euler equation, defines the generalized position , and the generalized velocity , wherein p is the center of mass coordinate of the unmanned aerial vehicle in the inertia world coordinate system , is the attitude rotation matrix of the unmanned aerial vehicle, is the linear velocity and angular velocity of the body, and the dynamics form is:

[0017]

[0018] wherein, is the inertia matrix, is the Coriolis force and centrifugal force term,​ is the gravity term, denotes the rotation matrix with the coordinate system index, is the normalized system control input, which takes values in the range [-1, 1], is the external disturbance, diag denotes the function that generates a diagonal matrix from a vector, is the mass of the UAV, is the 3*3 identity matrix, R is the rotation matrix, = [0, 0, -1];

[0019] The kinematics of the robotic arm is modeled using the DH method, which defines the joint angles , the joint axis displacement , the link length , the angle between the forward and rear coordinate axes , and the parameter set , denotes the real vector space of dimension 4, i denotes the i-th joint from the base of the robotic arm to the end, and the spatial transformation of the robotic arm end coordinate system relative to the robotic arm base coordinate system is denoted as:

[0020]

[0021] where, denotes the spatial transformation matrix of the coordinate system of the adjacent two joints, denotes the current joint angle of the motor;

[0022] The motor dynamics model, due to the delay in the dynamic response of the servo motor, the modeled motor response is a first-order lag system, the formula is:

[0023]

[0024] where, is the target joint angle of the motor, is the motor motion delay coefficient, is the unknown disturbance received by the motor;

[0025] S3: Design the model predictive control strategy MPC, in the MPC framework, the system state x and the control u input are defined as:

[0026]

[0027] Construct the error function, the formula is:

[0028]

[0029] where, ​denotes the current unmanned aerial manipulator end-effector position, denotes the target unmanned aerial manipulator end-effector position, 、 、 、 denotes the error function, denotes the matrix transpose, denotes the current manipulator end-effector reference pose, is the target manipulator end-effector reference pose, denotes the current unmanned aerial manipulator end-effector pose, is the vee operator, which converts the skew-symmetric matrix into a vector, is the manipulator end-effector reference generalized velocity, is the manipulator motor reference angle, is the system reference control signal, is the current manipulator joint angle observation, and the objective function is:

[0030]

[0031] wherein, is the optimal control sequence, and argmin denotes the optimization operator, denotes the prediction window length, is the manipulator trajectory, is the reference trajectory, is the prediction window length reference trajectory, and are both quadratic functions with respect to the tracking error, and n denotes the nth vector in the sequence; the system constraints are considered as follows:

[0032]

[0033] correspond to the aerial manipulator dynamics constraint , the initial state constraint , the operational state constraint , and the actuator output constraint , the continuous form of the aerial manipulator dynamics is given by the system dynamics modeling in the above step S2, denoted as , denotes the current state estimation of the system; the operational state constraint is represented as the system trajectory being within the reachable state set , including: self-collision avoidance constraint: ensuring that the manipulator does not collide with the unmanned aerial vehicle body structure during movement; environment collision avoidance constraint: preventing the unmanned aerial vehicle from contacting external obstacles; safe operation constraint: including speed limit and joint angle limit;

[0034] The actuator output constraint imposes a restriction on the actuator of the aerial manipulator, wherein and define the lower and upper limits of the control input, respectively, the lower limit being [0, 0, 0, 0] and the upper limit being ;

[0035] S4: Constructing L1 adaptive compensation for external disturbances received by the unmanned aerial vehicle and the manipulator motor during movement and unknown disturbances to achieve precise motion control;

[0036] S4-1: The unmanned aerial vehicle L1 compensation law is:

[0037]

[0038] wherein, represents the estimated generalized velocity, represents the body L1 adaptive disturbance compensation term, represents the projection gain matrix for velocity compensation, represents the exponential form of the projection matrix, represents a 6*6 unit matrix, represents the body differential disturbance compensation, represents a low-pass filtering operation;

[0039] S4-2: The manipulator compensation law is:

[0040]

[0041] wherein, represents the estimated angular velocity, is the motor L1 adaptive disturbance compensation term, is the disturbance compensation projection matrix, is the exponential term of the projection matrix, is a 4*4 unit matrix, is the motor differential disturbance compensation;

[0042] S5: Constructing a teleoperation system based on fusion of vision and pose;

[0043] Real-time capture of dual-view image information is performed by using an RGB camera installed on the end effector of a mechanical arm and an RGB camera installed on the base of a UAV, image features are obtained by using a feature extraction network ResNet-18 to encode and fuse the dual-view image information, and the end effector target trajectory of the user instruction is rendered in a three-dimensional virtual environment by using the obtained image features and real-time pose data; the spatial motion instruction of the mechanical arm end effector input by the user through the teleoperation device is converted into a command sequence of the UAV and the mechanical arm torque executable by the robot through the whole body model predictive controller through the end effector center interface, and the command sequence is a command sequence of the target UAV aerial mechanical arm end effector position and the target UAV aerial mechanical arm end effector attitude .

[0044] S6: Develop an action block policy network ACT based on the Transformer structure, and the policy network ACT output and observation are defined as follows:

[0045]

[0046] Wherein, ACT policy, network parameters, represent the current time, action block size, , RGB images from the base camera and the end effector camera, respectively, both with a resolution of 640*480, using ResNet-18 as the main network for feature extraction, the RGB images are encoded and input into the Transformer encoder, target UAV aerial mechanical arm end effector attitude.

[0047] The beneficial effects of the present application are as follows: improve the universality and operation efficiency; through the end effector center control paradigm, the task and platform are decoupled, which is suitable for various combinations of UAV and mechanical arm; the proposed mechanical arm end coordinate interface is intuitive and easy to use, which significantly reduces the operation complexity and improves the task execution efficiency. Improve control accuracy and robustness. Combined with whole body model predictive control and L1 adaptive compensation method, effectively deal with system modeling error and external disturbance, realize high precision trajectory tracking of end effector and robust control of system. Support imitation learning and multi-task adaptation. Construct the imitation learning strategy network based on Transformer, which can learn aerial operation skills from human demonstration, has good task generalization ability, and can adapt to diversified aerial operation task requirements. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is a system structure schematic diagram;

[0049] Figure 2 Trajectory tracking experimental results for end effector;

[0050] Figure 3 Trajectory tracking experimental results for end effector;

[0051] Figure 4 Trajectory tracking experimental results for end effector;

[0052] Figure 5 Trajectory tracking experimental results for end effector;

[0053] Figure 6 Trajectory tracking experimental results for end effector;

[0054] Figure 7 Trajectory tracking experimental results for end effector;

[0055] Figure 8 Imitation learning task execution results schematic diagram;

[0056] Figure 9 Imitation learning task execution results schematic diagram;

[0057] Figure 10 Imitation learning task execution results schematic diagram;

[0058] Figure 11 Imitation learning task execution results schematic diagram;

[0059] Figure 12 Flight process execution results schematic diagram;

[0060] Figure 13 Valve installation rotation execution results schematic diagram;

[0061] Figure 14 Pick and place execution schematic diagram;

[0062] Figure 15 Bulb disassembly results schematic diagram;

[0063] Figure 16 Bulb installation results schematic diagram;

[0064] Figure 17 The execution results schematic diagram of the application;

[0065] Figure 18 The end effector coordinate change curve schematic diagram of the application;

[0066] Figure 19 The reference coordinate system schematic diagram of the application. DETAILED DESCRIPTION

[0067] Example 1: System modeling and controller design;

[0068] The full-drive six-rotor unmanned aerial vehicle carried by the system can independently generate three-axis force and torque, and the mechanical arm is installed in the center of the body. The inertia parameters and DH parameters of the mechanical arm are identified through fitting algorithm, and the overall system dynamics model is constructed. This model can be used for multi-task simulation and verification.

[0069] Example 2: MPC trajectory controller solution and constraint design, MPC is the full-body model predictive controller;

[0070] Optimization solution is performed under discrete prediction range H=200 (2s). The trajectory error weight matrix is set through theoretical calculation and tuning in experiments, and the control input range , is the actual physical limit. The constraints include avoiding collision with itself and the environment, mechanical arm joint angle boundary, end acceleration limit, etc.

[0071] Example 3: L1 adaptive controller implementation;

[0072] For unmanned aerial vehicle control input , the disturbance compensation term is generated based on the estimated speed error, and the filter bandwidth is set through . Similarly, for each mechanical arm joint, set . The controller smoothes the disturbance estimate through a low-pass filter, and L1 adaptive control (L1-AC) is an advanced control method that estimates system uncertainty in real time and dynamically adjusts the control law, widely used in aircraft control and other fields.

[0073] Example 4: Teleoperation interface and user input design;

[0074] Use a remote controller or handle to control the 6D pose of the end effector, and establish a two-way communication channel between the user and the robot. The camera image is transmitted to the control terminal in real time, and the 3D trajectory feedback enhances the spatial perception ability. The system supports autonomous / teleoperation switching.

[0075] Example 5: Imitation learning strategy training and deployment;

[0076] Collect more than 100 teleoperation demonstration trajectories, input RGB image + end pose, and output future trajectory. During training, use CVAE structure to encode trajectory style, and use prior mean to achieve generalization during testing. The strategy network can achieve efficient migration in unknown tasks after deployment.

[0077] This system can be widely used in various task scenarios, including air writing, hole pinning, precision grabbing, bulb replacement, etc., and has good platform expansion and task adaptability.

[0078] Example 6: MPC control algorithm implementation details

[0079] The optimal control problem in the end-effector-centric MPC is implemented through ACADOS with a discrete-time step of 25 milliseconds and a fixed prediction horizon of 2.5 seconds, running at a frequency of 100 Hz. The control output is executed in a receding horizon fashion, i.e. in each iteration, only the first control input is applied to the control signal.

[0080] Example 7: System identification implementation details

[0081] Since both the L1 adaptive controller and the MPC controller require an accurate system modeling function to achieve effective control performance, we performed system identification to estimate the model uncertain parameters. We employed two types of motions to excite the system:

[0082] (1) Execute only the manipulator motion trajectory while keeping the UAV stationary, for calibrating the DH parameters and joint servo delay parameters . These trajectories ensure that the manipulator kinematic parameters and joint motor dynamics accurately reflect the actual response of the manipulator. (2) Perform UAV free-flight trajectory experiments to identify the aircraft dynamics model.

[0083] Example 8:

[0084] An end-effector-centric aerial manipulation and learning method, comprising:

[0085] An end-effector-centric interface that receives high-level policy instructions from a full-body model predictive controller MPC; An end-effector-centric teleoperation system and imitation learning policy

[0086] S1: Constructing a reference coordinate system includes: an inertial world coordinate system , a UAV body coordinate system , a manipulator base coordinate system , and an end-effector coordinate system ;

[0087] S2: System dynamics modeling includes: UAV dynamics modeling, manipulator kinematics modeling, motor dynamics model

[0088] S3: Designing a model predictive control policy MPC, constructing an error function, and setting system constraints

[0089] Constraint conditions include:

[0090] Aerial manipulator dynamic constraints, initial state constraints, operating state constraints and actuator output constraints;

[0091] Runtime constraints include:

[0092] Self-collision avoidance constraints, environmental collision avoidance constraints, and safe operation constraints;

[0093] S4: Using L1 adaptive compensation for external disturbances in the motion of the drone and robotic arm motors With unknown disturbance , precise motion control;

[0094] S5: Construct a teleoperation system based on vision and posture fusion; S6: Construct an action block strategy network ACT based on Transformer structure, such as Figure 19 As shown, step S1 of constructing the reference coordinate system includes:

[0095] UAV body coordinate system The origin is located at the center of mass of the drone, and the coordinate axis Pointing to the front, right and top of the drone respectively; the coordinate system of the robotic arm base The origin is located at the connection between the robotic arm base and the drone, and the directions of the coordinate axes are the same as the drone body coordinate system; the end effector coordinate system The origin is located at the end of the robotic arm, and the coordinates are , 、 They point to the roll axis direction and horizontal direction of the fourth joint of the robotic arm respectively. = * ; Inertial world coordinate system of The direction of the axis is opposite to the direction of gravity. 、 Axis direction and same.

[0096] Step S2 system dynamics modeling includes:

[0097] The UAV dynamics model uses the Newton-Euler equation to define the generalized position , generalized velocity , where p is the inertial world coordinate system Next, the coordinates of the center of mass of the UAV, is the UAV attitude rotation matrix, are the linear velocity and angular velocity of the body respectively, and the dynamic form is:

[0098]

[0099] in, is the inertia matrix, is the Coriolis and centrifugal force term, is the gravity term, denotes the rotation matrix with the coordinate system index, is the normalized system control input, which takes values in the range [-1, 1], is the external disturbance, diag denotes the function that generates a diagonal matrix from a vector, is the mass of the UAV, is the 3*3 identity matrix, R is the rotation matrix, = [0, 0, -1];

[0100] The kinematic model of the manipulator is modeled using the DH method, defining the joint angles , the joint axis displacement , the link lengths , the angles between the forward and rear coordinate axes , and the parameter set , denotes the real vector space of dimension 4, i denotes the i-th joint from the base to the end of the manipulator, then the spatial transformation of the manipulator end coordinate system relative to the manipulator base coordinate system is denoted as:

[0101]

[0102] where, denotes the spatial transformation matrix of the coordinate system of the adjacent two joints, denotes the current joint angle of the motor; the motor dynamics model, due to the delay in the dynamic response of the servo motor, the modeled motor response is a first-order lag system, the formula is:

[0103]

[0104] where, is the target joint angle of the motor, is the motor motion delay coefficient, is the unknown disturbance received by the motor;

[0105] S3: design the model predictive control strategy MPC, in the MPC framework, the system state x and the control u input are defined as:

[0106]

[0107] Construct the error function, the formula is:

[0108]

[0109] where, ​represents the current unmanned aerial manipulator end-effector position, represents the target unmanned aerial manipulator end-effector position;

[0110] 、 、 、 represents the error function, represents the matrix transpose, represents the current manipulator end-effector reference pose;

[0111] is the target manipulator end-effector reference pose, represents the current unmanned aerial manipulator end-effector pose, is the vee operator, which serves to convert the skew-symmetric matrix into a vector, is the manipulator end-effector reference generalized velocity, is the manipulator motor reference angle, is the system reference control signal, is the current manipulator joint angle observation, and the objective function is:

[0112]

[0113] wherein, is the optimal control sequence, and argmin represents the optimization operator, represents the prediction window length, is the manipulator trajectory, is the reference trajectory, is the prediction window length reference trajectory, and are both quadratic functions with respect to the tracking error, and n represents the nth vector in the sequence;

[0114] The system constraints are considered as follows:

[0115]

[0116] correspond to the aerial manipulator dynamics constraints , initial state constraints , operating state constraints , and actuator output constraints , respectively;

[0117] The continuous form of the aerial manipulator dynamics is given by the above step S2 system dynamics modeling, represented as , represents the current state estimate of the system;

[0118] The operating state constraints are represented as the system trajectory being in the reachable state set The inner loop includes: self-collision avoidance constraints: ensure that the robot arm does not collide with the UAV body structure during movement; environmental collision avoidance constraints: prevent the UAV from contacting external obstacles; safe operation constraints: including speed limits and joint angle limits; actuator output constraints apply actuator limits to the aerial manipulator, where and define the lower and upper limits of the control input, respectively, the lower limit is [0, 0, 0, 0], and the upper limit is ;

[0119] S4: Constructing L1 adaptive compensation for external disturbances received by the UAV and the robot arm during motor movement and unknown disturbances to achieve precise motion control;

[0120] S4-1: The UAV L1 compensation law is:

[0121]

[0122] where, denotes the estimated generalized velocity, denotes the body L1 adaptive disturbance compensation term, denotes the projection gain matrix for velocity compensation, denotes the exponential form of the projection matrix, denotes a 6*6 identity matrix, denotes the body differential disturbance compensation, denotes a low-pass filtering operation;

[0123] S4-2: The robot arm compensation law is:

[0124]

[0125] where, denotes the estimated angular velocity, is the motor L1 adaptive disturbance compensation term, is the disturbance compensation projection matrix, is the exponential term of the projection matrix, is a 4*4 identity matrix, is the motor differential disturbance compensation;

[0126] S5: Constructing a teleoperation system based on vision and pose fusion;

[0127] Real-time capture of dual-view image information by using an RGB camera mounted on the end effector of a mechanical arm and an RGB camera mounted on the base of a UAV, encoding and fusing the dual-view image information by using a feature extraction network ResNet-18 to obtain image features, and rendering the end effector target trajectory of the user instruction in a three-dimensional virtual environment by using the obtained image features and real-time pose data; through the end effector as the center interface, the spatial motion instruction of the mechanical arm end input by the user through the teleoperation device is converted into a command sequence of the UAV and the mechanical arm torque executable by the robot through the whole body model predictive controller, and the command sequence is the command sequence of the target UAV aerial mechanical arm end position and the target UAV aerial mechanical arm end pose .

[0128] S6: develop an action block policy network ACT based on the Transformer structure, and the output of the policy network ACT and observation are defined as follows:

[0129]

[0130] wherein, ACT policy, network parameters, current time, action block size, , RGB images from the base camera and the end effector camera, respectively, both with a resolution of 640*480, using ResNet-18 as the backbone network for feature extraction, and inputting the encoded RGB images into the Transformer encoder, target UAV aerial mechanical arm end pose.

[0131] As shown in Figure 2 , Figure 3 , Figure 4 , the end effector trajectory tracking experimental results are shown, the black line in the figure represents the target value, the blue line represents the experimental value of the application, the orange dashed line represents the MPC model experimental value, the green dashed line represents the L1 model experimental value, the horizontal coordinate represents time, unit: seconds / s, and the vertical coordinate represents distance, unit: meters / m;

[0132] As shown in Figure 5 , Figure 6 , Figure 7 , the imitation learning task execution results are shown, the blue line in the figure represents the experimental value, and the gray line represents the target value; as shown in Figure 8 , Figure 9 , Figure 10 , Figure 11 , the imitation learning task execution results are shown, and as shown in Figure 12As shown, the execution result of the flight process is displayed; as Figure 13 As shown, the execution result of the valve installation rotation is displayed; as Figure 14 The execution result of the taking and placing is displayed, Figure 15 The execution result of the bulb dismounting is displayed, Figure 16 The execution result of the bulb installation is displayed, as Figure 17 As shown, the execution result is displayed, wherein 1 represents a UAV takeoff schematic diagram, 2 represents a UAV flying towards a target schematic diagram, 3 represents a UAV insertion schematic diagram, 4 represents a UAV opening a clamp schematic diagram, 5 represents a UAV returning schematic diagram, and 6 represents a UAV landing schematic diagram. Figure 18 As shown, the end effector coordinate change curve is displayed, wherein the horizontal coordinate represents time, the unit is second / s, the vertical coordinate represents position, the unit is meter / m, the red line represents an end effector x-axis position curve, the green line represents an end effector y-axis position curve, the purple line represents an end effector z-axis position curve, and the black line represents a clamp position curve.

[0133] The method is divided into two parts of the end of the mechanical arm control and strategy learning. The end of the mechanical arm accurate control is the basis of strategy learning. The end of the mechanical arm control is defined as a model predictive control (MPC) problem, mainly including a system modeling step S2 and an MPC solving step S3. Meanwhile, because there is a modeling error and unknown disturbance between modeling and real machine, an L1 adaptive control step S4 is introduced to align the system response of the real machine with the modeling response.

[0134] The basic principles and main features of the present application and the advantages of the present application are shown and described above. It should be understood by those skilled in the art that the present application is not limited by the above examples. The above examples and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application. These changes and improvements fall within the scope of the present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. An end-effector-centered aerial control and learning method, characterized in that: include: S1: Constructing a reference coordinate system includes: Inertial world coordinate system , UAV body coordinate system , Robot arm base coordinate system , end effector coordinate system ; S2: System dynamics modeling includes: UAV dynamics modeling, robotic arm kinematics modeling, motor dynamics model; S3: Design the model predictive control strategy MPC, construct the error function, and set the system constraints; System constraints include: Aerial manipulator dynamic constraints, initial state constraints, operating state constraints and actuator output constraints; Runtime constraints include: Self-collision avoidance constraints, environmental collision avoidance constraints, and safe operation constraints; S4: Using L1 adaptive compensation for external disturbances in the motion of the drone and robotic arm motors With unknown disturbance , achieving precise motion control; S5: Build a teleoperation system based on vision and posture fusion; S6: Construct an action block strategy network ACT based on the Transformer structure.

2. The end-effector-centered aerial control and learning method according to claim 1, characterized in that: Step S1 builds the reference coordinate system including: UAV body coordinate system The origin is located at the center of mass of the drone, and the coordinate axis Pointing to the front, right and top of the drone respectively; the coordinate system of the robotic arm base The origin is located at the connection between the robotic arm base and the drone, and the directions of the coordinate axes are the same as the drone body coordinate system; the end effector coordinate system The origin is located at the end of the robotic arm, and the coordinates are , 、 They point to the roll axis direction and horizontal direction of the fourth joint of the robotic arm respectively. = * ; Inertial world coordinate system of The direction of the axis is opposite to the direction of gravity. 、 Axis direction and same.

3. The end-effector-centered aerial control and learning method according to claim 1, characterized in that: Step S2 system dynamics modeling includes: The UAV dynamics model uses the Newton-Euler equation to define the generalized position , generalized velocity , where p is the inertial world coordinate system Next, the coordinates of the center of mass of the UAV, is the UAV attitude rotation matrix, are the linear velocity and angular velocity of the body respectively, and the dynamic form is: in, is the inertia matrix, are the Coriolis force and centrifugal force terms, is the gravity term, Indicates adding a coordinate system subscript to the rotation matrix. is the normalized system control input, with a value range of [-1, 1], is an external disturbance, diag represents a diagonal matrix function generated according to a vector, For drone quality, is a 3*3 unit matrix, R is a rotation matrix, =[0, 0, -1]; The kinematic modeling of the robotic arm uses the DH method to define the joint angles , joint axial displacement , connecting rod length Angle between front and rear coordinate axes Parameter group , represents a real vector space with a dimension of 4; i represents the i-th joint from the base of the robot to the end, then the coordinate system of the end of the robot is Relative to the robot base coordinate system Space transformation Expressed as: in, Represents the spatial transformation matrix of two adjacent joint coordinate systems, Indicates the current joint angle of the motor; Motor dynamics model, due to the delay in the dynamic response of the servo motor, the modeled motor response is a first-order lag system, the formula is: in, is the motor target joint angle, is the motor motion delay coefficient, is the unknown disturbance to the motor; S3: Design the model predictive control strategy MPC. In the MPC framework, the system state x and control u input are defined as: Construct the error function, the formula is: in, Indicates the current end position of the unmanned aerial robotic arm. Indicates the end position of the target unmanned aerial manipulator, 、 、 、 represents the error function, represents the matrix transpose, Indicates the current reference posture of the end of the robotic arm. is the reference posture of the target robotic arm end, Indicates the current posture of the unmanned aerial manipulator end. It is the vee operator, which converts the antisymmetric matrix Converted to a vector, is the generalized reference velocity of the end of the manipulator, is the reference angle of the robot arm motor, is the system reference control signal, is the current robot arm joint angle observation value, and the objective function is: in, is the optimal control sequence, argmin represents the optimization operator, represents the prediction window length, is the trajectory of the robotic arm, is the reference trajectory, is the reference trajectory for the prediction window length, and are quadratic functions of tracking error, n represents the nth vector in the sequence; Consider the following system constraints: Corresponding to the dynamic constraints of the aerial manipulator , initial state constraints , operating state constraints and actuator output constraints , the continuous form of the aerial manipulator dynamics is given by the system dynamics modeling in step S2 above, which is expressed as , Represents the current state estimate of the system; The running state constraint is expressed as the system trajectory in the set of reachable states Including: Self-collision avoidance constraint: ensures that the robotic arm does not collide with the drone's main structure during movement; Environmental collision avoidance constraints: prevent the drone from coming into contact with external obstacles; Safety operation constraints: including speed limits and joint angle limits; The actuator output constraints impose actuator restrictions on the aerial manipulator, where and The lower limit and upper limit of the control input are defined respectively, the lower limit is [0, 0, 0, 0], and the upper limit is ; S4: Build L1 adaptive compensation for external disturbances in the motion of drone and manipulator motors With unknown disturbance , to achieve precise motion control; S4-1: The L1 compensation law of the drone is: in, represents the estimated generalized velocity, represents the L1 adaptive disturbance compensation term of the body, represents the projection gain matrix for velocity compensation, represents the exponential form of the projection matrix, represents the 6*6 identity matrix, represents the body differential disturbance compensation, Represents a low-pass filtering operation; S4-2: The compensation law of the robot arm is: in, represents the estimated angular velocity, is the motor L1 adaptive disturbance compensation term, is the perturbation compensation projection matrix, is the exponential term of the projection matrix, is the 4*4 identity matrix, is the motor differential disturbance compensation; S5: Build a teleoperation system based on vision and posture fusion; Using the RGB camera installed on the end effector of the robotic arm and the RGB camera installed on the base of the UAV, dual-view image information is captured in real time. The feature extraction network ResNet-18 is used to encode and fuse the dual-view image information to obtain image features. The obtained image features and real-time posture data are used to render the end effector target trajectory of the user's instructions in a three-dimensional virtual environment; with the end effector as the central interface, the spatial motion instructions of the robotic arm input by the user through the remote control device are converted into a command sequence of drone and robotic arm torque that can be executed by the robot through the whole-body model prediction controller. The command sequence is the target unmanned aerial robotic arm end position. and the target unmanned aerial manipulator end posture The instruction sequence; S6: Develop an action block strategy network ACT based on the Transformer structure. The output and observation definitions of the strategy network ACT are as follows: in, Indicates the ACT strategy, are network parameters, Indicates the current moment, is the action block size, 、 The RGB images are from the base camera and the end effector camera, respectively, with a resolution of 640*480. ResNet-18 is used as the backbone network for feature extraction. The RGB images are encoded and input into the Transformer encoder. Indicates the end pose of the target unmanned aerial manipulator.

Images