Adaptive control method and system for unmanned aerial vehicle automatic arm changing multi-task aerial operation

Abstract

The application discloses an adaptive control method and system for automatic arm replacement multi-task aerial operation of a UAV, and comprises the following steps: training a deep neural network by using a loss function according to a first training set to obtain a target disturbance force model; regarding the UAV as a mass center and regarding a mechanical arm as a spring, a mass center spring model is constructed by simulating the disturbance force of the mechanical arm by a spring force, and then the optimized expected trajectory of the UAV and the expected trajectory of the mechanical arm are obtained according to the disturbance force of the mechanical arm calculated according to an initial trajectory of the mechanical arm; when the mechanical arm is replaced, the disturbance moment of the mechanical arm is obtained by the target disturbance force model, and the dynamics model of the UAV is corrected according to the disturbance moment; finally, the inverse kinematics of the joints of the mechanical arm and the impedance control of each joint are combined, so that the replaced mechanical arm is used to perform an operation task in the flight process of the UAV; the application can accurately realize the multi-task aerial operation of the UAV after the mechanical arm is replaced, and can be widely applied to the technical field of the UAV.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to an adaptive control method and system for automatic arm-changing multi-mission aerial operations of UAVs. Background Technology

[0002] Aerial operation of drones has attracted increasing attention, but most research focuses on non-contact flight, such as object detection, precision agriculture, and disaster monitoring. This is because the interactive behavior of aerial operating systems can introduce additional disturbances to drones, affecting the stability of the unmanned operating system. To perform different tasks, unmanned operating systems require different robotic arms, and changing different robotic arms and interactions with the environment can introduce sudden disturbances.

[0003] For drones, aerial operation involves many challenges, such as: 1) Drones carrying payloads and robotic arms introduce uncertainty into the modeling of the entire system. In practice, due to the complexity of aerial operating systems, it is difficult to obtain an accurate model, and without an accurate model, dynamic programming and control of stable interactions will be quite difficult; 2) Compensating for disturbances caused by robotic arm movements to maintain the stability of the aerial operating system, and compensating for energy losses caused by robotic arm disturbances will shorten the flight time. Summary of the Invention

[0004] The main objective of this application is to propose an accurate and robust adaptive control method and system for automatic arm-changing multi-task aerial operations of unmanned aerial vehicles (UAVs). The aim is to achieve trajectory optimization and tracking control for automatic arm changing, enabling the completion of various aerial operation tasks without the need for manual adjustment of controller parameters.

[0005] To achieve the above objectives, one aspect of this application proposes an adaptive control method for automatic arm-switching multi-mission aerial operations of unmanned aerial vehicles (UAVs), the method comprising:

[0006] Based on the first training set, a loss function is used to train the deep neural network to obtain the target disturbance dynamics model; wherein, the first training set includes system flight data obtained by the UAV equipped with different robotic arms;

[0007] Using the drone as the center of mass and the robotic arm as a spring, the disturbance force of the robotic arm is simulated by the spring force to construct a center of mass spring model;

[0008] Based on the centroid spring model, the optimized expected trajectory of the UAV and the expected trajectory of the robotic arm are obtained by calculating the disturbance force of the robotic arm based on the initial trajectory of the robotic arm.

[0009] When replacing the robotic arm, the disturbance torque of the robotic arm is obtained through the target disturbance force model, and the UAV dynamics model is corrected based on the disturbance torque;

[0010] Based on the expected trajectory of the robotic arm and the expected trajectory of the drone, and combined with the inverse kinematics of the robotic arm joints and the impedance control of each joint, the drone uses the replaced robotic arm to perform operational tasks during flight; wherein, the drone flies based on the drone dynamics model.

[0011] In some embodiments, training the deep neural network using a loss function based on a first training set to obtain the target perturbation model includes:

[0012] Obtain the robotic arm number, aircraft attitude, relative position between the aircraft and the end effector, and total thrust of the aircraft to obtain the first training set;

[0013] By combining cross-entropy loss, the loss function of the deep neural network model is determined;

[0014] The deep neural network model is trained using the first training set until the loss function converges or the number of training iterations reaches a preset threshold, thereby obtaining the target perturbation model; wherein, the target perturbation model is used to classify and fit the perturbation forces of the robotic arm.

[0015] In some embodiments, in the step of constructing a center-of-mass spring model by taking the drone as the center of mass and the robotic arm as a spring, and simulating the disturbance force of the robotic arm with spring force, the center-of-mass spring model includes a first kinematic model and a robotic arm disturbance force model, wherein:

[0016] The expression for the first kinematic model is:

[0017]

[0018] The expression for the disturbance dynamics model of the robotic arm is:

[0019]

[0020] Where, p Q Indicates the position of the UAV's center of mass; p m The position of the robotic arm's end effector is indicated by: m; the coordinates of the manipulator are indicated by: l; the distance between the robotic arm's end effector and the drone's center of mass is indicated by: F. arm This represents the result of the disturbance force of the robotic arm; k represents the spring constant; b represents the damping coefficient of the spring; Δl represents the change in distance between the end of the robotic arm and the center of mass of the drone. ψ represents the velocity of the robotic arm's end effector relative to the drone's center of mass; ψ represents the yaw angle of the quadcopter; θ represents the angle between the straight line from the robotic arm's end effector to the drone's center of mass and the z-axis of the drone's body coordinate system.

[0021] In some embodiments, the step of obtaining the optimized expected trajectory of the UAV and the expected trajectory of the robotic arm based on the disturbance force of the robotic arm calculated from the initial trajectory of the robotic arm according to the centroid spring model includes the following step:

[0022] Based on the input initial robotic arm trajectory, determine the given waypoint of the UAV end effector;

[0023] Based on the initial robotic arm trajectory, the robotic arm disturbance force is calculated using the robotic arm disturbance force model.

[0024] Based on the given waypoint, the expected waypoint of the UAV is calculated using the first kinematic model of the center of mass spring model;

[0025] Based on the desired waypoint and the disturbance force of the robotic arm, the expected trajectory of the UAV is generated by a polynomial spline function.

[0026] In some embodiments, the step of obtaining the optimized expected trajectory of the UAV and the expected trajectory of the robotic arm based on the disturbance force of the robotic arm calculated from the initial trajectory of the robotic arm according to the centroid spring model includes the following step:

[0027] The direction of the perturbation force of the robotic arm is determined by the perturbation force model of the center of mass spring model.

[0028] Based on the direction of the disturbance force of the robotic arm, the expected trajectory of the robotic arm is generated by a polynomial spline function.

[0029] In some embodiments, when replacing the robotic arm, the perturbation torque of the robotic arm is obtained through the target perturbation force model, and the UAV dynamics model is corrected based on the perturbation torque, including:

[0030] Based on the target disturbance force model, the disturbance torque of the robotic arm is determined;

[0031] The feedforward control law of the aircraft is determined based on the nonlinear feedforward term, the reference controller feedback term, and the disturbance torque of the robotic arm.

[0032] Based on the feedforward control law, the UAV dynamics model is corrected through the feedback control law.

[0033] In some embodiments, the step of using the replaced robotic arm to perform operational tasks during drone flight, based on the expected trajectory of the robotic arm and the expected trajectory of the drone, combined with the inverse kinematics of the robotic arm joints and the impedance control of each joint, includes:

[0034] The control error of the robotic arm joint is determined based on the current angle of the robotic arm joint and the desired angle of the robotic arm joint.

[0035] The impedance control law of the robotic arm joint is determined based on the impedance control error of each of the robotic arm joints.

[0036] The robotic arm is controlled to perform operational tasks based on the expected trajectory and the impedance control law.

[0037] To achieve the above objective, another aspect of this application proposes an adaptive control system for automatic arm-switching multi-mission aerial operations of unmanned aerial vehicles (UAVs), the system comprising:

[0038] The first module is used to train a deep neural network using a loss function based on a first training set to obtain a target perturbation model; wherein, the first training set includes system flight data obtained by the UAV equipped with different robotic arms.

[0039] The second module is used to construct a center-of-mass spring model by using the drone as the center of mass and the robotic arm as a spring, and simulating the disturbance force of the robotic arm with spring force.

[0040] The third module is used to obtain the optimized expected trajectory of the UAV and the expected trajectory of the robotic arm based on the centroid spring model and the disturbance force of the robotic arm calculated from the initial trajectory of the robotic arm.

[0041] The fourth module is used to obtain the disturbance torque of the robotic arm through the target disturbance force model when replacing the robotic arm, and to correct the UAV dynamics model based on the disturbance torque.

[0042] The fifth module is used to, based on the expected trajectory of the robotic arm and the expected trajectory of the UAV, and in combination with the inverse kinematics of the robotic arm joints and the impedance control of each joint, enable the UAV to perform operational tasks using the replaced robotic arm during flight; wherein the UAV flies based on the UAV dynamics model.

[0043] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0044] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method.

[0045] The embodiments of this application include at least the following beneficial effects: This application provides an adaptive control method and system for multi-task aerial operation of UAVs with automatic arm changing. This scheme combines the dynamics module of the UAV with a single rigid body model to generate an optimized trajectory for the aerial operating system. The controller based on a deep neural network can classify and accurately learn the disturbance forces and disturbance torques generated by the interaction between different robotic arms and the environment. It has high accuracy and robustness in trajectory planning for UAVs and robotic arms, and can ultimately accurately realize the UAV to change robotic arms for multi-task aerial operation. Attached Figure Description

[0046] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0047] Figure 1 This is a flowchart of the adaptive control method for automatic arm-switching multi-task aerial operations of unmanned aerial vehicles provided in the embodiments of this application;

[0048] Figure 2 This is a flowchart illustrating the specific implementation of the adaptive control method for automatic arm-switching multi-task aerial operations of unmanned aerial vehicles provided in this application embodiment;

[0049] Figure 3 This is a schematic diagram of the spring center of mass model provided in the embodiments of this application;

[0050] Figure 4 This is a graph showing the neural network learning effect of a robotic arm carrying different load depths, as provided in the embodiments of this application.

[0051] Figure 5 This is a scene diagram provided in this application embodiment of a drone carrying a hook-attaching robotic arm flying to a cabinet to attach hooks;

[0052] Figure 6 This is a scene diagram of a drone carrying a hook-attaching robotic arm, as provided in an embodiment of this application, lowering the hook-attaching robotic arm.

[0053] Figure 7 This is a field image of the drone being equipped with a tag-hanging robotic arm, as provided in an embodiment of this application.

[0054] Figure 8 This is a scene image provided in an embodiment of this application, showing a drone carrying a robotic arm that hangs signs flying to retrieve signs.

[0055] Figure 9 This is a scene diagram provided in an embodiment of this application, showing a drone carrying a robotic arm that hangs signs flying to a cabinet and placing the signs onto the hooks.

[0056] Figure 10This is a scene diagram of a drone carrying a tag-hanging robotic arm flying back to its starting position, provided in an embodiment of this application.

[0057] Figure 11 This is a schematic diagram of the structure of the adaptive control system for automatic arm-changing multi-task aerial operation of unmanned aerial vehicles provided in the embodiments of this application;

[0058] Figure 12 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0060] Although functional modules are divided in the system diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100," "second / S200," etc., in the specification, claims, and the aforementioned figures are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0061] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0062] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0063] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0065] For drones, aerial operation involves many challenges. First, the payload and robotic arm carried by the drone introduce uncertainty into the modeling of the entire system. In practice, due to the complexity of the aerial operating system, it is difficult to obtain an accurate model. Without an accurate model, dynamic programming and control for stable interaction become extremely difficult. Second, it is necessary to compensate for disturbances caused by the robotic arm's movement to maintain the stability of the aerial operating system, but compensating for energy losses caused by robotic arm disturbances will shorten the flight time.

[0066] To achieve interactions with the environment, such as aerial grasping, opening drawers / doors, and sliding along irregular surfaces, trajectory optimization is crucial for aerial operating systems. Compared to single-drone systems, aerial operations focus more on the end effector trajectory of the robotic arm and less on the quadcopter's trajectory. Differential smoothing of the system can convert the drone state into the end effector state under a coupled dynamics model, but this relies on an accurate system dynamics model. This differential smoothing method, designed for highly coupled dynamics models, is highly dependent on model accuracy and is not suitable for drones with different robotic arms. Some methods decouple the mission trajectory planning of the aerial operating system into two parts: separate planning for the drone and the robotic arm, meaning their flight trajectories are independent. This approach can handle different tasks, but parameters must be manually adjusted when changing robotic arms.

[0067] In view of this, this application proposes a general, robust, and energy-efficient adaptive control method and system for multi-mission aerial operations of unmanned aerial vehicles (UAVs) with automatic arm-switching. The proposed control scheme, based on deep learning networks, aims to achieve online trajectory optimization and tracking control to complete various aerial operation tasks without manual adjustment of controller parameters. Furthermore, this scheme employs an energy-optimized trajectory planning method, combining the highly coupled dynamic model of the UAV operating system with a single rigid body model to generate the optimized trajectory of the aerial operating system. To address the precise control challenges encountered during aerial operation tasks, this application also proposes a controller based on deep neural networks, capable of classifying and accurately learning the forces and torques generated by the interaction between different robotic arms and the environment. In addition, by utilizing the forces generated by the robotic arm's motion as part of the UAV's power, energy savings can be achieved. Ultimately, this enables UAVs to switch between different robotic arms to perform multi-mission aerial operations, applicable to multi-mission aerial operations with various robotic arms.

[0068] Figure 1 This is an optional flowchart of the adaptive control method for automatic arm-switching multi-task aerial operations of unmanned aerial vehicles provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S100 to S500.

[0069] Step S100: Based on the first training set, a loss function is used to train the deep neural network to obtain the target disturbance force model; wherein, the first training set includes system flight data obtained by the UAV equipped with different robotic arms.

[0070] In step S200, the drone is used as the center of mass and the robotic arm is used as a spring. The spring force is used to simulate the disturbance force of the robotic arm to construct a center of mass spring model.

[0071] Step S300: Based on the center of mass spring model, the expected trajectory of the drone and the expected trajectory of the robotic arm are obtained by calculating the disturbance force of the robotic arm based on the initial trajectory of the robotic arm.

[0072] In step S400, when replacing the robotic arm, the disturbance torque of the robotic arm is obtained through the target disturbance dynamic model, and the UAV dynamic model is corrected based on the disturbance torque.

[0073] In step S500, based on the expected trajectory of the robotic arm and the expected trajectory of the UAV, and combined with the inverse kinematics of the robotic arm joints and the impedance control of each joint, the UAV uses the replaced robotic arm to perform the operation task during flight; wherein, the UAV flies based on the UAV dynamics model.

[0074] Steps S100 to S500 as shown in the embodiments of this application combine the dynamics module of the UAV with a single rigid body model to generate an optimized trajectory for the airborne operating system. The controller based on a deep neural network can classify and accurately learn the disturbance forces and disturbance torques generated by the interaction between different robotic arms and the environment. It has high accuracy and robustness in trajectory planning for UAVs and robotic arms, and can ultimately accurately realize the UAV to change robotic arms to perform multi-task airborne operations.

[0075] In some embodiments, step S100 may include, but is not limited to, steps S110 to S130:

[0076] Step S110: Obtain the robotic arm number, aircraft attitude, relative position between the aircraft and the end effector, and the total thrust of the aircraft to obtain the first training set.

[0077] Step S120: Combine cross-entropy loss to determine the loss function of the deep neural network model.

[0078] Step S130: The deep neural network model is trained using the first training set until the loss function converges or the number of training iterations reaches a preset threshold, thereby obtaining the target perturbation model; wherein, the target perturbation model is used to classify and fit the perturbation forces of the robotic arm.

[0079] The target disturbance force model obtained through the above step S100 can be used to classify and fit the disturbance forces of different robotic arms, and can be used for subsequent controller optimization.

[0080] In step S200 of some embodiments, the expression for the first kinematic model can be:

[0081]

[0082] The expression for the perturbation force model of the robotic arm can be:

[0083]

[0084] Where, p Q Indicates the position of the UAV's center of mass; m represents the coordinates of the operating tool; p m The position of the robotic arm's end effector is indicated by ; l represents the distance between the robotic arm's end effector and the drone's center of mass; F arm This represents the result of the disturbance force of the robotic arm; k represents the spring constant; b represents the damping coefficient of the spring; Δl represents the change in distance between the end of the robotic arm and the center of mass of the drone. ψ represents the velocity of the robotic arm's end effector relative to the drone's center of mass; ψ represents the yaw angle of the quadcopter; θ represents the angle between the straight line from the robotic arm's end effector to the drone's center of mass and the z-axis of the drone's body coordinate system.

[0085] The centroid spring model constructed through the above step S200 can be used to simulate the disturbance force of the robotic arm and optimize the trajectory.

[0086] In some embodiments, step S300, the step of obtaining the optimized expected trajectory of the UAV may include the following steps S310 to S340:

[0087] Step S310: Determine the given waypoint of the UAV end effector based on the input initial robotic arm trajectory.

[0088] Step S320: Based on the initial robotic arm trajectory and the robotic arm disturbance force model, calculate the robotic arm disturbance force.

[0089] Step S330: Based on the given waypoint, calculate the expected waypoint of the UAV using the first kinematic model based on the center-of-mass spring model.

[0090] Step S340: Based on the desired waypoint and the robotic arm disturbance force, generate the expected trajectory of the UAV using a polynomial spline function.

[0091] In some embodiments, step S300, obtaining the optimized expected trajectory of the robotic arm, may include the following steps S350 to S360:

[0092] Step S350: Determine the direction of the perturbation force of the robotic arm using the perturbation force model of the center of mass spring model.

[0093] Step S360: Based on the direction of the disturbance force of the robotic arm, generate the expected trajectory of the robotic arm through a polynomial spline function.

[0094] By performing the desired trajectory planning for the robotic arm and drone through the above step S300, the disturbance force generated by the robotic arm before and after replacement can be used as part of the aircraft's power, enabling the aircraft and robotic arm to complete their tasks efficiently and energy-savingly.

[0095] In some embodiments, step S400 may include, but is not limited to, the following steps S410 to S430:

[0096] Step S410: Determine the disturbance torque of the robotic arm based on the target disturbance force model.

[0097] Step S420: Determine the feedforward control law of the aircraft based on the nonlinear feedforward term, the reference controller feedback term, and the disturbance torque of the robotic arm.

[0098] Step S430: Based on the feedforward control law, the UAV dynamics model is corrected through the feedback control law.

[0099] By correcting the UAV dynamics model through the above step S400, the UAV controller can adaptively adjust according to the different robotic arms.

[0100] In some embodiments, step S500 may include, but is not limited to, the following steps S510 to S530.

[0101] Step S510: Determine the control error of the robotic arm joint based on the current angle and the desired angle of the robotic arm joint.

[0102] Step S520: Determine the impedance control law of the robotic arm joint based on the impedance control error of each robotic arm joint.

[0103] Step S530: Control the robotic arm to perform the operation task according to the expected trajectory and impedance control law.

[0104] Combining the steps S500 above, the drone can be equipped with different robotic arms to perform various tasks.

[0105] The following section provides a detailed description and explanation of the scheme in this application, taking the adaptive control of a quadcopter system carrying different robotic arms to perform different tasks as an example:

[0106] This application provides an adaptive control method for multi-mission aerial operations of unmanned aerial vehicles (UAVs) with automatic arm changing. This method can be applied to the multi-mission aerial operation control of UAVs with automatic arm changing. (Refer to...) Figure 2 The specific plan is as follows:

[0107] Step 1: Collect flight data of the unmanned operating system for different robotic arm movements, and train a deep neural network using an optimized loss function that combines cross-entropy loss to train a network that can classify and fit the perturbation data of different robotic arms.

[0108] Specifically, given the training data, the goal of learning is to train a neural network φ to approximate the perturbation force F of an arbitrary robotic arm. arm First, to generate training data for φ, both the quadcopter and the robotic arm used feedforward and feedback controllers to track some random trajectories. Training data The inputs are the robotic arm number, the quadcopter's attitude, the relative position between the quadcopter and the end effector, and the quadcopter's total thrust. Training sample data. The tag is Where, m q Indicates the total mass of the drone; This represents the acceleration of the quadrotor's center of mass; u FF It is a nonlinear feedforward term, u FD It is the PID (Reference Controller) feedback term. This represents the training data for the k-th robotic arm.

[0109] Therefore, the loss function of the neural network φ is defined as:

[0110]

[0111] Where, N k express The amount of data. In this example, a deep neural network (DNN) φ is used to represent the disturbance caused by the robotic arm.

[0112] Theoretically, as long as φ has enough neurons, F arm It can then be approximated by φ. However, for the same input... F arm The loss function will vary depending on the configuration of the robotic arm. To enable φ to classify different robotic arms, this application introduces another deep neural network h, and designs an improved loss function for the DNN:

[0113]

[0114] Here, the neural network h is the regularization learned by the DNN to avoid overfitting. This is the cross-entropy loss, and α≥0 is a hyperparameter of the deep neural network h. Intuitively, h is the discriminator that predicts the model index of the k-th robotic arm (outer loop maximization training), while φ is the parameter that is as close as possible to D. arm (Inner loop minimization training).

[0115] Step 2: Simplify the operation of the unmanned operating system of the drone and robotic arm as a center-of-mass spring model, simplify the drone as the center of mass and the robotic arm as a spring, and use the spring force to simulate the disturbance force of the robotic arm, thereby optimizing the trajectory.

[0116] Specifically, due to the quadcopter's insufficient power, the movement of the robotic arm first affects the quadcopter's attitude, and then changes its position. The robotic arm's movement affects the mass distribution of the unmanned operating system. The center mass of the entire system will shift in the direction of the robotic arm's movement, meaning that the z-axis of the body coordinate system will shift in that direction, and the quadcopter robot will also shift in that direction. Therefore, the robotic arm's disturbance can act as a "lever," moving the quadcopter's attitude to the desired direction. Therefore, this application proposes a simplified spring center-of-mass model, modeling the robotic arm as a spring (robotic arm) that pulls the quadcopter in its extension direction, referring to... Figure 3 The end position p of the robotic armm With respect to the position p of the quadrotor's center of mass Q The expression for the first kinematic model between them can be:

[0117]

[0118] Where ψ is the yaw angle of the quadcopter; θ is the angle between the line connecting the end of the robotic arm and the center of mass of the drone and the z-axis of the drone's body coordinate system; l is the distance between the end of the robotic arm and the center of mass of the drone; and m is the coordinate of the operating tool.

[0119] The expression for the perturbation force model of the robotic arm acting on a quadcopter by a spring is:

[0120]

[0121] Where k and b represent the spring constant and damping constant, respectively; Δl represents the change in distance between the end of the robotic arm and the center of mass of the drone. This represents the change in the position of the robotic arm's end effector relative to the drone's center of mass.

[0122] Step 3: Based on the center of mass spring model in Step 2, calculate the perturbation force of the robotic arm on the trajectory input by the unmanned operating system, and obtain the energy-optimized trajectory of the drone and the robotic arm.

[0123] Specifically, the trajectory optimization in this step can generate the trajectory of the quadcopter's hybrid robotic arm. The trajectory planning method in this application can utilize the forces and torques generated by the robotic arm as the power source for the quadcopter. First, compared with the given waypoint of the end effector, based on the position p of the robotic arm's end effector... m With respect to the position p of the quadrotor's center of mass Q The kinematic model between the two systems calculates the desired waypoint of the quadrotor. Then, the desired trajectory p of the quadrotor is generated using a polynomial spline function. Qd (t),(0 <t<t e The constraint condition for this trajectory is that the speed and acceleration of the quadcopter are both zero at the start and end positions.

[0124] To utilize the perturbation force of the robotic arm, the perturbation force exerted by the robotic arm on the quadcopter and the perturbation force on the quadcopter and the acceleration of the quadcopter cannot be in opposite directions. Assume Δl = l end -l0, then the target disturbance force model can be used to obtain the disturbance force F. arm The direction.

[0125] Therefore, if Then the expected trajectory of the robotic arm is s = [l d (t),θ d (t)] is as follows:

[0126]

[0127] if Then the expected trajectory of the robotic arm is s = [l d (t),θ d (t)] is as follows:

[0128]

[0129] Where a, d, e, and f are the parameters of the polynomial spline curve. The subscript end indicates the end state; l end Δl represents the expected distance between the end of the robotic arm and the center of mass of the drone in the final state; Δl represents the change in distance between the end of the robotic arm and the center of mass of the drone; l0 represents the expected distance between the end of the robotic arm and the center of mass of the drone in the initial state.

[0130] Step 4: Based on the target disturbance dynamic model obtained in Step 1, the disturbance torque of the robotic arm is obtained through a neural network, the quadcopter dynamic feedforward control law is calculated, and the error is corrected through the feedback control law to improve the adaptability of the unmanned operating system to cope with different tasks.

[0131] Specifically, the dynamic model of the quadcopter is as follows:

[0132]

[0133] Among them, u d It is the control input of the unmanned aerial vehicle (UAV) system; F arm It is the disturbance caused by the robotic arm, that is, the disturbing force of the robotic arm; p Q It is the location of the quadcopter's center of mass; The value represents the velocity of the quadrotor's center of mass; G is gravity; M and C are parameters of the quadrotor's dynamic model.

[0134] Tracking error of quadcopter e p for:

[0135] e p =p Q -p Qd

[0136] The control law of a quadcopter AD for:

[0137]

[0138]

[0139]

[0140] Among them, u FFIt is a nonlinear feedforward term, u FD φ is the PID feedback term, and φ is the neural network for learning the perturbation force of the robotic arm. K p ,K D and K I It is a feedback control law u FD The parameter is t; t represents time.

[0141] Generally, a standard PID controller (reference controller) incorporating a feedforward term can maintain dynamic stability under nonlinear dynamic models. However, this reference controller can only compensate for unknown disturbances through the PID term, resulting in a slow response to unmodeled disturbances. Therefore, this application improves this controller by learning different robotic arm disturbances through a neural network (DNN), referring to... Figure 4 , Figure 4 In the figure, the perturbation force of the robotic arm learned by the DNN almost coincides with the perturbation force caused by the movement of the robotic arm, indicating that the perturbation force of the robotic arm learned by the DNN can effectively compensate for the perturbation force caused by the movement of the robotic arm. The shaded area on the left represents the perturbation caused by the contraction action of the robotic arm, and the shaded area on the right represents the perturbation caused by the extension of the robotic arm. The dashed line represents the perturbation force caused by the movement of the robotic arm, and the solid line represents the perturbation force of the robotic arm learned by the DNN.

[0142] When φ approaches F infinitely arm Then the dynamic model of the quadcopter can be transformed into:

[0143]

[0144] Therefore, as long as an appropriate feedback control law u is selected... Dd parameter K P ,K D and K I The system is exponentially stable.

[0145] Step 5: Based on the robotic arm trajectory obtained in Step 3, the robotic arm performs different operational tasks through the inverse kinematics of the robotic arm joints and the impedance control of each joint.

[0146] Specifically, the impedance control error of each robotic arm joint is expressed as:

[0147]

[0148] Where, θ i θ is the current angle of joint i of the robotic arm. id The expected angle of joint i is calculated in step three from the expected trajectory s of the robotic arm.

[0149] arm joint The impedance control law is expressed as:

[0150]

[0151] Where, m d b d and k d These are the parameters of the impedance controller.

[0152] In summary, this application proposes a novel adaptive control method for automatic arm-switching in multi-mission aerial operations of unmanned aerial vehicles (UAVs), enabling automatic switching of robotic arms to perform different aerial tasks. This application simplifies the aerial operating system into a model containing springs and a center of mass; this simplified model can effectively plan and optimize trajectories for robotic arms with different masses and spatial configurations. Furthermore, the disturbances generated by the robotic arm's movement are cleverly used as part of the UAV's propulsion to conserve energy. The learning-based control law can efficiently learn and adapt online to disturbances caused by the interaction between various robotic arms and the environment. Practical experiments demonstrate that, using the method proposed in this application, the aerial operating system can automatically switch between various robotic arms to complete different flight missions.

[0153] A field execution process of an embodiment of this application can be referred to. Figures 5-10 , Figures 5-10 This illustration shows a rotary-wing unmanned aerial vehicle (UAV) using the adaptive control method of this application embodiment to attach a hook. Figure 5 and Figure 6 ), replace the robotic arm ( Figure 7 and Figure 8 ), hanging signs ( Figure 9 and Figure 10 A schematic diagram of the implementation site for the continuous operation of ) .

[0154] This drone, based on the adaptive control method described in the embodiments of this application, is equipped with, for example... Figure 5 The robotic arm 101 (which has already grasped the hook) moves to the cabinet to attach the hook 102 according to the planned route of the drone and robotic arm. The robotic arm 103, which hangs the sign, is placed on a shelf to the side, and the sign 104 is temporarily hung on the wooden ladder. After the hooking is completed, as shown... Figure 6 As shown, the drone returns to unload the hook-attaching robotic arm 101. Then refer to... Figure 7 The drone flew onto the frame on its own to install the tag-hanging robotic arm 103. Then, as... Figure 8 As shown, a drone carrying a robotic arm 103 flies onto a wooden ladder to grab a sign 104. For example... Figure 9 As shown, fly to the cabinet and hang sign 104 on hook 102. Then release sign 104 and return, as... Figure 10 As shown.

[0155] Please see Figure 11This application embodiment also provides an adaptive control system 200 for automatic arm-switching multi-mission aerial operations of unmanned aerial vehicles (UAVs), which can realize the above-mentioned adaptive control method for automatic arm-switching multi-mission aerial operations of UAVs. The system includes:

[0156] The first module 201 is used to train a deep neural network using a loss function based on a first training set to obtain a target disturbance model; wherein, the first training set includes system flight data obtained by the UAV equipped with different robotic arms.

[0157] The second module 202 is used to construct a center-of-mass spring model by taking the drone as the center of mass and the robotic arm as a spring, and simulating the disturbance force of the robotic arm with spring force.

[0158] The third module 203 is used to obtain the optimized expected trajectory of the UAV and the expected trajectory of the robotic arm based on the centroid spring model and the disturbance force of the robotic arm calculated from the initial trajectory of the robotic arm.

[0159] The fourth module 204 is used to obtain the disturbance torque of the robotic arm through the target disturbance force model when replacing the robotic arm, and to correct the UAV dynamics model based on the disturbance torque.

[0160] The fifth module 205 is used to, based on the expected trajectory of the robotic arm and the expected trajectory of the UAV, and in combination with the inverse kinematics of the robotic arm joints and the impedance control of each joint, enable the UAV to perform operational tasks using the replaced robotic arm during flight; wherein the UAV flies based on the UAV dynamics model.

[0161] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0162] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned adaptive control method for automatic arm-switching multi-task aerial operations of a UAV. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0163] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0164] Please see Figure 12 , Figure 12The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0165] The processor 301 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0166] The memory 302 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 302 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 302, and the processor 301 calls and executes the adaptive control method for automatic arm-switching multi-task aerial operations of unmanned aerial vehicles according to the embodiments of this application.

[0167] Input / output interface 303 is used to implement information input and output;

[0168] The communication interface 304 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0169] Bus 305 transmits information between various components of the device (e.g., processor 301, memory 302, input / output interface 303, and communication interface 304);

[0170] The processor 301, memory 302, input / output interface 303, and communication interface 304 are connected to each other within the device via bus 305.

[0171] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned adaptive control method for automatic arm-switching multi-task aerial operations of a UAV.

[0172] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0173] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0174] The adaptive control method and system for automatic arm-changing multi-task aerial operation of UAVs provided in this application combines the dynamics module of the UAV with a single rigid body model to generate an optimized trajectory for the aerial operating system. The controller based on a deep neural network can classify and accurately learn the disturbance forces and disturbance torques generated by the interaction between different robotic arms and the environment. It has high accuracy and robustness in trajectory planning for UAVs and robotic arms, and can ultimately accurately realize the UAV to change robotic arms for multi-task aerial operation.

[0175] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0176] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0177] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0178] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0179] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0180] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0181] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, apparatuses, or units, and may be electrical, mechanical, or other forms.

[0182] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0184] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0185] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. An adaptive control method for automatic arm-switching multi-mission aerial operations of unmanned aerial vehicles, characterized in that, include: Based on the first training set, a loss function is used to train the deep neural network to obtain the target disturbance dynamics model; wherein, the first training set includes system flight data obtained by the UAV equipped with different robotic arms; Using the drone as the center of mass and the robotic arm as a spring, the disturbance force of the robotic arm is simulated by the spring force to construct a center of mass spring model; Based on the centroid spring model, the optimized expected trajectory of the UAV and the expected trajectory of the robotic arm are obtained by calculating the disturbance force of the robotic arm based on the initial trajectory of the robotic arm. When replacing the robotic arm, the disturbance torque of the robotic arm is obtained through the target disturbance force model, and the UAV dynamics model is corrected based on the disturbance torque; Based on the expected trajectory of the robotic arm and the expected trajectory of the drone, and combined with the inverse kinematics of the robotic arm joints and the impedance control of each joint, the drone uses the replaced robotic arm to perform operational tasks during flight; wherein, the drone flies based on the drone dynamics model.

2. The method of claim 1, wherein, The step of training a deep neural network using a loss function based on the first training set to obtain a target perturbation model includes: The first training set is obtained by acquiring the robotic arm number, the aircraft attitude, the relative position between the aircraft and the end effector, and the total thrust of the aircraft. By combining cross-entropy loss, the loss function of the deep neural network model is determined; The deep neural network model is trained using the first training set until the loss function converges or the number of training iterations reaches a preset threshold, thereby obtaining the target perturbation model; wherein, the target perturbation model is used to classify and fit the perturbation forces of the robotic arm.

3. The method of claim 1, wherein, In the step of constructing a center-of-mass spring model by taking the drone as the center of mass and the robotic arm as a spring, and simulating the disturbance force of the robotic arm with spring force, the center-of-mass spring model includes a first kinematic model and a robotic arm disturbance force model, wherein: The expression for the first kinematic model is: The expression for the disturbance force model of the robotic arm is: Where, p Q Indicates the position of the UAV's center of mass; p m The position of the robotic arm's end effector is indicated by 'm'; the coordinates of the manipulator are 'l'; the distance between the robotic arm's end effector and the drone's center of mass is 'F'. arm Δl represents the result of the disturbance force of the robotic arm; k represents the spring constant; b represents the damping coefficient of the spring; Δl represents the change in distance between the end of the robotic arm and the center of mass of the drone; i represents the velocity of the end of the robotic arm relative to the center of mass of the drone; ψ represents the yaw angle of the quadcopter; θ represents the angle between the straight line from the end of the robotic arm to the center of mass of the drone and the z-axis of the drone's body coordinate system.

4. The method of claim 1, wherein, In the step of obtaining the optimized expected trajectory of the UAV and the expected trajectory of the robotic arm based on the centroid spring model and the disturbance force of the robotic arm calculated from the initial trajectory of the robotic arm, the step of obtaining the optimized expected trajectory of the UAV includes: Based on the input initial robotic arm trajectory, determine the given waypoint of the UAV end effector; Based on the initial robotic arm trajectory, the robotic arm disturbance force is calculated using the robotic arm disturbance force model. Based on the given waypoint, the expected waypoint of the UAV is calculated using the first kinematic model of the center of mass spring model; Based on the desired waypoint and the disturbance force of the robotic arm, the expected trajectory of the UAV is generated by a polynomial spline function.

5. The method of claim 1, wherein, In the step of obtaining the optimized expected trajectory of the UAV and the expected trajectory of the robotic arm based on the centroid spring model and the disturbance force of the robotic arm calculated from the initial trajectory of the robotic arm, the step of obtaining the optimized expected trajectory of the robotic arm includes: The direction of the perturbation force of the robotic arm is determined by the perturbation force model of the center of mass spring model. Based on the direction of the disturbance force of the robotic arm, the expected trajectory of the robotic arm is generated by a polynomial spline function.

6. The method of claim 1, wherein, When replacing the robotic arm, the perturbation torque of the robotic arm is obtained through the target perturbation force model, and the UAV dynamics model is corrected based on the perturbation torque, including: Based on the target disturbance force model, the disturbance torque of the robotic arm is determined; The feedforward control law of the aircraft is determined based on the nonlinear feedforward term, the reference controller feedback term, and the disturbance torque of the robotic arm. Based on the feedforward control law, the UAV dynamics model is corrected through the feedback control law.

7. The method of claim 1, wherein, The step of using the replaced robotic arm to perform operational tasks during drone flight, based on the expected trajectory of the robotic arm and the expected trajectory of the drone, combined with the inverse kinematics of the robotic arm joints and the impedance control of each joint, includes: The control error of the robotic arm joint is determined based on the current angle of the robotic arm joint and the desired angle of the robotic arm joint. The impedance control law of the robotic arm joint is determined based on the impedance control error of each of the robotic arm joints. The robotic arm is controlled to perform operational tasks based on the expected trajectory and the impedance control law.

8. An adaptive control system for automatic arm changing multi-task aerial operation of a UAV, characterized by, include: The first module is used to train a deep neural network using a loss function based on a first training set to obtain a target perturbation model; wherein, the first training set includes system flight data obtained by the UAV equipped with different robotic arms. The second module is used to construct a center-of-mass spring model by using the drone as the center of mass and the robotic arm as a spring, and simulating the disturbance force of the robotic arm with spring force. The third module is used to obtain the optimized expected trajectory of the UAV and the expected trajectory of the robotic arm based on the centroid spring model and the disturbance force of the robotic arm calculated from the initial trajectory of the robotic arm. The fourth module is used to obtain the disturbance torque of the robotic arm through the target disturbance force model when replacing the robotic arm, and to correct the UAV dynamics model based on the disturbance torque. The fifth module is used to, based on the expected trajectory of the robotic arm and the expected trajectory of the UAV, and in combination with the inverse kinematics of the robotic arm joints and the impedance control of each joint, enable the UAV to perform operational tasks using the replaced robotic arm during flight; wherein the UAV flies based on the UAV dynamics model.

9. An electronic device, comprising: Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 7.

10. A computer storage medium having stored thereon a program that is executable by a processor, the program comprising instructions for causing the processor to perform the method of any one of claims 1-9. The processor-executable program, when executed by the processor, is used to implement the method as described in any one of claims 1 to 7.

Images