Construction method, control method and system of compliant collision avoidance motion model, and medium

Abstract

The invention provides a construction method, a control method, a system and a medium of a compliant collision avoidance motion model, and the construction method comprises the steps: obtaining a motion track plan corresponding to a target object on a power transmission line in a power transmission network when a mechanical arm of an unmanned plane with an arm moves; building a multi-target reward function corresponding to the motion track planning; wherein the multi-target reward function at least comprises a motion precision reward function, a motion smoothness reward function, a motion energy consumption reward function and an additional reward function; and an improved speed potential field algorithm is adopted, a flexible obstacle avoidance function for obstacles in a motion area where the motion track planning is located is built, a near-end strategy optimization algorithm is adopted, and a corresponding flexible collision avoidance motion model for the mechanical arm to move to the target object is built based on the multi-target award function and the flexible obstacle avoidance function. The motion precision, the safety and the intelligent level of the mechanical arm of the unmanned aerial vehicle with the arm can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a construction method, a control method, a system and a medium of a compliant collision avoidance motion model. Background Art

[0002] With the widespread use of drones in aerial operations, safe and efficient flight of drones is of vital importance. In the existing technology, the robotic arms of drones with arms usually have the problem of difficulty in accurately and autonomously avoiding obstacles in complex environments. Summary of the invention

[0003] In order to solve the problems of the prior art, the present invention proposes a method for constructing a compliant collision avoidance motion model, a control method, a system and a medium, aiming to improve the motion accuracy, safety and intelligence level of the robotic arm of an arm-mounted drone.

[0004] The purpose of the present invention is achieved by adopting the following technical solutions:

[0005] In one aspect, the present invention provides a method for constructing a compliant collision avoidance motion model of a robotic arm, the method comprising:

[0006] Obtaining the motion trajectory planning corresponding to the movement of the robotic arm of the UAV with an arm to the target object on the power transmission line in the power transmission network;

[0007] Constructing a multi-objective reward function corresponding to the motion trajectory planning; wherein the multi-objective reward function at least includes: a motion accuracy reward function, a motion smoothness reward function, a motion energy consumption reward function, and an additional reward function;

[0008] Using an improved velocity potential field algorithm, a flexible obstacle avoidance function is constructed for obstacles within the motion area where the motion trajectory planning is located;

[0009] A proximal strategy optimization algorithm is adopted to construct a compliant collision avoidance motion model corresponding to the movement of the robotic arm to the target object based on the multi-objective reward function and the flexible obstacle avoidance function.

[0010] Optionally, the step of obtaining a motion trajectory plan corresponding to a movement of a mechanical arm of the drone with an arm to a target object on a power transmission line in the power transmission network includes:

[0011] Obtaining the location information of the target object and the environment information of the drone with arms;

[0012] Based on the forces and torques applied to the robotic arm during the movement, the multi-degree-of-freedom robotic arm kinematic model corresponding to the robotic arm, the position information and the environmental information, the motion trajectory planning corresponding to the movement of the robotic arm to the target object is determined.

[0013] Optionally, the obstacle includes: a dynamic obstacle, and the use of the improved velocity potential field algorithm to build a flexible obstacle avoidance function for obstacles in the motion area where the motion trajectory planning is located includes:

[0014] Using the improved velocity potential field algorithm, construct a first function representing the direction between the end effector of the robot arm and the target object, a second function representing the running speed of the dynamic obstacle, and a third function representing the tangential speed between the end effector of the robot arm and the target object;

[0015] The first function, the second function and the third function are integrated to obtain the flexible obstacle avoidance function.

[0016] Optionally, the adopting of the proximal strategy optimization algorithm to construct a compliant collision avoidance motion model corresponding to the movement of the robotic arm to the target object based on the multi-objective reward function and the flexible obstacle avoidance function includes:

[0017] Acquire a training data set; wherein the training data set includes at least: take-off data, hovering data, movement data, rotation data, grabbing data and landing data corresponding to the execution of historical basic tasks by the UAV with arms;

[0018] Using the proximal strategy optimization algorithm, based on the multi-objective reward function and the flexible obstacle avoidance function, an initial motion model is constructed;

[0019] The training data set is used to perform reinforcement learning training on the initial motion model to obtain the compliant collision avoidance motion model.

[0020] Optionally, the using the training data set to perform reinforcement learning training on the initial motion model to obtain the compliant collision avoidance motion model includes:

[0021] Using the training data set, performing reinforcement learning training on the initial motion model to obtain an intermediate motion model;

[0022] The intermediate motion model is linearized to obtain the compliant collision avoidance motion model.

[0023] Optionally, the linearizing the intermediate motion model to obtain the compliant collision avoidance motion model includes:

[0024] Performing linearization processing on the intermediate motion model to obtain a motion model to be adjusted;

[0025] Based on a simulation environment having a collision detection and risk assessment module, a performance evaluation is performed on the motion model to be adjusted to obtain an evaluation result;

[0026] Based on the collected historical collision data of the robotic arm and the evaluation result, multiple rounds of iterative training and performance evaluation are performed on the motion model to be adjusted until the compliant collision avoidance motion model is obtained whose performance evaluation index meets preset conditions.

[0027] Correspondingly, the present invention provides a system for constructing a compliant collision avoidance motion model of a robotic arm, the system comprising:

[0028] The first acquisition module is used to obtain the motion trajectory planning corresponding to the movement of the mechanical arm of the UAV with an arm to the target object on the power transmission line in the power transmission network;

[0029] A first building module is used to build a multi-objective reward function corresponding to the motion trajectory planning; wherein the multi-objective reward function at least includes: a motion accuracy reward function, a motion smoothness reward function, a motion energy consumption reward function, and an additional reward function;

[0030] The second building module is used to build a flexible obstacle avoidance function for obstacles in the motion area where the motion trajectory planning is located by using an improved velocity potential field algorithm;

[0031] A construction module is used to adopt a proximal strategy optimization algorithm to construct a compliant collision avoidance motion model corresponding to the movement of the robotic arm to the target object based on the multi-objective reward function and the flexible obstacle avoidance function.

[0032] In another aspect, the present invention provides a method for controlling an arm-mounted drone, the method comprising:

[0033] Obtain the current position of the current target on the transmission line in the power transmission network and the current environment information of the UAV with arms;

[0034] Inputting the current position and the current environment information into the compliant collision avoidance motion model constructed by any of the above methods to obtain a current motion trajectory;

[0035] Based on the current motion trajectory, the mechanical arm of the armed drone is controlled to move to the current position.

[0036] Correspondingly, the present invention provides a control system for an unmanned aerial vehicle with an arm, the system comprising:

[0037] The second acquisition module is used to obtain the current position of the current target object on the transmission line in the transmission network and the current environment information of the UAV with an arm;

[0038] An input module, used to input the current position and the current environment information into the compliant collision avoidance motion model constructed by the system as described above, to obtain a current motion trajectory;

[0039] The control module is used to control the mechanical arm of the armed drone to move to the current position based on the current motion trajectory.

[0040] In another aspect, the present application further provides a computing device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus;

[0041] The memory is used to store one or more programs;

[0042] When the one or more programs are executed by the at least one processor, the method for constructing a compliant collision avoidance motion model of a robotic arm as described in any of the above items and the control method of the drone with arms as described above are implemented.

[0043] On the other hand, the present application also provides a readable storage medium having an execution program stored thereon. When the execution program is executed, it implements the method for constructing a compliant collision avoidance motion model of a robotic arm as described in any of the above items, and the control method of the arm-equipped drone as described above.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention provides a method for constructing a compliant collision avoidance motion model of a robotic arm. First, the motion trajectory planning corresponding to the motion of the robotic arm of an unmanned aerial vehicle with an arm to a target object on a transmission line in a transmission network is obtained; then, a multi-objective reward function corresponding to the motion trajectory planning and a flexible obstacle avoidance function for obstacles in the motion area where the motion trajectory planning is located are constructed; finally, a PPO algorithm is used, combined with a multi-objective reward function and a flexible obstacle avoidance function constructed based on an improved velocity potential field algorithm, to construct a compliant collision avoidance motion model for the robotic arm to move to the target object. In this way, a highly comprehensive compliant collision avoidance motion model can be provided, that is, the compliant collision avoidance motion model can output a motion trajectory with high motion accuracy, safety, and intelligence level, so as to ensure that the compliant collision avoidance motion model is used to output the relevant motion trajectory, and can guide the high-precision, safe, and intelligent level of motion of the unmanned aerial vehicle with an arm in a complex and dynamic environment.

[0046] The present invention provides a control method for an unmanned aerial vehicle with arms, which uses a highly comprehensive compliant collision avoidance motion model, that is, a compliant collision avoidance motion model that can improve the motion accuracy, safety and intelligence level of the unmanned aerial vehicle with arms, to output the corresponding current motion trajectory, so as to guide and improve the motion accuracy, safety and intelligence level of the mechanical arm of the unmanned aerial vehicle with arms based on the current motion trajectory.

[0047] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions provided by the embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0049] Figure 1 A schematic flow chart of a method for constructing a compliant collision avoidance motion model of a robotic arm provided in an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of the relationship between the joints in the mechanical arm of the drone with arms provided in an embodiment of the present invention;

[0051] Figure 3 A schematic diagram of a process for constructing a compliant collision avoidance motion model using the construction method provided in an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of a control method of a drone with arms provided by an embodiment of the present invention;

[0053] Figure 5 A schematic diagram of the composition of a system for constructing a compliant collision avoidance motion model of a robotic arm provided by an embodiment of the present invention;

[0054] Figure 6 A schematic diagram of the composition of a control system of an unmanned aerial vehicle with arms provided in an embodiment of the present invention;

[0055] Figure 7 A schematic diagram of the composition of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0057] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0058] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the technical field of the embodiments of the present invention. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the embodiments of the present invention.

[0060] With the widespread use of drones in aerial operations, safe and efficient flight of drones is of vital importance. In the existing technology, the robotic arms of drones with arms usually have the problem of difficulty in accurately and autonomously avoiding obstacles in complex environments.

[0061] Based on the above problems, the embodiments of the present invention propose a method for constructing a compliant collision avoidance motion model, a control method, a system and a medium, aiming to improve the motion accuracy, safety and intelligence level of the robotic arm of an arm-mounted drone.

[0062] Embodiment 1:

[0063] See also Figure 1 FIG. 1 is a flow chart of a method for constructing a compliant collision avoidance motion model of a robot arm provided by the present invention. The construction method can be combined with Figure 1 The steps shown in the following instructions are:

[0064] Step 101: Obtain a motion trajectory plan corresponding to the movement of a robotic arm of an armed drone to a target object on a transmission line in a transmission network.

[0065] In some embodiments of the present invention, the drone with arms is a drone designed based on a vulture prototype, which can imitate the vulture's aerial hunting. The mechanical arm installed below it, which is similar to an eagle's claw, can grab objects outside the flight path of the drone.

[0066] In some embodiments of the present invention, the target object on the transmission line in the transmission network may be: related devices deployed on the transmission line, such as: aviation warning balls, bird repellent facilities, vibration monitoring devices, etc., or, the target object may be an obstacle on the transmission line, such as: bird nests, branches, hay, kites, balloons, plastic bags, advertising cloth, etc.

[0067] In some embodiments of the present invention, the kinematic model of the robotic arm of the armed drone, the environmental information of the armed drone, the forces and torques applied to the robotic arm during movement, and the position information of the target object can be first obtained, and then the obtained information can be input into the initial trajectory generation module, or a relevant trajectory generation algorithm can be used to generate a motion trajectory plan corresponding to the movement of the robotic arm deployed on the armed drone to the target object.

[0068] In some embodiments of the present invention, the above step 101 may be implemented by the following steps 1011 and 1012 (not shown in the figure):

[0069] Step 1011: Obtain the location information of the target object and the environment information of the drone with arms.

[0070] In some embodiments of the present invention, obtaining the location information of the target object can be achieved through two major steps: the first step is visual perception to detect and identify the target object on the transmission line; the second step is target positioning, that is, accurately positioning the target object to obtain its location information.

[0071] In some embodiments of the present invention, real-time detection and identification of targets on the transmission line can be performed based on the Vision Transformers (ViT) model. Here, environmental modeling and positioning based on Simultaneous Localization and Mapping (SLAM) can be further performed to further achieve high-precision detection and identification of targets on the transmission line.

[0072] Here, the ViT model is used to perform real-time detection and recognition of targets on the transmission line, which can be achieved through the following process:

[0073] In the first step, the target object image obtained by image acquisition can be processed, such as using filters (Gaussian filtering, mean filtering, median filtering, etc.) to reduce noise in the image to improve the quality of the target object image. The target object image can also be enhanced using histogram equalization, contrast enhancement and other technologies to improve the visual effect of the target object image, so as to provide more accurate parameter (image) support for the subsequent accurate identification of the target object in the target object image.

[0074] In the second step, target detection is performed on the image obtained in the first step (adjusted target image) based on the ViT model to achieve real-time detection and recognition of targets on the transmission line. The ViT model can be a Transformer architecture, which mainly consists of two parts: self-attention mechanism and position encoding (it should be noted that compared with traditional convolutional neural networks (CNN), the ViT model can better capture global information and details in the image through self-attention mechanism and position encoding, especially showing stronger flexibility and performance when processing high-resolution images and complex visual tasks).

[0075] Here, the Self-Attention Mechanism enables the ViT model to focus on the important parts of the input image data, thereby capturing the global and local information of the image. Its main execution logic is as follows:

[0076] First, input sequence: divide the input image (i.e. the adjusted target image) into patch vectors of fixed size (e.g. 16x16 pixels). Here, each patch vector can be flattened into a one-dimensional vector;

[0077] Secondly, linear projection: the flattened multiple patch vectors are projected into a high-dimensional vector space through linear transformation. Its execution logic is similar to mapping local features in an image into a unified representation space.

[0078] Then, calculate the attention weight: determine the dot product between each patch vector and all other patch vectors; where it can be normalized by the Softmax function to obtain the attention weight of each patch vector;

[0079] Finally, weighted summation: According to the attention weight of each patch vector, all patch vectors are weighted summed to obtain the final representation of each position (here, each patch vector is added with position information, which can be provided by position encoding). The computational complexity of the self-attention mechanism is O(N^2); where N is the length of the input sequence. This mechanism enables the ViT model to capture long-range dependencies and global information in the image.

[0080] Here, because the Transformer architecture does not have inherent position information for processing sequence data, positional encoding is used to supplement this defect, so that the ViT model can understand the positional relationship of elements in the image. The execution logic corresponding to positional encoding is as follows:

[0081] First, the encoding calculation: through the sine and cosine function calculation, a matrix with the same length and dimension as the input sequence mentioned above is generated to obtain the position encoding matrix;

[0082] Then, the encoding is added: the obtained position encoding matrix is ​​added to the input patch vector mentioned above and concatenated to preserve the position information of each patch vector.

[0083] Here, the encoding can also be updated, such as the position encoding can be updated during the training of the ViT model to adapt to different input resolutions and model requirements. Among them, the position encoding can help the ViT model better understand the relative position and order of elements in the adjusted target image, so as to better capture the structural information in the adjusted target image.

[0084] In some embodiments of the present invention, the training of the ViT model may be achieved by the following steps:

[0085] The first step is pre-training and fine-tuning: The ViT model can be pre-trained on large-scale datasets (such as the large-scale visualization database ImageNet and the ultra-large-scale dataset JFT-300M), and then fine-tuned on smaller datasets. High-resolution images are used in the pre-training stage, and the size of the patch vector is kept unchanged in the fine-tuning stage, increasing the length of the input sequence.

[0086] The second step is the classification head: a classification tag is added to the output of the Transformer encoder, and the final classification result is generated after processing by a multilayer perceptron (MLP).

[0087] Correspondingly, in some embodiments of the present invention, based on the PointNet++ multimodal neural network, combined with various sensor data (such as: visual sensors, LiDAR (Light Laser Detection and Ranging, LiDAR), force feedback), the feature extraction and fusion of multimodal data can be realized through a joint training network to provide comprehensive environmental information and perception of the target's location information, thereby achieving high-precision target position perception for the target, to ensure accurate positioning of the target's location information in complex and dynamic environments. Among them, the application of PointNet++ multimodal neural network in target positioning lies in its efficient three-dimensional point cloud feature extraction and hierarchical feature learning capabilities. The following are the technical details of the PointNet++ multimodal neural network to achieve target positioning:

[0088] First, data preprocessing: Before feature extraction, the original point cloud data (i.e., the collected data used to identify the precise positioning of the target object) must be preprocessed. Point cloud sampling is a key step in preprocessing. The point cloud sampling can be done by using the farthest point sampling (FPS) method to select a set of representative points from the original point cloud data. FPS ensures uniform coverage of the sampling points by ensuring the maximum distance distribution between the sampling points. This can greatly reduce the amount of data while retaining the main geometric features of the point cloud, laying a solid foundation for subsequent feature extraction.

[0089] Secondly, hierarchical feature learning: After data preprocessing is completed, hierarchical feature learning is performed. The first is grouping and feature extraction. Here, in each layer, the neighborhood points around the sampling point are grouped, and local areas can be formed by setting a fixed radius or a fixed number of neighborhood points. In these local areas, the PointNet method is used for feature extraction. Here, PointNet uses MLP to perform point-by-point operations on the features of each point, and then obtains the global features of the local area through the maximum pooling operation. In this way, the local geometric information of the point cloud can be effectively captured, providing the necessary parameter basis for the subsequent global feature construction.

[0090] Then, hierarchical feature transfer: After completing the local feature extraction, the hierarchical feature transfer stage is entered. In each layer, the features extracted from each local area are aggregated to form a higher-level feature representation. In this process, the layer-by-layer transfer and aggregation operations are crucial. By transferring features layer by layer, new sampling, grouping and feature extraction operations are continuously performed, and the expressive ability of features is gradually improved, thereby constructing a global feature representation. The feature transfer process ensures the fusion of features from local details to global structures, so that the model can fully understand the overall information of the point cloud.

[0091] Finally, target positioning: After the hierarchical feature learning and hierarchical transfer are completed, the target positioning stage is entered, which is performed in sequence: feature decoding, that is, using deconvolution or interpolation technology to map high-level features back to the original point cloud space, so as to obtain the feature representation of each point; through MLP, the position of each point feature is predicted to obtain the position coordinates of the target object. This step remaps the information in the high-dimensional feature space back to the three-dimensional space to achieve accurate positioning of the target object. Through the combination of feature decoding and position prediction, the PointNet++ multimodal neural network can achieve high-precision target positioning in complex three-dimensional environments, ensuring reliability and accuracy in dynamic environments.

[0092] It should be noted that through steps such as data preprocessing, hierarchical feature learning, hierarchical feature transfer, and target positioning, the PointNet++ multimodal neural network can effectively realize the comprehensive feature extraction and fusion of multimodal data, provide high-precision environment and target position perception, and provide reliable technical support for the precise positioning of target objects in complex and dynamic environments.

[0093] In some embodiments of the present invention, the environmental information of the arm-mounted drone can be obtained by collecting environmental data of the arm-mounted drone through sensors or sampling facilities deployed on the arm-mounted drone, including but not limited to: the height of the arm-mounted drone, ambient temperature, ambient humidity, geographical environment of multiple locations of the arm-mounted drone, wind speed, wind direction and other information.

[0094] Step 1012: determine the motion trajectory planning corresponding to the movement of the robotic arm to the target object based on the forces and torques applied to the robotic arm during the movement, the multi-degree-of-freedom robotic arm kinematic model corresponding to the robotic arm, the position information and the environmental information.

[0095] In some embodiments of the present invention, the multi-degree-of-freedom robotic arm kinematic model corresponding to the robotic arm of the drone with arms can be established by the Modified Denavit-Hartenberg (MDH) parameter method to ensure the continuity of the subsequent movement of the robotic arm. The corresponding execution logic may include: first, establishing a reference coordinate system for each joint in the robotic arm of the drone with arms; then, modeling based on the MDH parameter method and the corresponding reference coordinate system to establish a 6-degree-of-freedom robotic arm kinematic model, thereby ensuring the accurate positioning and coherent movement of each joint in the robotic arm.

[0096] Here, reference coordinate system can be established for each joint. Figure 2 As shown, axis i-1 (joint axis Z i-1 ), axis i (joint axis Z i ) and axis i+1 (joint axis Z i+1 ), wherein the axis i-1 is connected to the axis i through the connecting rod i-1, and the axis i is connected to the axis i+1 through the connecting rod i. The corresponding execution steps are as follows:

[0097] The first step is to determine the origin O of the connecting rod coordinate system. i That is, find the joint axis Z i and joint axis Z i+1 The common perpendicular line between the two, or the joint axis Z i With joint axis Z i+1 The intersection of the joint axis Z i With joint axis Z i+1 The intersection of the common perpendicular line and the joint axis Zi , joint axis Z i+1 The intersection point is taken as the origin O of the connecting rod coordinate system {i} i .

[0098] The second step is to stipulate The axis is along the joint axis Z i 's direction.

[0099] The third step is to stipulate The axis is along the common vertical line. If the joint axis Z i and joint axis Z i+1 If they intersect, then Axis is perpendicular to the joint axis Z i and joint axis Z i+1 The plane where it is located.

[0100] The fourth step is to Axis and Axis, determined by the right-hand rule Axis, corresponding, can continue to refer to Figure 2 shown.

[0101] Step 5: Repeat the above steps 1 to 4 until a joint reference coordinate system is established for each joint, such as the corresponding joint axis Z. i-1 On the top, establish the joint reference coordinate system, whose corresponding origin is O i-1 , and the corresponding axis, Axis and axis.

[0102] Continue to refer Figure 2 As shown, it can be seen that the coordinate system fixed on the link i-1 is through four DH parameters, namely: joint torsion angle α i-1 , connecting rod length a i-1 ( Figure 2 The connecting rod length a is also shown in i ), connecting rod offset d i and joint angle θ i , transformed to the coordinate system fixed on the connecting rod i, that is:

[0103] 1.A(a i-1 )=Along Axis, from joint axis Z i-1 Move to joint axis Z i Correspondingly, A(a i )=Along Axis, from joint axis Z i Move to joint axis Z i+1 distance.

[0104] 2.alpha(αi-1 )=Wrap Axis, from joint axis Z i-1 Rotate to joint axis Z i angle.

[0105] 3.D(d i )=Along Axis, from Axis moves to Axis distance.

[0106] 4. theta(θ i )=Along Axis, from Axis moves to The angle of the axis.

[0107] Its corresponding transformation matrix i-1 T i (α i-1 ,a i-1 ,θ i ,d i ), as shown in the following formula (1):

[0108]

[0109] Among them, R X (α i-1 )、D X (a i-1 ), R Z (θ i ) and D Z (d i ) are respectively those mentioned above: joint torsion angle, connecting rod length, joint rotation angle and connecting rod offset.

[0110] In this way, the mathematical model corresponding to the robotic arm of the UAV with arms is established through the MDH parameter method, such as the 6-DOF robotic arm kinematic model, which can further simplify the description of the relationship between the links inside the robotic arm to ensure that each joint in the robotic arm can be accurately positioned and moved.

[0111] In some embodiments of the present invention, the force applied to the robot during the movement can be obtained by using a fiber Bragg grating (FBG) sensor. Here, the FBG sensor can be installed at the end or joint of the robot, and the force can be measured by using the change in the optical propagation characteristics of the optical fiber when the force is applied. By real-time monitoring and feedback of the force, the safe and accurate operation of the robot when performing tasks is ensured, especially in the process of grasping objects by the robot. Among them, the FBG sensor writes a periodic refractive index variation structure in the optical fiber. When light passes through the FBG sensor, light of a specific wavelength is reflected and light of other wavelengths is transmitted. That is, the calculation method of the corresponding Bragg wavelength λB (reflection wavelength) is determined by the following formula (2):

[0112] λB=2neffΛ formula (2);

[0113] Among them, λB is the wavelength of the light wave reflected back under specific conditions, neff is the refractive index of the material in the center of the optical fiber, and Λ is the period of the spatial structure in the fiber grating.

[0114] It should be noted that when the optical fiber is subjected to stress or temperature changes, both λ and neff will change, resulting in a shift in the reflected wavelength. By measuring the shift ΔλB of λB, the strain on the optical fiber can be calculated. and / or temperature changes The relationship between strain and temperature change is shown in formula (3):

[0115]

[0116] In some embodiments of the present invention, the Newton-Euler method can also be used to construct the dynamic equation, based on Newton's second law and the Euler equation, to directly calculate the force and torque, which can be referred to as shown in formula (4):

[0117]

[0118] Among them, I is the moment of inertia matrix of the particle, ∝ and ω are the angular acceleration and angular velocity of the particle respectively, M is the external torque acting on the particle, F is the force, m is the mass of the object, and a is the acceleration of the object.

[0119] In this way, based on the accurately measured forces and moments at each joint of the robotic arm, the safety and accuracy of the robotic arm of the drone can be ensured in actual operation.

[0120] In this way, firstly, a 6-DOF robotic arm model is established through the MDH parameter method to ensure the accurate positioning and coherent movement of each joint in the robotic arm of the UAV with arms; secondly, FBG sensors are used as force / torque sensors to monitor the force and torque information borne by the robotic arm in real time to ensure the safety and accuracy of the robotic arm in operation; then, with the help of technologies such as the ViT model, accurate detection and environmental modeling of target objects and / or environmental information can be performed to improve the recognition accuracy of the UAV with arms for target objects and environmental information, and the PointNet++ multimodal neural network is used in combination with a variety of sensor data to achieve high-precision environmental and target position perception of the target's position information, thereby providing more accurate parameter support for motion trajectory planning, thereby improving the accuracy of the determined motion trajectory planning.

[0121] Step 102: construct a multi-objective reward function corresponding to the motion trajectory planning.

[0122] The multi-objective reward function includes at least: a motion accuracy reward function, a motion smoothness reward function, a motion energy consumption reward function, and an additional reward function.

[0123] In some embodiments of the present invention, a reinforcement learning framework can be used to convert the control strategy corresponding to the motion trajectory planning obtained in step 101 into a multi-objective optimization strategy, and correspondingly, a multi-objective reward function corresponding to the motion trajectory planning can be constructed; wherein the multi-objective optimization strategy may include: precision optimization strategy, smoothness optimization strategy, energy consumption optimization strategy, time optimization strategy, and pulse optimization strategy, etc. Correspondingly, the multi-objective reward function at least includes: a motion precision reward function corresponding to the precision optimization strategy, a motion smoothness reward function corresponding to the smoothness optimization strategy, a motion energy consumption reward function corresponding to the energy consumption optimization strategy, and other reward functions. In this way, by designing a comprehensive consideration of the multi-objective reward function, the proximal policy optimization (PPO) algorithm is further used to train the relevant model (the model can apply for the network architecture corresponding to the multi-objective reward function here) so as to obtain an optimal trajectory planning strategy that can balance the various performance indicators of the mechanical arm of the arm-carrying drone during flight.

[0124] In some embodiments of the present invention, the multi-objective reward function includes at least a motion accuracy reward function, a motion smoothness reward function, a motion energy consumption reward function, and an additional reward function, which may be respectively:

[0125] 1. Motion accuracy reward function: It is the accuracy cost function f corresponding to the motion trajectory of the robot arm at time t a (θ t ) is converted into a motion accuracy reward function Among them, the accuracy cost function f a (θ t ) can be constructed by the distance between the desired position of the end of the manipulator at time t and the current position. At the same time, the motion accuracy reward function can be referred to as shown in the following formula (5):

[0126]

[0127] Among them, ω a and σ a are the weight coefficients of the motion accuracy reward function respectively.

[0128] 2. Motion smoothness reward function: It is the smoothness function f corresponding to the motion trajectory of the robot arm at time t s (θ t ) is converted to a motion smoothness reward function Among them, the smoothness function f s (θ t ) is a function built to reduce the vibration impact caused by the movement of each joint of the robot arm at time t to achieve smooth motion trajectory. At the same time, the motion smoothness reward function can be referred to as shown in the following formula (6):

[0129]

[0130] Among them, ω s , υ , acc are the weight coefficients of the motion smoothness reward function, and and They respectively refer to the velocity and acceleration of the kth joint in the robotic arm, n is the number of joints in the robotic arm, and k refers to the kth joint in the robotic arm.

[0131] 3. Motion energy reward function: It is the energy cost function f corresponding to the motion trajectory of the robot arm at time t e (θ t ) is converted into a motion energy consumption reward function Among them, the energy cost function f e (θ t ) is used to solve the energy consumption problem related to the movement of the joints in the robot arm at time t. At the same time, the motion energy consumption reward function Please refer to the following formula (7):

[0132]

[0133] Among them, ω e represents the weight energy consumption bonus coefficient, Δθ k represents the angle change of the kth joint per unit time, τk represents the torque of the kth joint. Similarly, n is the number of joints in the robot, and k refers to the kth joint in the robot.

[0134] 4. Additional reward function: The corresponding additional reward function can be constructed using the accuracy cost function Here, if the distance between the current endpoint of the robot and the target endpoint is less than 0.005, an additional reward of 10 is given, otherwise the corresponding reward is: 10 / (1+10f a (θ t )), which is shown in the following formula (8):

[0135]

[0136] Based on the above description, the complete reward function corresponding to the motion trajectory planning, that is, the multi-objective reward function r can be expressed as shown in the following formula (9):

[0137]

[0138] Step 103: Using an improved velocity potential field algorithm, a flexible obstacle avoidance function is constructed for obstacles within the motion area where the motion trajectory planning is located.

[0139] In some embodiments of the present invention, the Improved Velocity Potential Field (IVPF) algorithm can be used to further build a flexible obstacle avoidance function for obstacles in the motion area where the motion trajectory planning is located. Here, the obstacles in the motion area where the motion trajectory planning is located include: static obstacles and dynamic obstacles.

[0140] It should be noted that the detection and identification of obstacles within the motion area where the motion trajectory planning is located can be perceived in real time by sensors deployed on the arm-mounted drone (such as lidar, camera), or can refer to the detection and identification of the position information of the target object described above. The present invention does not impose any limitation on this.

[0141] It should be noted that the IVPF algorithm is an improved algorithm proposed to solve the problems of the traditional velocity potential field (VPF) algorithm. The traditional VPF algorithm is prone to fall into local minima, cannot reach the target point, has poor dynamic obstacle avoidance capabilities, and has safety and efficiency issues. The IVPF algorithm can better handle dynamic obstacles and avoid falling into local minima by introducing directional factors, obstacle speed and tangential speed.

[0142] In some embodiments of the present invention, if the obstacles in the motion area where the motion trajectory is planned include: dynamic obstacles; correspondingly, the above step 103 can be implemented by the following steps 1031 and 1032 (not shown in the figure):

[0143] Step 1031: Use the improved velocity potential field algorithm to construct a first function that characterizes the direction between the end effector of the robot arm and the target object, a second function that characterizes the running speed of the dynamic obstacle, and a third function that characterizes the tangential velocity between the end effector of the robot arm and the target object.

[0144] In some embodiments of the present invention, the IVPF algorithm can be used to introduce direction factors into the corresponding gravitational field during the movement of the robot arm based on the information of dynamic obstacles, so as to comprehensively consider the direction between the end effector of the robot arm and the target object, thereby constructing the corresponding first function; the speed factor of the dynamic obstacle is introduced into the repulsive field corresponding to the movement of the robot arm to better avoid the dynamic obstacle, thereby constructing the corresponding second function; the tangential speed factor is introduced into the movement of the robot arm to help the robot arm get rid of the trouble of local minima, thereby constructing a third function that characterizes the tangential speed between the end effector of the robot arm and the target object.

[0145] Step 1032: The first function, the second function and the third function are integrated to obtain the flexible obstacle avoidance function.

[0146] In some embodiments of the present invention, the constructed first function, second function and third function may be integrated to form a flexible obstacle avoidance function for the robot arm to avoid the dynamic obstacle during the movement.

[0147] It should be noted that, through experimental comparison (i.e., comparison of specific experimental data), it is determined that the IVPF algorithm shows significant performance improvement in dealing with different types of obstacles, including better obstacle avoidance effect and smoother trajectory. In practical applications, the IVPF algorithm is usually used to provide a safe and efficient path planning algorithm for medical / surgical robot arms to ensure the safety of personnel and robots during human-machine collaboration. This algorithm can bring more flexible and safe possibilities for human-machine collaboration in medical / surgical scenarios, opening up new development opportunities. In the present invention, the IVPF algorithm is applied to the power transmission network.

[0148] In this way, the IVPF algorithm is used to achieve flexible obstacle avoidance of the robotic arm of the UAV when it moves to the target object on the transmission line in the transmission network, thereby providing technical support for obtaining better avoidance effects corresponding to dynamic obstacles and smoother motion trajectories.

[0149] Step 104: adopting a proximal strategy optimization algorithm, based on the multi-objective reward function and the flexible obstacle avoidance function, construct a compliant collision avoidance motion model corresponding to the movement of the robotic arm to the target object.

[0150] In some embodiments of the present invention, a network framework corresponding to the PPO algorithm can be first used to construct an initial motion model corresponding to motion trajectory planning based on a multi-objective reward function and a flexible obstacle avoidance function; and then the initial motion model is reinforced with learning using relevant training data sets to obtain a compliant collision avoidance motion model corresponding to the movement of the robotic arm to the target object (or grasping the target object).

[0151] In some embodiments of the present invention, the above step 104 may be implemented by the following steps 1041 and 1042 (not shown in the figure):

[0152] Step 1041: Obtain a training data set.

[0153] The training data set includes at least take-off data, hovering data, movement data, rotation data, grabbing data and landing data corresponding to the execution of historical basic tasks by the UAV with arms.

[0154] In some embodiments of the present invention, the arm-carrying drone can be remotely controlled to demonstrate related execution tasks to collect training data sets, which include various basic actions corresponding to the execution trajectory of the arm-carrying drone, such as: moving, rotating, grabbing, etc. These training data sets can provide a basis for subsequent model training.

[0155] Here, the training data set contains data corresponding to 100-1000 demonstration actions performed by the drone with arms.

[0156] Step 1042: Using the proximal strategy optimization algorithm, based on the multi-objective reward function and the flexible obstacle avoidance function, construct an initial motion model.

[0157] In some embodiments of the present invention, a network framework corresponding to the PPO algorithm is used to construct an initial motion model of the robotic arm, that is, a general motion model of the robotic arm, based on a multi-objective reward function and a flexible obstacle avoidance function.

[0158] Step 1043: Use the training data set to perform reinforcement learning training on the initial motion model to obtain the compliant collision avoidance motion model.

[0159] In some embodiments of the present invention, the above-mentioned training data set can be used to perform reinforcement learning training on the initial motion model of the manipulator to achieve fine-tuning and optimization of the initial motion model, so that the performance of the obtained compliant collision avoidance motion model in performing specific tasks (such as moving, rotating, grabbing, etc.) can be significantly improved through the fine-tuning and optimization process. However, this may also result in a decrease in the performance of the obtained compliant collision avoidance motion model on the original task. In order to solve this problem, the present solution adopts the following method to achieve seamless integration of new tasks into the new compliant collision avoidance motion model: Based on the fine-tuned and optimized strategy, the arm-carrying drone is usually deployed autonomously and performs tasks in a real environment to collect a large amount of new task data. Here, these new task data can be processed by "post-target re-labeling" (here, the post-target re-labeling technology can re-label the new task data in the autonomous execution process to improve the accuracy and effectiveness of the data) to form a self-improved data set. Then the new task data and the training data can be merged to serve as data for training the motion model of the next generation of the manipulator (i.e., the intermediate motion model obtained in the reinforcement learning training link). Through this iterative approach, drones can continuously learn new skills and integrate them into their own knowledge system to achieve continuous evolution.

[0160] It should be noted that if an arm-mounted drone wants to achieve autonomous data collection, that is, the "collecting a large amount of new mission data" mentioned above, two key problems need to be solved: successful detection and environmental reset.

[0161] Among them, success detection can detect whether the task is successfully completed by training a vision-based reward model. These reward models can evaluate the quality of the task performed by the arm-carrying drone in real time and provide accurate feedback for autonomous data collection. Here, by training a vision-based reward model, the arm-carrying drone can detect the completion of the task in real time. In addition, these reward models use the camera and other sensor data deployed on the arm-carrying drone to determine whether the task is successful, providing a basis for subsequent autonomous data collection and strategy optimization.

[0162] Environment reset, that is, using the "strategy pool" method to automatically reset the environment; the strategy pool is a set of strategies with overlapping start and end states, which can be "reused" to achieve environment reset and reduce manual intervention. In this way, it can ensure that the UAV with arms can quickly switch between various tasks, and automatically reset the environment after completing the task to prepare for the next round of task execution.

[0163] It should be noted that through fine-tuning, autonomous data collection and model iteration, the UAV with arms can continuously improve its capabilities and achieve the transition from a small number of demonstrations to high-level autonomous execution. This self-improvement process enables the UAV with arms to continuously adapt to new missions and environmental requirements.

[0164] In some embodiments of the present invention, the PPO algorithm is a kind of policy gradient method, which can effectively process continuous action space and learn random strategies. Its core components include: Actor network (responsible for selecting the best action according to the current state), and the output layer of the Actor network uses the Tanh activation function to generate continuous actions; Critic network (responsible for evaluating the advantage of taking actions in the current state), and the output layer of the Critic network uses a linear activation function to output a single value to represent the state value. That is to say, the above-mentioned PPO algorithm is used, based on the multi-objective reward function and the flexible obstacle avoidance function, to construct the initial motion model, which also includes the Actor network and the Critic network.

[0165] During the training of the initial motion model, the PPO algorithm will alternately update the Actor network and the Critic network to obtain the final compliant collision avoidance motion model. The training process of the initial motion model first requires sampling a batch of trajectory data from the environmental data of the arm-carrying drone, which at least includes: the state data, action data and reward data of the arm-carrying drone. Then, the improved policy gradient estimation method (Generalized Advantage Estimation, GAE) can be further used to estimate the advantage value of each state-action pair of the arm-carrying drone. Here, a special objective function LCLIP (θ) can be defined. The objective function LCLIP (θ) can limit the update amplitude of the Actor network by introducing a clipping mechanism to ensure that each training update link of the initial motion model is within the scope of the current Actor network.

[0166] In addition, the gradient descent method can be further used to optimize the parameters of the Actor network to maximize the objective function LCLIP(θ). At the same time, the mean square error loss can be used to optimize the parameters of the Critic network to predict more accurate state values.

[0167] In this way, during the training of the initial motion model, the above steps are repeated until the PPO algorithm converges. Here, the PPO algorithm maintains a balance between exploration and utilization by alternately updating the Actor network and the Critic network, and finally learns a compliant collision avoidance motion model corresponding to a stable and efficient strategy.

[0168] In some embodiments of the present invention, an attenuation (episode) mechanism can be introduced based on the PPO algorithm during the training of the initial motion model. This attenuation mechanism can automatically set the current attenuation step number as the current step number when the training accuracy reaches the set value, thereby achieving adaptive adjustment in the trajectory planning process and ensuring the trajectory planning performance output by the smooth collision avoidance motion model.

[0169] In this way, the self-improvement method based on iterative learning of small sample data (training data set) uses the PPO algorithm to construct the initial motion model based on the multi-objective reward function and the flexible obstacle avoidance function, and obtains the compliant collision avoidance motion model through gradual optimization and iterative training. In this way, the compliant collision avoidance motion model can have stronger autonomous operation ability and environmental adaptability. This can not only improve the execution efficiency and task accuracy of the compliant collision avoidance motion model, but also greatly reduce the reliance of the compliant collision avoidance motion model on manual intervention, thereby opening up new possibilities for the subsequent application of UAVs with arms based on the compliant collision avoidance motion model in complex and dynamic environments.

[0170] In some embodiments of the present invention, the above step 1043 may also be implemented by the following process:

[0171] Firstly, the training data set is used to perform reinforcement learning training on the initial motion model to obtain an intermediate motion model.

[0172] In some embodiments of the present invention, the description of obtaining the intermediate motion model may refer to the above description of step 1043, which will not be repeated here.

[0173] Then, the intermediate motion model is linearized to obtain the compliant collision avoidance motion model.

[0174] In some embodiments of the present invention, in order to ensure the stability of the robotic arm of the UAV during the obstacle avoidance process, it is necessary to linearize the joint angle trajectory in the robotic arm to eliminate the discontinuities in the corresponding motion trajectory. Here, the intermediate motion model that needs to output the relevant motion trajectory later can be linearized to output a relatively stable and smooth collision avoidance motion model. The linearization process includes but is not limited to the following methods, and can also be a random fusion of the following methods:

[0175] Method 1: trajectory interpolation, that is, using the interpolation method to smooth the joint angle trajectory of the robot arm output by the intermediate motion model to ensure the continuity of the motion trajectory of each joint in the robot arm on the time axis;

[0176] Method 2: Speed ​​and acceleration smoothing, that is, by optimizing the changes in speed and acceleration output by the intermediate motion model, the sharp changes in the motion trajectory of each joint in the robot arm are reduced to ensure the smoothness and stability of the robot arm's motion;

[0177] Method three, path optimization, is to introduce an optimization algorithm into the path planning output by the intermediate motion model to further smooth the changes in joint angles and reduce vibration and impact during movement.

[0178] In this way, the motion trajectory output by the obtained compliant collision avoidance motion model can be reduced in discontinuities, thereby improving the smoothness and stability of the subsequent running trajectory of the robot arm output by the compliant collision avoidance motion model.

[0179] Furthermore, the above process of "linearizing the intermediate motion model to obtain the compliant collision avoidance motion model" can also be implemented by the following steps:

[0180] The first step is to linearize the intermediate motion model to obtain the motion model to be adjusted.

[0181] Here, reference may also be made to the relevant description of the above step 1043, and the description of the motion model to be adjusted will not be repeated here.

[0182] The second step is to perform a performance evaluation on the motion model to be adjusted based on a simulation environment having a collision detection and risk assessment module to obtain an evaluation result.

[0183] In some embodiments of the present invention, a simulation environment with a flexible obstacle avoidance control strategy for a robotic arm can be designed based on the deep learning framework PyTorch (PyTorch is an open source deep learning framework for machine learning and deep learning) and the reinforcement learning experimental environment library (Gym). Here, the simulation environment can verify the joint simulation effect of the motion model to be adjusted based on the PPO algorithm and IVPF algorithm mentioned above by constructing a computationally efficient robotic arm kinematic model that can stably simulate various collision effects. At the same time, a collision detection and risk assessment module can be further designed in the simulation environment to evaluate the computational efficiency and simulation robustness of the motion model to be adjusted.

[0184] It should be noted that the construction of the simulation environment with collision detection and risk assessment modules may involve the following considerations:

[0185] 1. Environment initialization: A Gym-based simulation environment is created to simulate the motion and obstacle avoidance behavior of the robot arm. The simulation environment can contain the kinematic parameters of the six-axis robot arm and set the constraints of each joint in the robot arm. In addition, the kinematic model of the robot arm can be further defined, and the joint constraints and physical parameters can be set to build a typical live working scenario, which includes wires, robot arms and other obstacles. At the same time, possible collision obstacles are set for the environment, and necessary sensors are configured to detect collisions.

[0186] Reward function design: To ensure the safety and efficiency of the obstacle avoidance strategy, a reward function can be defined and the relevant weight coefficients can be set. The reward function mainly considers the following factors: collision detection (rewarding the behavior of avoiding collisions and punishing the occurrence of collisions), path smoothness (rewarding smooth trajectories and punishing violent motion changes) and target arrival (rewarding the robot arm to reach the target point).

[0187] In terms of the implementation of the PPO algorithm, the PPO algorithm is used for reinforcement learning training, and relevant hyperparameters can be set, including: learning rate, discount factor, update frequency, etc. Among them, the designed neural network structure can include: input layer, hidden layer and output layer, where the input is the current state and action, and the output is the next action. The specific steps include: building a neural network structure, setting the hyperparameters of the PPO algorithm, and implementing the training process, using the simulation environment for data collection and training.

[0188] The third step is to perform multiple rounds of iterative training and performance evaluation on the motion model to be adjusted based on the collected historical collision data of the robotic arm and the evaluation results, until the compliant collision avoidance motion model is obtained whose performance evaluation indicators meet preset conditions.

[0189] In some embodiments of the present invention, the historical collision data of the manipulator is obtained by historical collection. At the same time, based on the historical collision data and the evaluation results, the motion model to be adjusted is subjected to multiple rounds of iterative training and performance evaluation until the compliant collision avoidance motion model whose performance evaluation index meets the preset conditions is obtained. Here, the performance evaluation index at least includes: accuracy, precision, and recall rate, etc., and the present invention does not impose any limitation on this.

[0190] It should be noted that for the training and optimization of the motion model to be adjusted, a corresponding deep reinforcement learning network can be constructed. The corresponding input training data is: the current and historical state characteristics of the collision, and the output is the avoidance action of the robot arm, and the training is carried out with the reward signal of delaying the collision. The deep reinforcement learning network is trained by using the data in the collision database, and the network structure and hyperparameters are adjusted through iterative optimization to improve the detection and avoidance effect of the model. Closed-loop testing is carried out in a simulation environment to generate collision situations and observe the detection and avoidance effect of the robot arm, collect evaluation indicators such as detection success rate, avoidance reaction time, trajectory deviation, and further improve the model based on the test results.

[0191] In addition, in order to realize the collision detection and risk assessment module, a module is designed that can detect the distance between the robot arm and the obstacle in real time and evaluate the collision risk. This module calculates the distance between the end of the robot arm and the obstacle in real time through distance calculation, and designs a collision risk assessment algorithm to consider different operation objectives and environmental factors to evaluate the collision risk.

[0192] Here, in order to further improve the generalization performance of the collision avoidance algorithm, a collision monitoring device can be added during the actual operation of the robot arm. Once a collision is detected, high-speed data acquisition is immediately triggered to obtain various state data at the moment of collision. These data include: visual images, joint positions, collision forces and other multi-source sensor information. By analyzing the recorded collision data, a collision database is established, and augmented reality technology is used to generate more diverse simulated collision cases.

[0193] At the same time, during the data collection and analysis process, the simulation platform can be used to collect a large amount of process data of collisions encountered by the robot during operation. These process data include: the state before the collision, the dynamic response of the collision, and the effect after the collision. The corresponding specific steps include: collecting collision data (such as: the joint position, speed, collision force and other information of the robot), annotating data (indicating the time, location, collision object, etc. of the collision), and establishing a collision database for classification and retrieval.

[0194] It should be noted that the flexibility and robustness of avoidance can be improved by continuously iterating and optimizing the avoidance algorithm. We have built an online learning system framework to continuously collect data and optimize the algorithm during the actual operation of the robot arm. We have designed an intelligent collision cost function that fully considers multiple factors such as collision risk and operation impact, so that the avoidance strategy can output a more flexible and smooth avoidance trajectory. We have expanded the network structure, added a memory module for historical avoidance effects, and introduced a model prediction module to evaluate the long-term effect of the current avoidance decision.

[0195] In some embodiments of the present invention, future work can further enhance the network generalization ability of the compliant collision avoidance motion model, and continuously improve the intelligence level of the compliant collision avoidance motion model through continuous case learning and algorithm iterative optimization, so that the robot arm has the ability to autonomously avoid and evade complex actual environments. Here, actual experimental data show that the improved compliant collision avoidance motion model significantly improves the robot arm's adaptability to complex environments and the compliance and efficiency of collision avoidance. Therefore, through continuous optimization and iteration, the computational efficiency and robustness of the simulation model can be further enhanced, providing an important reference for the future development of live-line working robot technology.

[0196] In this way, PyTorch and Gym simulation environments are used for joint simulation to design collision detection and risk assessment modules to improve the computational efficiency and robustness of the compliant collision avoidance motion model. Thus, the compliant collision avoidance motion model can output an efficient, safe and intelligent robot arm motion strategy to make it suitable for high-precision operations in complex and dynamic environments.

[0197] Combined with the above description, firstly, the 6-DOF manipulator model corresponding to the manipulator of the UAV with arms is established using the MDH parameter method to define and transform the coordinate system and parameters, and ensure the positioning and movement coherence of each joint in the manipulator. Then, FBG sensors are used as force / torque sensors to monitor the force and torque of the manipulator by measuring the change of the optical fiber reflection wavelength, thereby ensuring the safety and accuracy of the operation of the manipulator. Secondly, the ViT model and other technologies are used to perform real-time detection and recognition of targets and model the environment in which the UAV is located, and the processing and recognition accuracy of the image of the target is improved through the self-attention mechanism and position encoding. In addition, it is further possible to further combine vision, LiDAR and force feedback data based on the PointNet++ multimodal neural network to perform high-precision environment and target position perception and achieve accurate target positioning. Then, the PPO algorithm is used for manipulator trajectory planning, and a multi-objective optimization reward function is designed to improve the accuracy, smoothness and energy efficiency of the manipulator trajectory planning, and the IVPF algorithm is used to achieve flexible obstacle avoidance. In addition, it is also possible to achieve continuous optimization and environmental adaptation of the UAV with arms through small sample iterative learning, remote control demonstration and autonomous data collection, and improve the autonomous operation capability of the UAV with arms. Joint simulation is performed in the deep learning framework PyTorch and the Gym simulation environment, and a collision detection and risk assessment module is designed to evaluate the computational efficiency and simulation robustness of the corresponding compliant collision avoidance motion model to optimize the obstacle avoidance strategy. In this way, a highly comprehensive compliant collision avoidance motion model can be provided to ensure the high precision, safety and intelligence level of the UAV with arms that uses the compliant collision avoidance motion model to output the relevant motion trajectory in complex and dynamic environments, thereby improving the motion precision, safety and intelligence level of the mechanical arm of the UAV with arms during the movement process.

[0198] The present invention provides a method for constructing a compliant collision avoidance motion model of a robotic arm. First, the motion trajectory planning corresponding to the motion of the robotic arm of an unmanned aerial vehicle with an arm to a target object on a transmission line in a transmission network is obtained; then, a multi-objective reward function corresponding to the motion trajectory planning and a flexible obstacle avoidance function for obstacles in the motion area where the motion trajectory planning is located are constructed; finally, a PPO algorithm is used, combined with a multi-objective reward function and a flexible obstacle avoidance function constructed based on an improved velocity potential field algorithm, to construct a compliant collision avoidance motion model for the motion of the robotic arm to the target object. In this way, a highly comprehensive compliant collision avoidance motion model can be provided, that is, the compliant collision avoidance motion model can output a motion trajectory with high motion accuracy, safety and intelligence level, so as to ensure that the compliant collision avoidance motion model is used to output the relevant motion trajectory, and can guide the high-precision, safe and intelligent level motion of the unmanned aerial vehicle with an arm in a complex and dynamic environment.

[0199] Correspondingly, such as Figure 3As shown, it is a schematic diagram of a process of applying the construction method provided by an embodiment of the present invention to construct a compliant collision avoidance motion model, and the corresponding execution steps are as follows:

[0200] 301. Construct a multi-DOF robotic arm kinematic model based on MDH. Here, a 6-DOF robotic arm kinematic model can be constructed. The 6-DOF robotic arm kinematic model can be used to establish a reference coordinate system for each joint in the robotic arm (determined by determining the origin of the link coordinate system, specifying the direction of the joint axis and the common vertical line, etc.), and describe the relationship between the links in the robotic arm through a transformation matrix.

[0201] 302. Strategies corresponding to force and torque sensing systems: Here, it can be an FBG sensor installed at the end or joint of the robotic arm, which monitors the force and torque of the robotic arm by measuring the change of the reflected wavelength of the optical fiber.

[0202] 303. Visual perception and target positioning of targets and / or obstacles. Here, firstly, the real-time detection and recognition of relevant objects can be realized based on technologies such as the ViT model, and the image processing and recognition accuracy can be further improved through the self-attention mechanism and position encoding. Among them, filtering and image enhancement technology can be used to improve the image quality. Then, the positioning perception technology of targets and / or obstacles based on the PointNet++ multimodal neural network combines vision, LiDAR and force feedback data to perform feature extraction and fusion of multimodal data.

[0203] 304. A corresponding model is built for the high-precision robotic arm flexible obstacle avoidance control strategy based on the PPO algorithm, that is, the trajectory planning problem is transformed into a multi-objective optimization problem using the reinforcement learning framework, the relevant network is trained by designing the reward function and the PPO algorithm, and the flexible obstacle avoidance is achieved using the IVPF algorithm.

[0204] 305. The constructed model is iteratively learned based on small samples to achieve a self-improvement method. Here, the initial training data set can be collected and the model fine-tuned through remote control demonstrations of drones with arms, and the drones with arms can be used to perform tasks in real environments and collect new task data, and the "post-target re-labeling" technology can be used to form a self-improvement data set.

[0205] 306. Simulation experiment. Use the deep learning framework PyTorch and the Gym simulation environment for joint simulation, and design collision detection and risk assessment modules to evaluate the computational efficiency and simulation robustness of the model. Among them, the reward function can be further designed to consider factors such as collision detection, path smoothness, and target arrival, and the neural network structure can be constructed to set the hyperparameters of the PPO algorithm to achieve relevant simulation and training.

[0206] in this way, Figure 3That is, a method for generating a compliant collision avoidance strategy (compliant collision avoidance motion model) for an arm-carrying UAV is provided, and the corresponding compliant collision avoidance motion model includes: creating joint nodes, collision avoidance detection nodes, collision avoidance strategy nodes, and intelligent operation planning nodes, so as to realize the construction of a compliant collision avoidance and intelligent operation knowledge graph. By obtaining a set of collision avoidance data groups and generating collision avoidance strategy information and intelligent operation planning information, technical support is provided for the subsequent autonomous collision avoidance and efficient operation of the robotic arm in a complex environment, thereby improving the safety and operation efficiency of the arm-carrying UAV in a complex environment.

[0207] Example 2

[0208] The present invention provides a control method for an unmanned aerial vehicle with an arm, see Figure 4 As shown, it is a schematic diagram of a control method of a drone with arms provided by the present invention; wherein, in combination with Figure 4 As shown, the control method of the drone with arms is described as follows:

[0209] Step 401: Obtain the current position of the current target object on the transmission line in the transmission network and the current environment information of the drone with arms.

[0210] Step 402: Input the current position and the current environment information into the compliant collision avoidance motion model constructed by the method described in any of the above embodiments to obtain the current motion trajectory.

[0211] Step 403: Based on the current motion trajectory, control the mechanical arm of the drone with arms to move to the current position.

[0212] In some embodiments of the present invention, the acquired current environmental information of the drone with arms and the current position of the current target object can be directly input into the compliant collision avoidance motion model constructed in any of the above embodiments. Here, other information such as the forces and torques exerted on the drone with arms during the flight can be further input to obtain the current motion trajectory of the drone with arms to the current position, so as to control and guide the robotic arm of the drone with arms to move to the current position based on the current motion trajectory, so as to grab the current target object, etc.

[0213] It should be noted that the description of the compliant collision avoidance motion model here can refer to the description of the above-mentioned method for constructing the compliant collision avoidance motion model of the robotic arm, and the present invention will not go into details therein.

[0214] In this way, a control method for an arm-mounted drone provided by the present invention uses a highly comprehensive compliant collision avoidance motion model, that is, a compliant collision avoidance motion model that can improve the motion accuracy, safety and intelligence level of the arm-mounted drone, to output the corresponding current motion trajectory, so as to guide and improve the motion accuracy, safety and intelligence level of the mechanical arm of the arm-mounted drone based on the current motion trajectory.

[0215] Example 3

[0216] The present invention based on the same inventive concept also provides a system 500 for constructing a compliant collision avoidance motion model of a robot arm, see Figure 5 FIG. 1 is a schematic diagram of a system for constructing a compliant collision avoidance motion model of a robot arm provided by an embodiment of the present invention, the system comprising:

[0217] The first acquisition module 501 is used to obtain the motion trajectory planning corresponding to the movement of the mechanical arm of the armed drone to the target object on the power transmission line in the power transmission network;

[0218] The first building module 502 is used to build a multi-objective reward function corresponding to the motion trajectory planning; wherein the multi-objective reward function at least includes: a motion accuracy reward function, a motion smoothness reward function, a motion energy consumption reward function, and an additional reward function;

[0219] The second building module 503 is used to build a flexible obstacle avoidance function for obstacles in the motion area where the motion trajectory planning is located by using an improved velocity potential field algorithm;

[0220] The construction module 504 is used to adopt a proximal strategy optimization algorithm to construct a compliant collision avoidance motion model corresponding to the movement of the robotic arm to the target object based on the multi-objective reward function and the flexible obstacle avoidance function.

[0221] In some embodiments of the present invention, the first acquisition module 501 specifically acquires the position information of the target object and the environmental information of the drone with arms; based on the forces and torques applied to the robotic arm during the movement, the multi-degree-of-freedom robotic arm kinematic model corresponding to the robotic arm, the position information and the environmental information, the motion trajectory planning corresponding to the movement of the robotic arm to the target object is determined.

[0222] In some embodiments of the present invention, the obstacle includes: a dynamic obstacle, a second building module 503, which is specifically used to use the improved velocity potential field algorithm to build a first function representing the direction between the end effector of the robot arm and the target object, a second function representing the running speed of the dynamic obstacle, and a third function representing the tangential velocity between the end effector of the robot arm and the target object; the first function, the second function and the third function are integrated to obtain the flexible obstacle avoidance function.

[0223] In some embodiments of the present invention, the construction module 504 is specifically used to obtain a training data set; wherein the training data set includes at least: take-off data, hovering data, movement data, rotation data, grasping data and landing data corresponding to the execution of historical basic tasks by the arm-mounted drone; using the proximal strategy optimization algorithm, based on the multi-objective reward function and the flexible obstacle avoidance function, an initial motion model is constructed; using the training data set, reinforcement learning training is performed on the initial motion model to obtain the compliant collision avoidance motion model.

[0224] In some embodiments of the present invention, the construction module 504 is specifically used to use the training data set to perform reinforcement learning training on the initial motion model to obtain an intermediate motion model; and linearize the intermediate motion model to obtain the compliant collision avoidance motion model.

[0225] In some embodiments of the present invention, the construction module 504 is specifically used to linearize the intermediate motion model to obtain the motion model to be adjusted; based on a simulation environment with a collision detection and risk assessment module, perform a performance evaluation on the motion model to be adjusted to obtain an evaluation result; based on the collected historical collision data of the robotic arm and the evaluation result, perform multiple rounds of iterative training and performance evaluation on the motion model to be adjusted until the compliant collision avoidance motion model is obtained whose performance evaluation indicators meet preset conditions.

[0226] It should be noted that the description of the corresponding embodiment of the system for constructing the compliant collision avoidance motion model of the robot arm is similar to the description of the embodiment of the method for constructing the compliant collision avoidance motion model of the robot arm, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the system embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0227] Embodiment 4:

[0228] The present invention based on the same inventive concept also provides a control system 600 of an unmanned aerial vehicle with arms, see Figure 6 FIG. 1 is a schematic diagram of a control system of an unmanned aerial vehicle with arms provided by an embodiment of the present invention, the system comprising:

[0229] The second acquisition module 601 is used to obtain the current position of the current target object on the transmission line in the transmission network and the current environment information of the UAV with an arm;

[0230] An input module 602 is used to input the current position and the current environment information into the compliant collision avoidance motion model constructed by the system as described in the above embodiment to obtain a current motion trajectory;

[0231] The control module 603 is used to control the mechanical arm of the armed drone to move to the current position based on the current motion trajectory.

[0232] It should be noted that the description of the control system of the drone with arms corresponds to the description of the control method embodiment of the drone with arms, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the system embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0233] Example 5

[0234] Based on the same inventive concept, Figure 7 As shown, the present invention also provides an electronic device, which may be a computer device, a single-chip device, an intelligent mobile device, etc. The electronic device in this embodiment may include at least one processor 701, a memory 702, a transceiver component 703, etc. The processor 701, the memory 702 and the transceiver component 703 are connected via a bus 704; the memory 702 can be used to store an execution program, and an exemplary execution program may include instructions; the processor 701 is used to execute the instructions stored in the memory. The memory 1002 can also be used to store data, which can be called and / or modified when executing instructions.

[0235] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in a storage medium to implement the corresponding method flow or corresponding functions, so as to implement the steps of the method for constructing a compliant collision avoidance motion model of the robotic arm described in any of the above embodiments, and the steps of the control method of the above-mentioned drone with arms.

[0236] Example 6

[0237] Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It is understandable that the storage medium here can include both built-in storage media in electronic devices and, of course, extended storage media supported by electronic devices. The storage medium provides a storage space that stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more execution programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor loads and executes one or more instructions stored in the storage medium to implement the steps of the method for constructing a compliant collision avoidance motion model of the robotic arm described in any one of the above embodiments, and the steps of the control method of the above-mentioned drone with arms.

[0238] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0239] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0240] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0241] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0242] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for constructing a compliant collision avoidance motion model of a robotic arm, characterized in that: The method comprises: Obtaining the motion trajectory planning corresponding to the movement of the robotic arm of the UAV with an arm to the target object on the power transmission line in the power transmission network; Constructing a multi-objective reward function corresponding to the motion trajectory planning; wherein the multi-objective reward function at least includes: a motion accuracy reward function, a motion smoothness reward function, a motion energy consumption reward function, and an additional reward function; Using an improved velocity potential field algorithm, a flexible obstacle avoidance function is constructed for obstacles within the motion area where the motion trajectory planning is located; A proximal strategy optimization algorithm is adopted to construct a compliant collision avoidance motion model corresponding to the movement of the robotic arm to the target object based on the multi-objective reward function and the flexible obstacle avoidance function.

2. The method according to claim 1, characterized in that The step of obtaining a motion trajectory plan corresponding to a movement of a mechanical arm of an armed drone to a target object on a power transmission line in a power transmission network includes: Obtaining the location information of the target object and the environment information of the drone with arms; Based on the forces and torques applied to the robotic arm during the movement, the multi-degree-of-freedom robotic arm kinematic model corresponding to the robotic arm, the position information and the environmental information, the motion trajectory planning corresponding to the movement of the robotic arm to the target object is determined.

3. The method according to claim 1 or 2, characterized in that: The obstacles include: dynamic obstacles, and the improved velocity potential field algorithm is used to build a flexible obstacle avoidance function for obstacles in the motion area where the motion trajectory planning is located, including: Using the improved velocity potential field algorithm, construct a first function representing the direction between the end effector of the robot arm and the target object, a second function representing the running speed of the dynamic obstacle, and a third function representing the tangential speed between the end effector of the robot arm and the target object; The first function, the second function and the third function are integrated to obtain the flexible obstacle avoidance function.

4. The method according to claim 1, characterized in that The proximal strategy optimization algorithm is used to construct a compliant collision avoidance motion model corresponding to the movement of the robot arm to the target object based on the multi-objective reward function and the flexible obstacle avoidance function, including: Acquire a training data set; wherein the training data set includes at least: take-off data, hovering data, movement data, rotation data, grabbing data and landing data corresponding to the execution of historical basic tasks by the UAV with arms; Using the proximal strategy optimization algorithm, based on the multi-objective reward function and the flexible obstacle avoidance function, an initial motion model is constructed; The training data set is used to perform reinforcement learning training on the initial motion model to obtain the compliant collision avoidance motion model.

5. The method according to claim 4, characterized in that The method of using the training data set to perform reinforcement learning training on the initial motion model to obtain the compliant collision avoidance motion model includes: Using the training data set, performing reinforcement learning training on the initial motion model to obtain an intermediate motion model; The intermediate motion model is linearized to obtain the compliant collision avoidance motion model.

6. The method according to claim 5, characterized in that The linearizing the intermediate motion model to obtain the compliant collision avoidance motion model includes: Performing linearization processing on the intermediate motion model to obtain a motion model to be adjusted; Based on a simulation environment having a collision detection and risk assessment module, a performance evaluation is performed on the motion model to be adjusted to obtain an evaluation result; Based on the collected historical collision data of the robotic arm and the evaluation result, multiple rounds of iterative training and performance evaluation are performed on the motion model to be adjusted until the compliant collision avoidance motion model is obtained whose performance evaluation index meets preset conditions.

7. A system for constructing a compliant collision avoidance motion model of a robotic arm, characterized in that: The system comprises: The first acquisition module is used to obtain the motion trajectory planning corresponding to the movement of the mechanical arm of the UAV with an arm to the target object on the power transmission line in the power transmission network; A first building module is used to build a multi-objective reward function corresponding to the motion trajectory planning; wherein the multi-objective reward function at least includes: a motion accuracy reward function, a motion smoothness reward function, a motion energy consumption reward function, and an additional reward function; The second building module is used to build a flexible obstacle avoidance function for obstacles in the motion area where the motion trajectory planning is located by using an improved velocity potential field algorithm; A construction module is used to adopt a proximal strategy optimization algorithm to construct a compliant collision avoidance motion model corresponding to the movement of the robotic arm to the target object based on the multi-objective reward function and the flexible obstacle avoidance function.

8. A control method for an unmanned aerial vehicle with arms, characterized in that: The method comprises: Obtain the current position of the current target on the transmission line in the power transmission network and the current environment information of the UAV with arms; Inputting the current position and the current environment information into a compliant collision avoidance motion model constructed by the method according to any one of claims 1 to 6 to obtain a current motion trajectory; Based on the current motion trajectory, the mechanical arm of the armed drone is controlled to move to the current position.

9. A control system for an unmanned aerial vehicle with arms, characterized in that: The system comprises: The second acquisition module is used to obtain the current position of the current target object on the transmission line in the transmission network and the current environment information of the UAV with an arm; An input module, used for inputting the current position and the current environment information into the compliant collision avoidance motion model constructed by the system according to claim 7 to obtain a current motion trajectory; The control module is used to control the mechanical arm of the armed drone to move to the current position based on the current motion trajectory.

10. A readable storage medium, characterized in that: An execution program is stored thereon, and when the execution program is executed, the method for constructing a compliant collision avoidance motion model of a robotic arm as described in any one of claims 1 to 6 and the control method for an arm-carrying drone as described in claim 8 are implemented.

Images