High-altitude foreign matter treatment flying mechanical arm and using method

Abstract

The invention discloses a high-altitude foreign matter treatment flying mechanical arm and a using method, the high-altitude foreign matter treatment flying mechanical arm comprises a flying unmanned aerial vehicle, and a connecting base is arranged at the bottom of the flying unmanned aerial vehicle; the bottom of the connecting base is connected with a three-degree-of-freedom mechanical arm, a mechanical gripper is arranged at the bottom of the three-degree-of-freedom mechanical arm, a visual module is arranged on the mechanical gripper, and the visual module can recognize high-altitude foreign matter and transmit a monitoring picture to an upper computer; by combining the mechanical arm with the unmanned aerial vehicle, the unmanned aerial vehicle can have the autonomous operation capacity, passive operation is converted into active operation, and therefore more complex tasks are completed.

Description

Technical Field

[0001] The invention belongs to the technical field of mechanical structures, and in particular relates to a high-altitude foreign matter processing flying mechanical arm and a use method thereof. Background Art

[0002] The handling of high-altitude foreign objects is a common problem in many industries such as electricity, construction, and transportation. Especially on high-altitude facilities such as high-voltage cables, the hanging of foreign objects may seriously affect the safe operation of the facilities. At present, high-altitude foreign objects are mainly kites, sky lanterns, woven bags, dust nets, greenhouse plastic sheets, etc., which may be blown onto high-voltage cables or other high-altitude facilities due to natural factors such as strong winds. These foreign objects may cause short circuits in power supply lines, signal interference, and even fires, posing a threat to public safety. Traditional handling methods rely on manual climbing or the use of tools such as long poles for cleaning. This method is inefficient, time-consuming, and poses a threat to the safety of operators. Or using a flamethrower to burn foreign objects may cause damage to high-voltage cables and the environment.

[0003] With the rapid development of drone technology, drones play an important role in aerial photography, site investigation, environmental modeling, rescue missions and other fields. However, the above tasks do not involve contact interaction and perception between drones and the environment. Summary of the invention

[0004] In response to the above problems, the present invention proposes a high-altitude foreign object handling flying robotic arm and a method of use. The idea is to combine the robotic arm with a drone, so that the drone can have the ability to operate autonomously, convert passive operations into active operations, and thus complete more complex tasks.

[0005] A high-altitude foreign object processing flying robotic arm includes a flying drone 1, a connecting base 2 is provided at the bottom of the flying drone 1; a three-degree-of-freedom robotic arm 3 is connected to the bottom of the connecting base 2, a robotic gripper 5 is provided at the bottom of the three-degree-of-freedom robotic arm 3, and a visual module 4 is provided on the robotic gripper 5. The visual module 4 can identify high-altitude foreign objects and transmit monitoring images to a host computer.

[0006] More specifically, the connecting base 2 includes a butterfly-shaped connecting plate 7, which is connected to the bottom of the flying drone 1 through a screw 6; the bottom of the butterfly-shaped connecting plate 7 is connected to the shoulder joint base 9 through a vertical internal threaded cylindrical pin 8, and the bottom of the shoulder joint base 9 is connected to the three-degree-of-freedom robotic arm 3.

[0007] More specifically, the three-degree-of-freedom robotic arm 3 includes a shoulder joint 10, the top of the shoulder joint 10 is connected to the shoulder joint base 9, and the bottom of the shoulder joint 10 is connected to the first link 14; the first link 14 is connected to the second link 12 through the elbow joint 11; the second link 12 is connected to the third link 15 through the wrist joint 13, and the third link 15 is the robotic gripper 5.

[0008] More specifically, the rotation angles of the shoulder joint 10 , the elbow joint 11 , and the wrist joint 13 are all between 0° and 180°.

[0009] More specifically, flying drone 1 is a DJI M350RTK drone.

[0010] More specifically, the steps of the method for using the high-altitude foreign matter handling flying mechanical arm of the present invention are as follows:

[0011] A. Manual mode;

[0012] A1. The operator remotely controls the drone 1 to fly to the high-altitude foreign object to be cleaned, ensuring that the high-altitude foreign object is within the range of the three-degree-of-freedom robotic arm 3;

[0013] A2. The operator controls the forward and reverse rotation of the shoulder joint 10, elbow joint 11 and wrist joint 13 of the three-degree-of-freedom robot arm 3 individually through the joint control buttons in the external robot arm control unit; the operator observes the position of the robot gripper 5 in real time through the image sent back by the visual module 4, and adjusts the robot gripper 5 to a suitable position;

[0014] A3. The operator controls the opening and closing of the robot claw 5 through the external robot arm control unit to grab foreign objects in the air;

[0015] B. Automatic mode;

[0016] B1. The operator presses the automatic control grab button of the external robotic arm control unit. The host computer uses the center of the bottom of the drone as the base coordinate to establish the DH coordinates of the three-degree-of-freedom robotic arm and construct the transformation matrix between the links to obtain the spatial position of the three-degree-of-freedom robotic arm. The DH coordinates of the three-degree-of-freedom robotic arm are

[0017]

[0018] where α i For z i to z i+1 Around x i The rotation angle of i Indicates along x i Axis, from z i Move to z i+1 Distance; d i From x i-1 to x i Along z i Distance in direction; θ i From x i-1 to x i Around z i The rotation angle of

[0019] The joint transformation matrix of the three-degree-of-freedom robotic arm is: The transformation matrix can be used to derive the position transformation matrix of the i-th joint relative to the world coordinate system and the position coordinates of the i-th joint. In a three-degree-of-freedom robot, if the three joint angles are known to be θ1, θ2, and θ3, in order to obtain the position P (X, Y, Z, R) of the end relative to the base coordinate system, x ,R y ,R z ), then we can establish (X,Y,Z,R x ,R y ,R z )=H(θ1,θ2,θ3);

[0020] Among them, H(θ1,θ2,θ3) is the relationship function between the robot arm joint angle and the robot arm end posture;

[0021] B2. Build a self-made high-altitude foreign object dataset, pre-process the label information of the images in the dataset, and divide the dataset into a validation set and a test set;

[0022] Specifically, we took images of various types of high-altitude foreign objects in different scenarios, collected publicly available recycled object image datasets on the Internet, and classified and sorted the collected images according to kites, sky lanterns, woven bags, dust-proof nets, greenhouse plastic sheets, etc. We used annotation tools to accurately mark the recycled objects in each image to ensure the accuracy and consistency of the labels; we divided the dataset into training set, validation set, and test set to balance the needs of model training and performance evaluation; we used the YOLOv7 target detection model to train the training set, and continuously improved the effect of the target detection model through the validation set and test set;

[0023] The visual module 4 uses the trained target detection model to identify high-altitude foreign objects and obtain the type of high-altitude foreign objects. Then, the contour of the high-altitude foreign objects is used as the feature point, and the velocity change matrix of the manipulator gripper is associated with the change rate of the contour feature point to construct the image Jacobian matrix. The differential relationship is as follows:

[0024] in represents the speed of the end effector of the robot arm, Indicates the rate of change of image feature points, is the image Jacobian matrix, which is expressed as:

[0025]

[0026] The Jacobian matrix has m rows and n columns. When using the image Jacobian to calculate the end effector velocity, two cases need to be considered: when the Jacobian matrix rank is n, is a singular matrix, If the inverse matrix of exists, then we can find:

[0027]

[0028] When the Jacobian matrix rank is not n, The inverse matrix does not exist, so let's assume Full rank, compute the least squares solution:

[0029]

[0030] in yes The pseudo-inverse of , b is an arbitrary vector of suitable dimension;

[0031] B3. The host computer constructs the objective function Where e is the image error function, which is related to the robot arm joint angle θ and time t; the Gauss-Newton iteration method is used to obtain the global minimum of the objective function, thereby continuously feedback controlling the movement of the robot arm joint until the target object is grasped.

[0032] 1. The beneficial effects of the present invention are: cleaning foreign objects at high altitudes does not require manual climbing to the foreign objects, which greatly reduces the risk of personnel working at high altitudes and avoids falling accidents caused by manual climbing. In addition, for foreign objects on high-voltage cables, there is no need to interrupt the power supply, and there is no need to burn the foreign objects, which reduces the damage to the high-voltage cables and the environment.

[0033] 2. The automatic control mode can quickly identify the target and perform the removal action, significantly improving the speed and efficiency of foreign body removal and reducing the operation time. The manual control mode allows the operator to make fine adjustments according to the actual situation, improving the ability to handle complex or irregular foreign bodies. The operator can choose one of the two modes alone or use them in combination, which has strong adaptability to complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a structural diagram of a high-altitude foreign matter handling flying mechanical arm of the present invention.

[0035] Figure 2 It is a structural diagram of the connection base of the present invention.

[0036] Figure 3 It is a structural diagram of the three-degree-of-freedom robot arm 3 of the present invention.

[0037] Figure 4 It is a schematic diagram of the structure of the UAV remote control interface of the present invention. DETAILED DESCRIPTION

[0038] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0039] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “clockwise”, “counterclockwise”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0040] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0041] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection, an electrical connection, or communication with each other; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0042] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0043] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0044] The present invention will be described in detail below in conjunction with exemplary embodiments with reference to the accompanying drawings.

[0045] A high-altitude foreign object processing flying robotic arm includes a flying drone 1, a connecting base 2 is provided at the bottom of the flying drone 1; a three-degree-of-freedom robotic arm 3 is connected to the bottom of the connecting base 2, a robotic gripper 5 is provided at the bottom of the three-degree-of-freedom robotic arm 3, and a visual module 4 is provided on the robotic gripper 5. The visual module 4 can identify high-altitude foreign objects and transmit monitoring images to a host computer.

[0046] More specifically, the connecting base 2 includes a butterfly-shaped connecting plate 7, which is connected to the bottom of the flying drone 1 through a screw 6; the bottom of the butterfly-shaped connecting plate 7 is connected to the shoulder joint base 9 through a vertical internal threaded cylindrical pin 8, and the bottom of the shoulder joint base 9 is connected to the three-degree-of-freedom robotic arm 3.

[0047] More specifically, the three-degree-of-freedom robotic arm 3 includes a shoulder joint 10, the top of the shoulder joint 10 is connected to the shoulder joint base 9, and the bottom of the shoulder joint 10 is connected to the first link 14; the first link 14 is connected to the second link 12 through the elbow joint 11; the second link 12 is connected to the third link 15 through the wrist joint 13, and the third link 15 is the robotic gripper 5.

[0048] More specifically, the rotation angles of the shoulder joint 10 , the elbow joint 11 , and the wrist joint 13 are all between 0° and 180°.

[0049] More specifically, flying drone 1 is a DJI M350RTK drone.

[0050] More specifically, the steps of the method for using the high-altitude foreign matter handling flying mechanical arm of the present invention are as follows:

[0051] A. Manual mode;

[0052] A1. The operator remotely controls the drone 1 to fly to the high-altitude foreign object to be cleaned, ensuring that the high-altitude foreign object is within the range of the three-degree-of-freedom robotic arm 3;

[0053] A2. The operator controls the forward and reverse rotation of the shoulder joint 10, elbow joint 11 and wrist joint 13 of the three-degree-of-freedom robot arm 3 individually through the joint control buttons in the external robot arm control unit; the operator observes the position of the robot gripper 5 in real time through the image sent back by the visual module 4, and adjusts the robot gripper 5 to a suitable position;

[0054] A3. The operator controls the opening and closing of the robot claw 5 through the external robot arm control unit to grab foreign objects in the air;

[0055] B. Automatic mode;

[0056] B1. The operator presses the automatic control grab button of the external robotic arm control unit. The host computer uses the center of the bottom of the drone as the base coordinate to establish the DH coordinates of the three-degree-of-freedom robotic arm and construct the transformation matrix between the links to obtain the spatial position of the three-degree-of-freedom robotic arm. The DH coordinates of the three-degree-of-freedom robotic arm are

[0057]

[0058] where α i For z i to z i+1 Around x i The rotation angle of i Indicates along x i Axis, from z i Move to z i+1 Distance; d i From x i-1 to x i Along z i Distance in direction; θ i From x i-1 to x i Around z i The rotation angle of

[0059] The joint transformation matrix of the three-degree-of-freedom robotic arm is: The transformation matrix can be used to derive the position transformation matrix of the i-th joint relative to the world coordinate system and the position coordinates of the i-th joint. In a three-degree-of-freedom robot, if the three joint angles are known to be θ1, θ2, and θ3, in order to obtain the position P (X, Y, Z, R) of the end relative to the base coordinate system, x ,R y ,R z ), then we can establish (X,Y,Z,R x ,R y ,R z )=H(θ1,θ2,θ3);

[0060] Among them, H(θ1,θ2,θ3) is the relationship function between the robot arm joint angle and the robot arm end posture;

[0061] B2. Build a self-made high-altitude foreign object dataset, pre-process the label information of the images in the dataset, and divide the dataset into a validation set and a test set;

[0062] Specifically, we took images of various types of high-altitude foreign objects in different scenarios, collected publicly available recycled object image datasets on the Internet, and classified and sorted the collected images according to kites, sky lanterns, woven bags, dust-proof nets, greenhouse plastic sheets, etc. We used annotation tools to accurately mark the recycled objects in each image to ensure the accuracy and consistency of the labels; we divided the dataset into training set, validation set, and test set to balance the needs of model training and performance evaluation; we used the YOLOv7 target detection model to train the training set, and continuously improved the effect of the target detection model through the validation set and test set;

[0063] The visual module 4 uses the trained target detection model to identify high-altitude foreign objects and obtain the type of high-altitude foreign objects. Then, the contour of the high-altitude foreign objects is used as the feature point, and the velocity change matrix of the manipulator gripper is associated with the change rate of the contour feature point to construct the image Jacobian matrix. The differential relationship is as follows:

[0064] in represents the speed of the end effector of the robot arm, Indicates the rate of change of image feature points, is the image Jacobian matrix, which is expressed as:

[0065]

[0066] The Jacobian matrix has m rows and n columns. When using the image Jacobian to calculate the end effector velocity, two cases need to be considered: when the Jacobian matrix rank is n, is a singular matrix, If the inverse matrix of exists, then we can find:

[0067]

[0068] When the Jacobian matrix rank is not n, The inverse matrix does not exist, so let's assume Full rank, compute the least squares solution:

[0069]

[0070] in yes The pseudo-inverse of , b is an arbitrary vector of suitable dimension;

[0071] B3. The host computer constructs the objective function Where e is the image error function, which is related to the robot arm joint angle θ and time t; the Gauss-Newton iteration method is used to obtain the global minimum of the objective function, thereby continuously feedback controlling the movement of the robot arm joint until the target object is grasped.

[0072] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A high-altitude foreign body handling flying robot arm, characterized by: The invention comprises a flying unmanned aerial vehicle (1), wherein a connecting base (2) is provided at the bottom of the flying unmanned aerial vehicle (1); a three-degree-of-freedom mechanical arm (3) is connected to the bottom of the connecting base (2); a mechanical gripper (5) is provided at the bottom of the three-degree-of-freedom mechanical arm (3); a visual module (4) is provided on the mechanical gripper (5); and the visual module (4) can identify foreign objects in the high altitude and transmit monitoring images to an upper computer.

2. The high-altitude foreign body handling flying mechanical arm according to claim 1, characterized in that: The connecting base (2) comprises a butterfly-shaped connecting plate (7), which is connected to the bottom of the flying drone (1) via screws (6); the bottom of the butterfly-shaped connecting plate (7) is connected to the shoulder joint base (9) via a vertical internal thread cylindrical pin (8), and the bottom of the shoulder joint base (9) is connected to the three-degree-of-freedom mechanical arm (3).

3. A high-altitude foreign body handling flying mechanical arm according to claim 2, characterized in that: The three-degree-of-freedom robotic arm (3) comprises a shoulder joint (10), the top of the shoulder joint (10) is connected to a shoulder joint base (9), and the bottom of the shoulder joint (10) is connected to a first connecting rod (14); the first connecting rod (14) is connected to a second connecting rod (12) via an elbow joint (11); the second connecting rod (12) is connected to a third connecting rod (15) via a wrist joint (13), and the third connecting rod (15) is a robotic gripper (5).

4. The high-altitude foreign body handling flying mechanical arm according to claim 3 is characterized by: The rotation angles of the shoulder joint (10), the elbow joint (11) and the wrist joint (13) are all 0° to 180°.

5. The high-altitude foreign matter handling flying mechanical arm according to claim 1 is characterized by: Flying drone 1 is DJI M350RTK.

6. According to claim 3, the method for using a high-altitude foreign matter handling flying mechanical arm comprises the following steps: A. Manual mode; A1. The operator remotely controls the drone (1) to fly to the location of the high-altitude foreign object to be cleaned, ensuring that the high-altitude foreign object is within the range of the three-degree-of-freedom robotic arm (3); A2. The operator controls the forward and reverse rotation of the shoulder joint (10), elbow joint (11) and wrist joint (13) of the three-degree-of-freedom robot arm (3) individually through the joint control buttons in the external robot arm control unit; the operator observes the position of the robot gripper (5) in real time through the image transmitted back by the visual module (4), thereby adjusting the robot gripper 5 to a suitable position; A3. The operator controls the opening and closing of the robot claw (5) through the external robot arm control unit to grab foreign objects at high altitude; B. Automatic mode; B1. The operator presses the automatic control grab button of the external robotic arm control unit. The host computer uses the center of the bottom of the drone as the base coordinate to establish the DH coordinates of the three-degree-of-freedom robotic arm and construct the transformation matrix between the links to obtain the spatial position of the three-degree-of-freedom robotic arm. The DH coordinates of the three-degree-of-freedom robotic arm are where α i For z i to z i+1 Around x i The rotation angle of i Indicates along x i Axis, from z i Move to z i+1 Distance; d i From x i-1 to x i Along z i Distance in direction; θ i From x i-1 to x i Around z i The rotation angle of The joint transformation matrix of the three-degree-of-freedom robotic arm is: The transformation matrix can be used to derive the position transformation matrix of the i-th joint relative to the world coordinate system and the position coordinates of the i-th joint. In a three-degree-of-freedom robot, if the three joint angles are known to be θ1, θ2, and θ3, in order to obtain the position P (X, Y, Z, R) of the end relative to the base coordinate system, x ,R y ,R z ), then we can establish (X,Y,Z,R x ,R y ,R z )=H(θ1,θ2,θ3); in, H(θ1,θ2,θ3) is the relationship function between the robot arm joint angle and the robot arm end posture; B2. Build a self-made high-altitude foreign object dataset, pre-process the label information of the images in the dataset, and divide the dataset into a validation set and a test set; Specifically, we took images of various types of high-altitude foreign objects in different scenes, collected available recycled object image datasets on the Internet, and classified and sorted the collected images according to kites, sky lanterns, woven bags, dust-proof nets, greenhouse plastic sheets, etc. We used annotation tools to accurately mark the recycled objects in each image to ensure the accuracy and consistency of the labels; we divided the dataset into training set, validation set, and test set to balance the needs of model training and performance evaluation; we used the YOLOv7 target detection model to train the training set, and continuously improved the effect of the target detection model through the validation set and test set; The target recognition unit uses the trained target detection model to identify high-altitude foreign objects and obtain the type of high-altitude foreign objects. Then, the contour of the high-altitude foreign object is used as the feature point, and the velocity change matrix of the manipulator gripper is associated with the change rate of the contour feature point to construct the image Jacobian matrix. The differential relationship is as follows: in represents the speed of the end effector of the robot arm, Indicates the rate of change of image feature points, is the image Jacobian matrix, which is expressed as: The Jacobian matrix has m rows and n columns. When using the image Jacobian to calculate the end effector velocity, two cases need to be considered: when the Jacobian matrix rank is n, is a singular matrix, If the inverse matrix of exists, then we can find: When the Jacobian matrix rank is not n, The inverse matrix does not exist, so let's assume Full rank, compute the least squares solution: in yes The pseudo-inverse of , b is an arbitrary vector of suitable dimension; B3. The host computer constructs the objective function Where e is the image error function, which is related to the robot arm joint angle θ and time t; the Gauss-Newton iteration method is used to obtain the global minimum of the objective function, thereby continuously feedback controlling the movement of the robot arm joint until the target object is grasped.

Images