3D Vision-Based Drone Refueling Method and System

Abstract

This invention proposes a 3D vision-based drone refueling method and system, including a drone lifting platform, a robot system, a 3D vision system, a control system, a gripper, and a fuel nozzle. The drone lifting system controls the drone to rise and fall into the field of view of the camera in the 3D vision system. The 3D vision system is responsible for collecting information about the drone's fuel tank, acquiring 3D point cloud information of the fuel tank. The control system identifies the spatial pose of the fuel tank cap and fuel tank port through the 3D image data of the fuel tank. The robot system, carrying a gripper, unscrews the drone's fuel tank cap and carries a fuel nozzle to automatically refuel the drone. This invention is a novel 3D vision-based drone refueling system and method that can automatically identify the location of the drone's fuel tank and achieve rapid automatic refueling. It is simple to operate and effectively improves the degree of automation.

Description

Technical Field

[0001] This invention belongs to the fields of automatic refueling of industrial drones and machine vision technology. More specifically, it relates to a drone refueling method and system based on 3D vision. Background Technology

[0002] Currently, refueling of gasoline-powered drones is mostly done manually. For refueling large numbers of drones in a drone nest, manual operation is difficult, and the level of automation and intelligence is low, which seriously affects the operational efficiency of drones. Traditional robot guidance can only be taught through a teach pendant, which cannot meet the needs of complex and changing working environments. When the position of the workpiece or the working environment changes, it is necessary to re-teach, which greatly limits the robot's working efficiency and working scenarios. Summary of the Invention

[0003] The purpose of this invention is to provide a 3D vision-based method and system for refueling unmanned aerial vehicles (UAVs), which can effectively solve the problems of slow refueling efficiency and inability to automatically refuel. It automatically locates the fuel tank, achieving automatic refueling and meeting the requirements of high precision and fast cycle time. The technical solution adopted in this paper is as follows:

[0004] A drone refueling method based on 3D vision includes the following steps:

[0005] Step 1: Establish four coordinate systems and establish coordinate transformation relationships between them to achieve unification between related coordinate systems. Finally, store the coordinate transformation relationships in the control system.

[0006] The coordinate systems introduced include: the oil gun end coordinate system, the gripper end coordinate system, the 3D vision system coordinate system, and the robot base coordinate system;

[0007] The robot coordinate system is based on a robot system, which includes a robot.

[0008] The gripper and oil gun are installed at the end of the robot; the 3D vision system is fixed to the end of the robot.

[0009] Coordinate transformation relationships include: 1. Transformation relationship between the 3D vision system coordinate system and the robot base coordinate system; 2. Transformation relationship between the oil gun end coordinate system and the robot base coordinate system; 3. Transformation relationship between the gripper end coordinate system and the robot base coordinate system.

[0010] Step 2: Point cloud data collection near the fuel tank, specifically including the following steps:

[0011] Step 21: The drone's lifting system is located near the docked drone;

[0012] Step 22: The control system requests a photo capture signal and sends the photo capture signal to the 3D vision system;

[0013] Step 23: The 3D vision system acquires images of the point cloud near the drone's fuel tank and sends the point cloud data to the control system.

[0014] Step 3: Obtain the spatial pose information of the fuel tank cap, which specifically includes the following steps:

[0015] Step 31: Locate the fuel tank cap image position: The control system processes the received point cloud data into a 2D image, generates the fuel tank cap ROI bounding box, and obtains the fuel tank cap ROI bounding box point cloud;

[0016] Step 32: Obtain the point cloud of the fuel tank cap, which includes the following steps:

[0017] Step 321: Based on the ROI box, map it to the three-dimensional coordinates of the point cloud data output in step 23 to obtain the point cloud information at the fuel tank cap.

[0018] Step 322: Obtain the point cloud at the fuel tank cap by point cloud clipping;

[0019] Step 33: Obtain the spatial pose of the fuel tank cap, which includes the following steps:

[0020] Step 331: In the gripper coordinate system, establish a reference template formed when the gripper clamps the fuel tank cap, and define the initial pose of the gripper origin in the reference template;

[0021] Specifically, the following steps are included:

[0022] Set up a CAD model of the fuel tank cover and import it into the simulation software to match it with the gripper model in the robot system model of the simulation software to form a reference template;

[0023] In this pose, the gripper can clamp the fuel tank cap and unscrew it. At this time, the initial pose of the gripper origin is {X, Y, Z, Rx, Ry, Rz}, where X, Y, and Z are coordinates, and Rx, Ry, and Rz are Euler angles.

[0024] Step 332: Obtain the point cloud at the fuel tank cap output in step 322 and the registration matrix T during the registration with the CAD model of the fuel tank cap, specifically including:

[0025] Register the point cloud at the fuel tank cap in step 322 with the CAD model of the fuel tank cap in step 331 to obtain the registration matrix T;

[0026] Among them, the point cloud is based on the 3D vision system coordinate system, and the registration matrix T is generated in the 3D vision system coordinate system.

[0027] Step 333: The position conversion of the fuel tank cap, specifically including the following steps:

[0028] Step 333A: Based on transformation relation one and transformation relation three, transform the registration matrix T into the gripper coordinate system to obtain matrix T1;

[0029] Where, T1=(T 转换关系三 -1 )*((T 转换关系一 )*T*(T 转换关系一 -1 ))*T 转换关系三 ;

[0030] Step 333B: Based on matrix T1 and the initial pose {X, Y, Z, Rx, Ry, Rz} of the gripper origin, by multiplying the initial pose by T1, obtain the pose {X1, Y1, Z1, Rx1, Ry1, Rz1} of the gripper origin in the gripper coordinate system each time the fuel tank cap is clamped by the gripper.

[0031] Specifically, each time the drone's docking position changes, the pose of any point X on the fuel tank cap in the gripper coordinate system is different; the position of point X on the fuel tank cap relative to the origin of the gripper remains fixed.

[0032] Each time the drone's landing position changes (with small fluctuations), the corresponding registration matrix T changes.

[0033] Step 333C: Store the pose {X1, Y1, Z1, Rx1, Ry1, Rz1} to the control system for use in step 5;

[0034] Step 4: Obtain the spatial pose information of the fuel tank opening, which specifically includes the following steps:

[0035] Step 41: Obtain the main body of the fuel tank opening point cloud, i.e., the point set {P}: Based on the point cloud at the fuel tank cap output in step 322, perform point cloud trimming and noise removal to obtain the main body of the fuel tank opening point cloud, including the fuel tank opening port point cloud port and the nearby point cloud.

[0036] Step 42: Obtain the point cloud of the fuel tank port, i.e., the point set {Pb}, which includes the following steps:

[0037] Edge extraction was performed using the rolling ball method to obtain the point cloud of the fuel tank cap edge, i.e., the point set {P}. new};

[0038] Calculate the centroid C of the point cloud at the edge of the fuel tank cap, and scale the point cloud according to the centroid to form the point cloud {Pb} at the fuel tank opening.

[0039] Step 43: Determine the orientation of the oil gun, specifically including:

[0040] Perform plane fitting on the point cloud {Pb} at the fuel tank port to obtain the plane equation A3x+B3y+C3z+D=0 for the points at the fuel tank port.

[0041] Among them, the normal vector (A3,B3,C3) of the plane equation is the port vector of the fuel tank opening;

[0042] Among them, the centroid C and the normal vector (A3,B3,C3) are both based on the coordinate system of the 3D vision system;

[0043] Step 44: Obtain the spatial pose of the fuel tank port, which includes the following steps:

[0044] Step 441: Based on transformation relation one and transformation relation two, transform the centroid C and the normal vector (A3,B3,C3) to the oil gun coordinate system, and obtain the corresponding coordinates C. new Vector (A) new B new C new );

[0045] The new transformation relationship is: (T 转换关系二 -1 )*((T 转换关系一 )*T*(T 转换关系一 -1 ))*T 转换关系二

[0046] Step 442: Determine the orientation of the oil gun, specifically including:

[0047] Judgment vector (A) new B new C new ) and default refueling position (A) in B in C in Is the angle between the two points greater than 90 degrees? If so, the direction of the oil gun is C. new , {-A new ,-B new ,-C new Otherwise, the direction of the oil gun is C. new Vector (A) new B new C new );

[0048] Among them, the default refueling position (A) in B in C in () is the set value;

[0049] Step 443: Orient the oil gun toward the storage control system for use in step 6;

[0050] Step 5: The robot system unscrews the fuel tank cap: According to the pose information input to the control system in step 333C, the robot end effector gripper is moved to the fuel tank cap, the gripper clamps the fuel tank cap, and then the fuel tank cap is unscrewed.

[0051] Step 6: Refueling the robot system: Based on the oil nozzle orientation input to the control system in step 443, move the oil nozzle to the corresponding position to refuel;

[0052] Step 7: The robot completes refueling and returns to its original position.

[0053] Preferably, step 42 specifically includes the following steps:

[0054] Step 421: Set a discrimination radius R, and draw a circle with radius R through any two points P1 and P2 in the point set {P};

[0055] If there are no other data points inside the circle, determine that points P1 and P2 are boundary points, and the line connecting them, P1P2, is the boundary line segment.

[0056] Step 422: Traverse any two points in the point set {P}, repeating step 1 to obtain the boundary point set {P}. new};

[0057] Step 423: Calculate the point set {P} new We obtain the centroid C, which is defined by spatial coordinates {x, y, z}.

[0058] Step 424: Set the point set {P} new} Scale the scale by a factor of k according to the centroid C to obtain the point cloud {Pb} of the fuel tank port; where k is a set value.

[0059] Preferably, step 31 specifically includes the following steps:

[0060] An alpha mean filter is used to filter the 2D image M of the left camera in the 3D vision system to obtain a new M1, and nonlinear correction is used to correct the image contrast.

[0061] The SURF target recognition method is used to identify the drone fuel tank cap, locate the ROI box, and obtain the ROI box coordinates.

[0062] Preferably, in step 43, the least squares method is used to fit the plane equation.

[0063] Preferably, step 7 specifically includes:

[0064] Once the robot has finished refueling, the oil nozzle stops dispensing oil according to the stop signal sent by the control system. The robot then moves the oil nozzle out of the refueling port, moves the gripper to the oil tank opening and tightens it again. After tightening, the gripper is released and the robot moves back to the initial position.

[0065] A drone refueling system based on 3D vision includes: a drone lifting system, a robot system, a 3D vision system, a control system, grippers, and a fuel nozzle;

[0066] The drone lifting system is responsible for lifting the drone, moving the drone into the field of view of the 3D vision system's camera.

[0067] The robot system is fixed to one side of the drone's lifting system and includes the robot body. It operates by carrying grippers or oil guns through information sent by the control system.

[0068] The 3D vision system includes a 3D camera, which is fixed to the end of the robot and connects to the control system via a medium. It is responsible for acquiring fuel tank image data and transmitting it back to the control system.

[0069] The control system includes a host computer that calculates the poses of the drone's fuel tank cap and fuel tank opening based on data transmitted from the 3D camera, and sends the two poses to the robot system.

[0070] The gripper and fuel nozzle are installed at the end of the robot. The gripper is responsible for unscrewing the fuel tank cap of the drone, and the fuel nozzle is responsible for inserting into the fuel tank opening to refuel.

[0071] Compared with the prior art, the advantages of the present invention are:

[0072] 1. The system acquires point cloud data of the fuel tank cap through a 3D vision system, and can automatically locate the spatial pose of the fuel tank cap and fuel tank opening, and adjust the pose of the gripper and fuel nozzle in real time to achieve automatic refueling with high recognition accuracy and high efficiency.

[0073] 2. By accurately locating and identifying the fuel tank cap and fuel tank opening of the drone, the system can adapt to various models of fuel-powered drones, improve the level of flexible robot production, and increase the competitiveness of enterprises. Attached Figure Description

[0074] Figure 1 A schematic diagram of a 3D vision-based drone refueling system;

[0075] Figure 2 Flowchart of a 3D vision-based drone refueling method;

[0076] Figure 3 This is a schematic diagram of the reference template and the initial pose of the gripper;

[0077] Figure 4 This is a diagram illustrating the default refueling position.

[0078] Among them, 1. Drone lifting system; 2. Robot system; 3. Drone; 4. Oil gun; 5. Gripper; 6. 3D vision system. Detailed Implementation

[0079] The following will describe in more detail the 3D vision-based drone refueling method and system of the present invention with reference to the schematic diagrams, which illustrate preferred embodiments of the invention. It should be understood that those skilled in the art can modify the invention described herein while still achieving its advantageous effects. Therefore, the following description should be understood as being of general knowledge to those skilled in the art and is not intended to limit the invention.

[0080] like Figure 1 The 3D vision-based drone refueling system includes: a drone lifting system 1, a robot system 2, a 3D vision system 6, a control system, a gripper 5, and a fuel nozzle 4.

[0081] The drone lifting system 1 is responsible for lifting and lowering the drone 3, and moves the drone into the field of view of the camera of the 3D vision system 6 by lifting and lowering it.

[0082] The robot system 2 is fixed to one side of the drone lifting system 1 and includes the robot body. It carries the gripper 5 or the oil gun 4 to operate by sending information through the control system.

[0083] The 3D vision system 6 includes a 3D camera, which is fixed to the end of the robot and connects to the control system via a medium. It is responsible for acquiring fuel tank image data and transmitting it back to the control system.

[0084] The control system includes a host computer that calculates the poses of the drone's fuel tank cap and fuel tank opening using data transmitted from the 3D camera, and sends the two poses to the robot system 2.

[0085] The gripper 5 and the oil gun 4 are installed at the end of the robot. The gripper 5 is responsible for unscrewing the fuel tank cap of the drone, and the oil gun 4 is responsible for inserting into the fuel tank opening to refuel.

[0086] like Figures 2-4 A drone refueling method based on 3D vision includes the following steps:

[0087] Step 1: Establish four coordinate systems and establish coordinate transformation relationships between them to achieve unification between related coordinate systems. Finally, store the coordinate transformation relationships in the control system.

[0088] The coordinate systems introduced include: the oil gun end coordinate system, the gripper end coordinate system, the 3D vision system coordinate system, and the robot base coordinate system;

[0089] The robot coordinate system is based on a robot system, which includes a robot.

[0090] The gripper and oil gun are installed at the end of the robot; the 3D vision system is fixed to the end of the robot.

[0091] Coordinate transformation relationships include: 1. Transformation relationship between the 3D vision system coordinate system and the robot base coordinate system; 2. Transformation relationship between the oil gun end coordinate system and the robot base coordinate system; 3. Transformation relationship between the gripper end coordinate system and the robot base coordinate system.

[0092] Step 2: Point cloud data collection near the fuel tank, specifically including the following steps:

[0093] Step 21: The drone's lifting system is located near the docked drone;

[0094] Step 22: The control system requests a photo capture signal, and the control system sends the photo capture signal to the 3D vision system;

[0095] Step 23: The 3D vision system acquires images of the point cloud near the drone's fuel tank and sends the point cloud data to the control system.

[0096] Step 3: Obtain the spatial pose information of the fuel tank cap, which specifically includes the following steps:

[0097] Step 31: Locate the fuel tank cap image position: The control system processes the received point cloud data into a 2D image, generates the fuel tank cap ROI frame, and obtains the fuel tank cap ROI frame point cloud.

[0098] Step 31 specifically includes the following steps:

[0099] An alpha mean filter is used to filter the 2D image M of the left camera in the 3D vision system to obtain a new M1, and nonlinear correction is used to correct the image contrast.

[0100] The SURF target recognition method is used to identify the drone fuel tank cap, locate the ROI box, and obtain the ROI box coordinates.

[0101] Step 32: Obtain the point cloud of the fuel tank cap, which includes the following steps:

[0102] Step 321: Based on the ROI box, map it to the three-dimensional coordinates of the point cloud data output in step 23 to obtain the point cloud information at the fuel tank cap.

[0103] Step 322: Obtain the point cloud at the fuel tank cap by point cloud clipping;

[0104] Step 33: Obtain the spatial pose of the fuel tank cap, which includes the following steps:

[0105] Step 331: In the gripper coordinate system, establish a reference template formed when the gripper clamps the fuel tank cap, and define the initial pose of the gripper origin in the reference template;

[0106] Specifically, the following steps are included:

[0107] Set up a CAD model of the fuel tank cover and import it into the simulation software to register it with the gripper model in the robot system model of the simulation software to form a reference template;

[0108] In this pose, the gripper can clamp the fuel tank cap and unscrew it. At this time, the initial pose of the gripper origin {X, Y, Z, Rx, Ry, Rz} is obtained by simulation software. Among them, X, Y, and Z are coordinates, and Rx, Ry, and Rz are Euler angles.

[0109] Step 332: Obtain the registration matrix T between the point cloud at the fuel tank cap output in step 322 and the CAD model of the fuel tank cap, specifically including:

[0110] Register the point cloud of the fuel tank cap output in step 322 with the CAD model of the fuel tank cap in step 331 to obtain the registration matrix T.

[0111] Among them, the point cloud is based on the 3D vision system coordinate system, and the registration matrix T is generated in the 3D vision system coordinate system.

[0112] Step 333: The position conversion of the fuel tank cap, specifically including the following steps:

[0113] Step 333A: Based on transformation relation one and transformation relation three, transform the registration matrix T into the gripper coordinate system to obtain matrix T1;

[0114] Where, T1=(T 转换关系三 -1 )*((T 转换关系一 )*T*(T 转换关系一 -1 ))*T 转换关系三 ;

[0115] Step 333B: Based on matrix T1 and the initial pose {X, Y, Z, Rx, Ry, Rz} of the gripper origin, the pose of the gripper origin in the gripper coordinate system is obtained by multiplying the initial pose by T1 to the left, thereby obtaining the pose {X1, Y1, Z1, Rx1, Ry1, Rz1} of the gripper coordinate system when the fuel tank cap is clamped.

[0116] Among them, when the drone's docking position changes each time, the pose of point X on the fuel tank cap in the gripper coordinate system is different each time; the position of point X on the fuel tank cap relative to the origin of the gripper is fixed.

[0117] When the drone's next docking location changes (with a small fluctuation relative to the previous docking location), the corresponding registration matrix T changes.

[0118] Specifically, in the next refueling step, the registration matrix T can be recalculated; the initial poses of the reference template, coordinate system, and gripper origin {X, Y, Z, Rx, Ry, Rz} are universal and do not require the creation of new reference templates, new coordinate systems, or new initial poses of the gripper origin. That is, execute steps 2, 3 (excluding step 331), and steps 4 to 7.

[0119] Step 333C: Store the pose {X1, Y1, Z1, Rx1, Ry1, Rz1} to the control system for use in step 5;

[0120] Step 4: Obtain the spatial pose information of the fuel tank opening, which specifically includes the following steps:

[0121] Step 41: Obtain the main body of the fuel tank opening point cloud, i.e., the point set {P}: Based on the point cloud at the fuel tank cap output in step 322, perform point cloud trimming and noise removal to obtain the main body of the fuel tank opening point cloud, including the fuel tank opening port point cloud port and the nearby point cloud.

[0122] Step 42: Obtain the point cloud of the fuel tank port, i.e., the point set {Pb}, which includes the following steps:

[0123] Edge extraction was performed using the rolling ball method to obtain the point cloud of the fuel tank cap edge, i.e., the point set {P}. new};

[0124] Calculate the centroid C of the point cloud at the edge of the fuel tank cap, and scale the point cloud according to the centroid to form the point cloud {Pb} at the fuel tank opening.

[0125] Step 42 specifically includes the following steps:

[0126] Step 421: Set a discrimination radius R, and draw a circle with radius R through any two points P1 and P2 in the point set {P};

[0127] If there are no other data points inside the circle, determine that points P1 and P2 are boundary points, and the line connecting them, P1P2, is the boundary line segment.

[0128] Step 422: Traverse any two points in the point set {P}, repeating step 1 to obtain the boundary point set {P}. new};

[0129] Step 423: Calculate the point set {P} new We obtain the centroid C, which is defined by spatial coordinates {x, y, z}.

[0130] Step 424: Set the point set {P} new} Scale the scale by a factor of k according to the centroid C to obtain the point cloud {Pb} of the fuel tank port; where k is a set value.

[0131] Step 43: Determine the orientation of the oil gun, specifically including:

[0132] The least squares method is used to fit the plane of the point cloud {Pb} at the fuel tank port to obtain the plane equation A3x+B3y+C3z+D=0.

[0133] Among them, the normal vector (A3,B3,C3) of the plane equation is the port vector of the fuel tank opening;

[0134] Among them, the centroid C and the normal vector (A3,B3,C3) are both based on the coordinate system of the 3D vision system;

[0135] Step 44: Obtain the spatial pose of the fuel tank port, which includes the following steps:

[0136] Step 441: Based on transformation relation one and transformation relation two, transform the centroid C and the normal vector (A3,B3,C3) to the oil gun coordinate system, and obtain the corresponding coordinates C. new Vector (A) new B new C new );

[0137] The new transformation relationship is: (T 转换关系二 -1 )*((T 转换关系一 )*T*(T 转换关系一 -1 ))*T 转换关系二 ;

[0138] Step 442: Determine the orientation of the oil gun, specifically including:

[0139] Judgment vector (A) new B new C new ) and default refueling position (A) in B in C in Is the angle between the two points greater than 90 degrees? If so, the direction of the oil gun is C. new , {-A new ,-B new ,-C new Otherwise, the direction of the oil gun is C. new Vector (A) new B new C new );

[0140] Among them, the default refueling position (A) in B in C in () is the set value;

[0141] Step 443: Orient the oil gun toward the storage control system for use in step 6;

[0142] Step 5: The robot system unscrews the fuel tank cap: According to the pose information input to the control system in step 333C, the robot end effector gripper is moved to the fuel tank cap, the gripper clamps the fuel tank cap, and then the fuel tank cap is unscrewed.

[0143] Step 6: Refueling the robot system: Based on the oil nozzle orientation input to the control system in step 443, move the oil nozzle to the corresponding position to refuel;

[0144] Step 7: The robot completes the refueling. According to the stop signal sent by the control system, the oil gun stops refueling, the robot moves the oil gun out of the refueling port, the gripper moves the oil cap to the oil tank opening and tightens it again. After tightening, the gripper is released and the robot moves to the initial position.

[0145] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the protection scope of the present invention.

Claims

1. A 3D vision-based unmanned aerial vehicle refueling method, characterized in that, The method comprises the following steps: Step 1, establish four coordinate systems, and establish the coordinate transformation relationship between the coordinate systems to realize the unity between the related coordinate systems, and finally store the coordinate transformation relationship to the control system; Among them, the introduced coordinate system includes: the oil gun end coordinate system, the gripper end coordinate system, the 3D vision system coordinate system, and the robot base coordinate system; The robot base coordinate system is based on a robot system, and the robot system comprises a robot; The gripper and the oil gun are installed at the end of the robot; the 3D vision system is fixed at the end of the robot; The coordinate transformation relationship includes: the transformation relationship one between the 3D vision system coordinate system and the robot base coordinate system, the transformation relationship two between the oil gun end coordinate system and the robot base coordinate system, and the transformation relationship three between the gripper end coordinate system and the robot base coordinate system; Step 2, oil tank nearby point cloud data acquisition, specifically comprising the following steps: Step 21, the unmanned aerial vehicle lifting system is near the parked unmanned aerial vehicle; Step 22, the control system requests a photographing signal and sends the photographing signal to the 3D vision system; Step 23, the 3D vision system collects the image of the point cloud near the unmanned aerial vehicle oil tank, and sends the point cloud data to the control system; Step 3, obtain the spatial pose information of the oil tank cover, specifically comprising the following steps: Step 31, position the oil tank cover image: the control system processes the 2D image receiving the point cloud data, generates an oil tank cover ROI frame, and obtains the oil tank cover ROI frame point cloud; Step 32: obtain the oil tank cover point cloud, specifically comprising the following steps: Step 321, based on the ROI frame, map the three-dimensional coordinates of the point cloud data output in step 23 to obtain the point cloud information at the oil tank cover; Step 322, obtain the point cloud at the oil tank cover by point cloud clipping; Step 33: obtain the spatial pose of the oil tank cover, specifically comprising the following steps: Step 331: in the gripper coordinate system, establish a reference template formed when the gripper clamps the oil tank cover, and define the initial pose of the gripper origin in the reference template; Specifically comprising the following steps: Set up the oil tank cover CAD model and import it into the simulation software to pair with the gripper model in the simulation software robot system model to form a reference template; At this pose, the gripper can clamp the oil tank cover and unscrew it, at this time, the initial pose of the gripper origin is {X, Y, Z, Rx, Ry, Rz}, wherein X, Y, Z are coordinates, Rx, Ry, Rz are Euler angles; Step 332: obtain the registration matrix T when the oil tank cover point cloud output in step 322 is registered with the oil tank cover CAD model, specifically comprising: Register the oil tank cover point cloud in step 322 with the oil tank cover CAD model in step 331 to obtain the registration matrix T; Wherein, the point cloud is based on the 3D vision system coordinate system, and the registration matrix T is generated in the 3D vision system coordinate system; Step 333: pose conversion of the oil tank cover, specifically comprising the following steps: Step 333A, based on the transformation relationship one and the transformation relationship three, convert the registration matrix T to the gripper coordinate system to obtain the matrix T1; wherein T1= (T 转换关系三 -1 ) * ( (T 转换关系一 ) * T * (T 转换关系一 -1 ) ) * T 转换关系三 ; Step 333B, based on the matrix T1 and the initial pose {X, Y, Z, Rx, Ry, Rz} of the gripper origin, the pose {X1, Y1, Z1, Rx1, Ry1, Rz1} of the gripper origin in the gripper coordinate system when the fuel tank cover is clamped by the gripper each time is obtained by left multiplying T1 by the initial pose; Wherein, when the position of the UAV changes each time, the pose of any X point on the fuel tank cover in the gripper coordinate system is different each time; the position of the X point on the fuel tank cover relative to the gripper origin is fixed; Step 333C, store the pose {X1, Y1, Z1, Rx1, Ry1, Rz1} to the control system for step 5 to call; Step 4, obtain the spatial pose information of the fuel tank port, specifically including the following steps: Step 41, obtain the fuel tank port point cloud body, i.e. the point set {P}: according to the point cloud at the fuel tank cover output in step 322, perform point cloud clipping and noise removal to obtain the fuel tank port point cloud body, including the fuel tank port point cloud port and the point cloud nearby; Step 42, obtain the fuel tank port point cloud, i.e. the point set {Pb}, specifically including the following steps: Edge extraction, edge extraction is carried out by using a rolling ball method, and an edge point cloud of the oil tank cover, i.e. a point set {P new} is obtained. Calculate the fuel tank cover edge point cloud centroid C, and scale the point cloud according to the centroid to form the fuel tank port point cloud {Pb}; Step 43, obtain the orientation of the oil gun, specifically including: Perform plane fitting on the fuel tank port point cloud {Pb} to obtain the plane equation A3 x+B3 y+C3 z+D=0 where the X, Y, Z points on the fuel tank port lie; Wherein, the normal vector (A3, B3, C3) of the plane equation, i.e. the fuel tank port vector; Wherein, the centroid C and the normal vector (A3, B3, C3) are based on the 3D vision system coordinate system; Step 44, obtain the spatial pose of the fuel tank port, specifically including the following steps: Step 441, based on the conversion relationship one and the conversion relationship two, the centroid C and the normal vector (A3, B3, C3) are converted to the oil gun coordinate system, respectively corresponding to C new , vector (A new , B new , C new ) wherein the new conversion relationship is: (T 转换关系二 -1 ) * ( (T 转换关系一 ) * T * (T 转换关系一 -1 ) ) * T 转换关系二 Step 442, determine the orientation of the oil gun, specifically including: determining whether the angle between the vector (A new ,B new ,C new ) and the default refueling pose (A in ,B in ,C in ) is greater than 90 degrees; if yes, the orientation of the nozzle is C new , {-A new ,-B new ,-C new}, otherwise, the orientation of the nozzle is C new , vector (A new ,B new ,C new ); wherein the default refueling pose (A in ,B in ,C in ) is a set value; Step 443, store the orientation of the oil gun to the control system for step 6 to call; Step 5: the robot system unscrews the fuel tank cover: according to the pose information input to the control system in step 333C, move the robot end gripper to the fuel tank cover, clamp the fuel tank cover with the gripper, and then unscrew the fuel tank cover; Step 6: the robot system refuels: according to the orientation of the oil gun input to the control system in step 443, move the oil gun to the corresponding position to refuel; Step 7: the robot completes refueling and returns to the home position. 2.The 3D vision based UAV refueling method of claim 1, wherein, Step 42 specifically includes the following steps: Step 421, set a judgment radius R, and draw a circle with a radius R through any two points P1 and P2 in the point set {P}; If there are no other data points in the circle, it is judged that the points P1 and P2 are boundary points, and the line segment P1P2 is a boundary line segment; Step 422, traverse any two points in the point set {P}, repeat step 1 to obtain the boundary point set {P new}; Step 423, calculate the point set {P new} centroid, get the centroid C, the centroid is a spatial coordinate {x, y, z}; Step 424, scaling the point set {P new} by k times according to the centroid C, and obtaining the point cloud {Pb} of the port of the tank mouth; wherein k is a set value. 3.The 3D vision based UAV refueling method of claim 1, wherein, Step 31 specifically includes the following steps: An alpha mean filter is used to filter the left camera 2D image M in the 3D vision system to obtain a new M1, and a nonlinear correction is performed for image contrast correction; A SURF target recognition method is used to recognize the UAV fuel tank cover, perform ROI frame positioning, and obtain the ROI frame coordinates.

4. The 3D vision based UAV refueling method of claim 1, wherein, In step 43, the least squares method is used to fit the plane equation.

5. The 3D vision-based UAV refueling method according to claim 1, wherein Step 7 specifically includes: The robot completes the refueling, according to the stop signal sent by the control system, the oil gun stops refueling, the robot carries the oil gun out of the refueling port, the clamping jaw moves the oil cover to the oil tank port and is retightened, the clamping jaw is loosened after being retightened, and the robot moves to the initial position.

6. A 3D vision based UAV refueling system based on the 3D vision based UAV refueling method of any of claims 1-5, characterized in that, Comprise: The unmanned aerial vehicle lifting system, the robot system, the 3D vision system, the control system, the clamping jaw and the oil gun; The unmanned aerial vehicle lifting system is responsible for the lifting of the unmanned aerial vehicle, and the unmanned aerial vehicle is moved into the camera field of view of the 3D vision system through lifting; The robot system is fixed on one side of the unmanned aerial vehicle lifting system and comprises a robot body, carries out operation through the clamping jaw or the oil gun sent by the control system; The 3D vision system comprises a 3D camera, is fixed at the tail end of the robot, is connected with the control system through a medium, is responsible for oil tank image data acquisition and returns the control system; The control system comprises a host computer, calculates the unmanned aerial vehicle oil tank cover and oil tank port pose through the data returned by the 3D camera, and sends the two poses to the robot system; The clamping jaw and the oil gun are installed at the tail end of the robot, the clamping jaw is responsible for unscrewing the unmanned aerial vehicle oil tank cover, and the oil gun is responsible for inserting the oil tank port for refueling.

Images