Automatic oil gun grabbing method based on three-dimensional vision

Abstract

The invention discloses an automatic oil gun grabbing method based on three-dimensional vision, and belongs to the technical field of automatic oil filling of fuel vehicles. According to the technical scheme, a depth camera is used for collecting video images, image processing is used for recognizing a grabbing hole and obtaining image data of the grabbing hole, a robot is controlled to change the pose, a conical connector connected with the robot is used for being connected with a fixing sleeve, and therefore the robot can accurately grab an oil gun; the grabbing hole is formed in the fixing sleeve connected with the oil gun. The method is achieved through an automatic grabbing system. The automatic grabbing system comprises processing equipment, input equipment, a displayer, a camera, a robot, a fixing assembly, a pneumatic device, an oil gun, an oil gun rack, a power source, a fixing sleeve and a conical connector. The device has the advantages of being accurate and reliable in measurement, light in grabbing structure, large in grabbing positioning range, stable, high in practicability and the like, can be widely applied to automatic refueling of gas stations, and has good economic benefits and wide market prospects.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic refueling of fuel vehicles, and in particular relates to a method for automatically grabbing a refueling gun based on three-dimensional vision. Background Art

[0002] The Chinese patent application number CN202411348693.3 discloses a vision-based grasping system and grasping method thereof. The grasping system includes a composite actuator, a grasping material collection unit, a discharge material collection unit, and an overhead camera; the composite actuator is installed at the output end of the motion module, and the housing of the composite actuator forms a clearance gap for making way for the camera; the grasping material collection unit includes a grasping station and a grasping camera, the grasping station is located within the execution range of the composite actuator, and the grasping camera is fixed above the grasping station and cooperates with the clearance gap; the discharge material collection unit includes a placement station and a placement camera, the placement station is located within the execution range of the composite actuator, and the placement camera is fixed above the placement station and cooperates with the clearance gap; the overhead camera is set within the execution range of the composite actuator and is located between the grasping station and the placement station. This invention is used for grasping at a fixed station and has no depth data, which is not suitable for occasions with uncertain positions and depth requirements.

[0003] Currently, there are some products that automatically grab gas guns, which simulate the way people grab gas guns from both sides. The whole grabbing structure is complex and heavy. In this case, a special robotic arm is needed to clamp the gas gun, which makes the whole automatic refueling equipment occupy a large area, has high equipment costs and complex control. In addition, there is a long oil pipe connected to the bottom of the gas gun. When the oil pipe is placed in a different position or receives external force, or the gas gun is placed in a different position or receives external force, the position of the gas gun on the gas gun rack and the angle between the gas gun and the ground may be different. If the gas gun is grabbed without identification, the grabbing may fail or cause equipment damage due to inaccurate grabbing.

[0004] Most existing mechanical connectors are circular connectors (or circular positioners). When the connector is inserted into the circular hole of the object to be fixed, the sliding rod is pushed by the pneumatic device to make the steel ball in the circular connector pop outward, connecting the circular connector to the object to be fixed. The shortcomings of this type of circular connector are: (1) the insertion positioning range is relatively high. The positioning range is the difference between the outer diameter of the circular connector and the inner diameter of the circular hole of the object to be fixed, and this difference is usually very small. This is because if this difference is large, after the circular connector is connected to the object, when the object to be fixed is subjected to force or when the circular connector is subjected to force, the object to be fixed will shake within this difference range, making the fixing effect worse; (2) When the circular connector is connected to the object to be fixed, the point on the surface of the steel ball is subjected to force, and the force-bearing area is small; therefore, this type of connector cannot withstand large forces, and the fixed gap will become larger and larger after being used for a period of time; (3) This type of connector is only positioned in the axial direction, and cannot be positioned in the circumferential direction; therefore, this type of connector is not suitable for occasions where the fueling gun has a large load, the placement position and angle change, and the six degrees of freedom need to be fixed when grabbing the fueling gun. Summary of the invention

[0005] In order to overcome the deficiencies of the above-mentioned prior art, the present invention provides a method for automatically grasping a refueling gun based on three-dimensional vision; based on image processing, the grasping hole on the refueling gun is accurately identified and its image data is obtained, and then the robot is controlled to change its posture so that the conical connector connected to the robot and the fixed sleeve connected to the refueling gun are connected, so that the robot can grasp the refueling gun stably and reliably.

[0006] The purpose of the present invention is achieved through the following technical solutions: A method for automatically grasping a refueling gun based on three-dimensional vision, wherein the method is implemented by an automatic grasping system, wherein the automatic grasping system comprises a processing device, an input device, a display, a camera, a robot, a fixing component, a pneumatic device, a refueling gun, a refueling gun stand, and a power supply, wherein the automatic grasping system further comprises a fixing sleeve and a conical connector, wherein the steps of the method are as follows: Step A.1, control the robot to change its posture so that the camera meets the measurement conditions; the measurement conditions include that the image contains the grabbing hole, and the distance from the camera to the refueling gun is within a preset range; the image refers to the image captured by the camera; the grabbing hole refers to the grabbing hole in the fixing sleeve; the fixing sleeve is a fixing sleeve fixedly connected to the refueling gun; the preset range is the range in which the image can contain the grabbing hole and the image processing algorithm can identify the grabbing hole; The steps of controlling the robot to change its posture so that the camera meets the measurement conditions are as follows: Controlling the robot to change its posture so that the image includes a grasping hole; The steps of controlling the robot to change its posture so that the image includes the grasping hole are as follows: Controlling the robot to change its posture so that the camera optical axis is parallel to the X axis of the robot coordinate system and the horizontal direction of the image is parallel to the XOY plane of the robot coordinate system; Controlling the robot to change its posture so that the image captured by the camera includes the grabbing hole; Controlling the robot to change its posture so that the distance from the camera to the refueling gun is within a preset range; Step A.2, acquiring image data of the grabbing hole, controlling the robot to change its posture, making the camera meet the posture condition, and obtaining the insertion movement amount; the posture condition includes that the camera meets the measurement condition, and the camera optical axis is perpendicular to the surface of the grabbing hole; the insertion movement amount is the movement amount of the robot to make the tapered connector in a position to be connected to the fixing sleeve; the surface of the grabbing hole is the surface of the fixing sleeve containing the grabbing hole; The steps of obtaining the image data of the grabbing hole, controlling the robot to change the posture so that the camera meets the posture condition, and obtaining the insertion movement amount are as follows: Acquire image data of the grabbing hole, and control the robot to change its posture so that the camera meets the posture condition; Acquire image data of the grab hole and obtain the insertion movement amount; Step A.3, automatically grab the refueling gun according to the insertion movement amount.

[0007] The steps of obtaining the image data of the grab hole according to the present invention are as follows: The processing device captures a frame of image and a frame of depth image from the video image output by the camera; the image is a color image; the camera is a depth camera; identifying a gripping hole from the image using an image processing algorithm; The image data of the grabbing hole is obtained according to the identified grabbing hole; the image data of the grabbing hole at least includes the contour of the grabbing hole, the area of ​​the grabbing hole, the center point of the grabbing hole area, and the depth value of each pixel point in the surface area of ​​the grabbing hole.

[0008] The steps of automatically grabbing the refueling gun according to the insertion movement amount are as follows: According to the insertion movement amount, controlling the robot to move so that the tapered connector is inserted into the grabbing hole; Control the piston rod in the pneumatic device to extend, so that the upper convex slider and the lower convex slider in the conical connector extend out of the conical seat in the conical connector, and the upper convex slider is connected to the upper groove in the fixed sleeve, and the lower convex slider is connected to the lower groove in the fixed sleeve, so as to realize the connection between the robot and the refueling gun; The robot is controlled to move so that the refueling gun is taken out from the refueling gun rack.

[0009] The fixing sleeve is fixedly connected to the refueling gun; the fixing sleeve has a grabbing hole; the grabbing hole is in the shape of a concave quadrangular pyramid, and the shape and angle of the concave quadrangular pyramid are the same as the shape and angle of the conical seat in the conical connector; an upper groove is provided on the upper side of the concave quadrangular pyramid, and a lower groove is provided on the lower side of the concave quadrangular pyramid; the shapes of the upper groove and the lower groove are both quadrangular pyramids; The conical connector comprises a conical seat, a sliding rod, an upper convex slider, a lower convex slider, a spring, a screw for fixing the spring, and a nut for fixing the spring; wherein, The conical seat has a quadrangular cone shape, and contains the upper convex slider, the lower convex slider, the spring, the screw for fixing the spring, and the nut for fixing the spring; The sliding rod is fixedly connected to the piston rod in the pneumatic device; the upper and lower surfaces of the front of the sliding rod are conical; The shape of the upper convex slider is a quadrangular pyramid; the shape of the lower convex slider is a quadrangular pyramid; the upper convex slider is fixedly connected to the upper end of the spring, and the lower convex slider is fixedly connected to the lower end of the spring; the shape and angle of the quadrangular pyramid above the upper convex slider are the same as the shape and angle of the upper groove; the shape and angle of the quadrangular pyramid below the lower convex slider are the same as the shape and angle of the lower groove; when the sliding rod is not inserted between the upper convex slider and the lower convex slider, the upper convex slider and the lower convex slider are both in the conical seat; when the sliding rod is inserted between the upper convex slider and the lower convex slider, the upper convex slider extends out of the conical seat and is connected with the upper groove in the fixing sleeve, and the lower convex slider extends out of the conical seat and is connected with the lower groove in the fixing sleeve; wherein, the connection refers to a tight fit connection of the four surfaces of the quadrangular pyramid.

[0010] Compared with the prior art, the significant technical features of the present invention are: The technical route is: use a depth camera to collect video images, use image processing to identify the grasping hole and obtain its image data, control the robot to change its posture so that the conical connector connected to the robot is inserted into the grasping hole, and then control the pneumatic device to insert the sliding rod in the conical connector between the upper convex slider and the lower convex slider, so that the upper convex slider extends from the conical seat and is tightly connected with the upper groove in the fixed sleeve, and at the same time, the lower convex slider extends from the conical seat and is tightly connected with the lower groove in the fixed sleeve, and the shape and angle of the four-corner pyramid of the convex slider are the same as the shape and angle of the four-corner pyramid of the groove; The technical purpose is to enable the robot to grasp the refueling gun stably and reliably.

[0011] The invention can be widely used in automatic refueling of gas stations, and has good economic benefits and broad market prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a structural schematic diagram of an automatic grabbing refueling gun system according to an embodiment of the present invention; Figure 2 This is a flow chart of automatically grabbing a refueling gun according to an embodiment of the present invention; Figure 4 A schematic diagram of making a camera meet measurement conditions according to an embodiment of the present invention; Figure 5 A schematic diagram of making a camera meet posture conditions according to an embodiment of the present invention; Figure 6 A schematic diagram of measuring surface depth data of a grab hole according to an embodiment of the present invention; Figure 7 A schematic diagram of a conical connector for a robot connected to a fixing sleeve according to an embodiment of the present invention; Figure 7-1 for Figure 7 BB cross-section view of the robot connecting the fixed sleeve with a conical connector; Figure 8 A front view of an upward convex sliding block according to an embodiment of the present invention; Figure 8-1 for Figure 8 A top view of the upper convex slider; Figure 8-2 for Figure 8-1 CC section view of the top view of the upper convex slider; Figure 8-3 for Figure 8-1 DD partial cross-sectional view of the top view of the upper convex slider; Fig. 9 A front view of a downward convex sliding block according to an embodiment of the present invention; Figure 9-1 for Fig. 9 Bottom view of the lower convex slider; Figure 9-2 for Figure 9-1 EE sectional view of the bottom view of the lower convex slider; Figure 9-3 for Figure 9-1 FF partial cross-sectional view of the bottom view of the lower convex slider; Fig.10 A front view of a conical seat according to an embodiment of the present invention; Figure 10-1 for Fig.10 Left side view (full section) of the conical seat; Figure 10-2 for Fig.10 Top view of the conical seat; Fig.11 It is a front view of the oil gun stand according to an embodiment of the present invention; Figure 11-1 for Fig.11 Left side view of the grease gun stand.

[0013] In the figure: 1, refueling island, 2, robot, 3, robot end axis, 4, camera fixing plate, 5, camera, 6, camera axis, 7, refueling gun, 8, oil gun rack, 9, fixing sleeve, 10, conical connector, 11, piston rod, 12, cylinder, 13, fixing base plate, 14, ground, 15, fixing plate, 16. Upper convex slider, 17. Upper groove in the fixed sleeve, 18. Lower convex slider, 19. Lower groove in the fixed sleeve, 20. Spring, 21. Screw for fixing the upper end of the spring, 22. Nut for fixing the upper end of the spring, 23. Screw for fixing the lower end of the spring, 24. Nut for fixing the lower end of the spring, 25. Sliding rod, 26. Conical seat, 27. Fixed block of the fuel gun, 28. Upper part of the upper convex slider, 29. Lower part of the upper convex slider, 30. Screw for fixing the upper part and the lower part of the upper convex slider, 31. Lower part of the lower convex slider, 32. Upper part of the lower convex slider, 33. Screw for fixing the upper part and the lower part of the lower convex slider, 34. Fixing strip, 35. Conical seat fixing screw, 36. Fixed surface of the oil gun holder 8, 37. Fixed groove of the oil gun holder 8.

[0014] Attached Figure 3In the XRYRZR coordinate system, the origin is OR, and the XR axis, YR axis, and ZR axis are parallel to the X axis, Y axis, and Z axis of the robot 2 coordinate system, respectively; wherein the origin OR is point RCP; the XvYvZv coordinate system is a coordinate system formed by rotating the XRYRZR coordinate system in space with the origin OR as the center, and the intersection line of the plane XRORYR in the XRYRZR coordinate system and the XvOvYv plane in the XvYvZv coordinate system is InterLine, and the origin of the XvYvZv coordinate system is Ov and point Ov is also point RCP ; a is the angle between the XR axis and the intersection line InterLine, b is the angle between the ZR axis and the Zv axis, and c is the angle between the intersection line InterLine and the Xv axis; when the horizontal direction of the image captured by the camera 5 is parallel to the XOY plane and the camera optical axis 6 is parallel to the X axis, the point RCP (x0, y0, z0, a0, b0, c0) changes to RCP (x1, y1, z1, a1, b1, c1), and the point OP (x0, y0, OPz0) changes to OP (x1, y1, OPz1); where a1=b1=c1=0; Attached Figure 4 In the figure, when the robot 2 is translated and the point GP is located at the center point ImgC of the image, the point RCP (x2, y2, z2, a2, b2, c2) changes to RCP (x3, y3, z3, a3, b3, c3), and the point OP (x2, y2, OPz2) changes to OP(x3,y3,OPz3); GP is the center point of the grab hole; EP is the front end center point of the conical connector 10; EPXv and EPZv are the X-direction deviation value and the Z-direction deviation value of the point EP to the point RCP respectively; DeepTv and DeepDv are the average depth of the camera 5 to the area TopRect and the average depth to the area DnRect respectively; AveDeepV is the average depth of the camera 5 to the surface of the grab hole; Distance is the X-direction distance from the point RCP to the point BP; Point BP is the center point of the grab hole when the refueling gun 7 is in an ideal state. The refueling gun 7 is in an ideal state when the back of the refueling gun 7 is in contact with the oil gun rack 8 and the refueling gun 7 is located in the middle of the oil gun rack 8 and the surface of the grab hole is perpendicular to the ground 14; BasCP is the center point of the base of the robot 2 and is also the origin O of the robot 2 coordinate system; X, Z are the X-axis and Z-axis of the robot 2 coordinate system respectively.

[0015] Attached Figure 5 In the figure, after robot 2 rotates and moves, the point RCP (x3, y3, z3, a3, b3, c3) changes to RCP(x4,y4,z4,a4,b4,c4), point OP(x3,y3,OPz3) changes to OP(x4,y4,OPz4); where x4=x3+KWx, y4=y3, z4=z3+KWz, a4=a3, b4=b3+σ, c4=c3; KWx=int[0.5+Kx-Wx], KWz=int[0.5+Kz+Wz]; σ is the rotation angle between the surface of the gripping hole and the Z axis of the robot 2 coordinate system; Wx and Wz are the X-direction offset and Z-direction offset of point OP caused by the rotation of point RCP by angle σ; Kx and Kz are the points to which point RCP(x3,y3,z3,a3,b3,c3) moves When RCP(x4,y4,z4,a4,b4,c4), it is the movement amount of robot 2 in the X direction and the movement amount of robot 2 in the Z direction.

[0016] Attached Figure 6 In the figure, TopRect and DnRect are areas for measuring the surface depth of the grab hole, and are located above GRect and below GRect respectively; GRect is the minimum circumscribed rectangle of the grab hole; the width and height of TopRect and DnRect are the same; point GP is the center point of rectangle GRect; GRectH and GRectW are the width and height of rectangle GRect respectively; rectangle FaceRect is the surface area of ​​the grab hole, FRectH and FRectW are the width and height of rectangle FaceRect respectively; rw and rh are the width and height of area TopRect and area DnRect; the above and below refer to the above and below in the figure.

[0017] Attached Figure 7 IntoDeep is the depth of the front end of the conical seat 26 inserted into the fixing sleeve 9 from the point GP.

[0018] Attached Fig.11 , H1 is the height of the four fixing slots 37 of the oil gun rack 8. DETAILED DESCRIPTION

[0019] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided to make the present invention more comprehensive and complete and to fully convey the concepts of example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and their repeated description will be omitted.

[0020] The words expressing positions and directions described in the present invention are all explained with reference to the accompanying drawings as examples, but they can be changed as needed, and the changes made are all included in the protection scope of the present invention.

[0021] Before introducing the embodiment scheme, the parameters and terms used in the embodiment are first introduced: Quadrangular pyramid: The shape of a pyramid with a quadrilateral base or a quadrilateral top.

[0022] Six degrees of freedom: linear motion of an object along the X-axis, linear motion of an object along the Y-axis, linear motion of an object along the Z-axis, rotational motion of an object around the X-axis, rotational motion of an object around the Y-axis, and rotational motion of an object around the Z-axis.

[0023] Fix the six degrees of freedom: make the object unable to move in any of the six degrees of freedom.

[0024] Tight fit: A tight fit is a fit in which the clearance between two parts is relatively small and they are tightly fitted together.

[0025] Pixel point: Pixel point p(x,y) is a pixel point in the image area PicWH; the value range of the X direction of the image area PicWH is 0 to W-1, and the value range of the Y direction is 0 to H-1; p(x,y)∈PicWH means that the pixel point p(x,y) takes every pixel in the image area PicWH.

[0026] Image point: An image point is an image point with an image value G(x,y) on a pixel p(x,y); for example, a grayscale image point is an image point with a grayscale value Gray(x,y) on a pixel p(x,y), 0≤Gray(x,y)≤V1; a color image point is an image point with a color value Color(x,y) formed by an image point with a red value Red(x,y), an image point with a green value Green(x,y) and an image point with a blue value Blue(x,y) on a pixel p(x,y), 0≤Red(x,y)≤V2, 0≤Green(x,y)≤V2, 0≤Blue(x,y)≤V2; contour image point It is an image point with an outline value OutlineP(x,y) on the pixel point p(x,y), 0≤OutlineP(x,y)≤V3; the depth image point is an image point with a depth value d(x,y) on the pixel point p(x,y), DeepMinV≤d(x,y)≤DeepMaxV; wherein, V1≥255, V2≥255, V3≥255, 30mm≤DeepMinV≤200mm, 300mm≤DeepMaxV≤3000mm; in Example 1, V1=255, V2=255, V3=255; DeepMinV=200mm, DeepMaxV=2000mm.

[0027] Image: An image is an image composed of each image point G(x,y). p(x,y)∈PicWH; For example, a grayscale image is a grayscale image composed of image points with grayscale values ​​Gray(x,y). p(x,y)∈PicWH; a color image is a color image consisting of image points with color values ​​Color(x,y). p(x,y)∈PicWH; the contour image is the contour image composed of each image point with contour value OutlineP(x,y), p(x,y)∈PicWH; the depth image is a depth image composed of each image point with a depth value d(x,y), p(x,y)∈PicWH.

[0028] Video image: Video image refers to an image that is output continuously at a certain rate (such as 25 frames / s).

[0029] Depth value: The depth value d(x, y) is the actual distance value from the camera 5 to the point on the object corresponding to the image point d(x, y).

[0030] The average depth of the region: the average of all depth values ​​that are not 0 in the region; for example, the average depth value DeepAR from camera 5 to region AR is calculated by the following formula: DeepAR=

[0031] st |AR'|≠0 In the formula, the depth value d(x, y)' is a depth value d(x, y) that is not zero in the region AR; and the region AR' is a region in which the depth value d(x, y) is not zero in the region AR.

[0032] The outline of an object: the set of each outline point of an object in the image; OutlineSet={P(xt,yt)|P(xt,yt) is the outline point of the object, t=1,2,…,T}; where t is the sequence of the outline points formed according to a set rule (such as in a clockwise direction according to the chain code), T is the number of outline points, T≥5; outline points refer to points on the boundary of an object in the image.

[0033] The actual pixel points of an object: The actual pixel points of an object refer to the number of pixels in the X and Y directions of the object.

[0034] Neighborhood: The area consisting of pixels whose distance from point CP(x,y) is ≤Rn, where Rn is the number of pixels in the image.

[0035] 8 Neighborhood: The area consisting of points whose distance from point p(x,y) is less than 2 pixels.

[0036] Depth Camera 5: A camera that can output color video images and depth video images.

[0037] Camera optical axis 6: the center line of the lens of the camera 5 and the extension line of the center line.

[0038] The pose of camera 5: The pose of camera 5 refers to the position and rotation of point OP (OPx, OPy, OPz) in space, which are determined by the pose point RCP (x, y, z, a, b, c) of robot 2; wherein point OP is the center point of the target surface of camera 5; OPx=x+dx, OPy=y+dy, OPz=z+dz; in Example 1, dx=0, dy=0, dz=100mm, so OPx=x, OPy=y, OPz=z+100mm, and further OP(OPx, OPy, OPz)=OP(x, y, OPz) ; Because the camera 5 is installed parallel to the end of the robot 2, when x or y or z in the point RCP (x, y, z, a, b, c) translates, the point OP (x, y, OPz) also translates; when a or b or c in the Euler angle of the point RCP (x, y, z, a, b, c) rotates, the point OP (x, y, OPz) rotates at the same angle, but the center point of rotation is the point RCP, and the installation of the camera 5 also satisfies that when a=b=c=0, the camera optical axis 6 is parallel to the X-axis of the coordinate system of the robot 2 and the horizontal direction of the camera 5 to collect images is parallel to the XOY plane of the coordinate system of the robot 2.

[0039] Robot 2: Robot 2 is composed of a robot body with an integrated controller, a robot teaching pendant, a power adapter and cables, and a network interface; the power adapter and cables provide the power required by Robot 2; the network interface is the interface for Robot 2 to communicate with external devices over the network; the robot 2 teaching pendant is a controller used to manually control Robot 2; the main technical indicators of Robot 2 are explosion-proof level, number of axes E, maximum load, maximum working radius, minimum parallel movement in the X, Y and Z directions, minimum rotation of Euler angles a, b, c, and minimum rotation of each axis; the posture of Robot 2 refers to the posture of the center point RCP (x, y, z, a, b, c) of the end of Robot 2 in the coordinate system of Robot 2; wherein x, y, z are the coordinate values ​​of point RCP in the X, Y and Z directions in the coordinate system of Robot 2, respectively, and a, b, c are the Euler angles of rotation in space with point RCP as the center; changing the posture of Robot 2 refers to the processing equipment using The network port and the network cable connect robot 2. The network port controls robot 2 to change the posture of robot 2; the change of posture refers to a change in at least one of the position and the rotation; the coordinate system of robot 2 refers to the center point BasCP of the bottom surface of the base of robot 2 as the origin O, the line from the origin O to the left side of the refueling island 1 and parallel to the long side of the refueling island 1 is the Y positive axis, the line from the origin O to the front of the refueling island 1 is the X positive axis and the X axis is perpendicular to the Y axis, and the XOY plane is parallel to the ground 5, and the coordinate system from the origin O to the vertical upward is the Z positive axis and the Z axis is perpendicular to the XOY plane; in Example 1, the main technical indicators of robot 2 are: explosion-proof grade ExdIIBT4, number of axes E=6, maximum load of 12Kg, maximum working radius of 1.3m, minimum parallel movement in X, Y and Z directions are all ≤1mm, minimum rotation of Euler angles a, b, c are all ≤0.01°, and minimum rotation of each axis is ≤0.01°.

[0040] Gripping Hole Surface: The surface of the retaining sleeve that contains the gripping holes.

[0041] Embodiment 1, a method for automatically grasping a refueling gun based on three-dimensional vision, wherein the method is implemented by an automatic grasping system, wherein the automatic grasping system comprises a processing device, an input device, a display, a camera, a robot, a fixing component, a pneumatic device, a refueling gun, a refueling gun stand, and a power supply, wherein the automatic grasping system further comprises a fixing sleeve and a conical connector, wherein the steps of the method are as follows: Step A.1, controlling the robot 2 to change its posture so that the camera 5 meets the measurement conditions; The steps of controlling the robot 2 to change its posture so that the camera 5 meets the measurement conditions are as follows: Step A.1.1, controlling the robot 2 to change its posture so that the image Img includes a grabbing hole; The steps of controlling the robot 2 to change its posture so that the image Img includes the grasping hole are as follows: Step A.1.1.1, control the robot 2 to change its posture so that the camera optical axis 6 is parallel to the X axis of the coordinate system of the robot 2 and the horizontal direction of the image Img is parallel to the XOY plane of the coordinate system of the robot 2; The steps of controlling the robot 2 to change its posture so that the camera optical axis 6 is parallel to the X axis of the robot 2 coordinate system and the horizontal direction of the image Img is parallel to the XOY plane of the robot 2 coordinate system are as follows: If the camera optical axis 6 is not parallel to the X axis of the robot 2 coordinate system or the horizontal direction of the image Img is not parallel to the XOY plane of the robot 2 coordinate system, the processing device controls the robot 2 to rotate the Euler angles a, b, c in the point RCP, so that the Xv axis, Yv axis, and Zv axis in the XvYvZv coordinate system are respectively parallel to the X axis, Y axis, and Z axis in the robot 2 coordinate system; the XvYvZv coordinate system is a coordinate system formed by rotating the XRYRZR coordinate system in space with the origin OR as the center of the circle; the XRYRZR coordinate system is a coordinate system with the origin OR as the center and the XR axis, YR axis, and ZR axis as the center. The axis and ZR axis are respectively parallel to the X axis, Y axis and Z axis of the coordinate system of the robot 2, and the origin OR is the point RCP; after the rotation, the camera optical axis 6 is parallel to the X axis of the coordinate system of the robot 2 and the horizontal direction of the image Img is parallel to the XOY plane of the coordinate system of the robot 2; the rotation changes the point RCP (x0, y0, z0, a0, b0, c0) to RCP (x1, y1, z1, a1, b1, c1), and the point OP (x0, y0, OPz0) changes to OP (x1, y1, OPz1); wherein a1=0, b1=0, c1=0; as shown in the attached figure Figure 3 As shown; Step A.1.1.2, controlling the robot 2 to change its posture so that the image Img captured by the camera 5 includes the grabbing hole; Step A.1.2, controlling the robot 2 to change its posture so that the distance from the camera 5 to the refueling gun 7 is within a preset range; the preset range is a range in which the image can contain the grabbing hole and the image processing algorithm can identify the grabbing hole; The steps of controlling the robot 2 to change its posture so that the distance from the camera 5 to the refueling gun 7 is within a preset range are as follows: Control the robot 2 to change its posture so that the point RCP changes from the point RCP (x1, y1, z1, a1, b1, c1) to RCP (x2, y2, z2, a2, b2, c2); wherein x2 = Bx-Distance, y2 = By, z2 = Bz, a2 = a1, b2 = b1, c2 = c1, and the distance Distance is within the preset range MinRang to MaxRang; Distance is the distance from the point RCP to the point BP (Bx , By, Bz); Bx, By, Bz are respectively the X-axis coordinate value, Y-axis coordinate value, and Z-axis coordinate value of the point BP in the coordinate system of the robot 2, and Bx, By, Bz are determined by the installation data of the robot 2 and the oil gun rack 8; BP is the center point of the grab hole when the oil gun 7 is in an ideal state, and the oil gun 7 is in an ideal state when the back of the oil gun 7 is in contact with the oil gun rack 8 and the oil gun 7 is located in the middle of the oil gun rack 8 and the surface of the grab hole is perpendicular to the ground 14; In this embodiment, MinRang=200mm, MaxRang=1000mm; the specific calculation of the preset range MinRang and MaxRang is as follows: When MinRang=200mm, the actual width TotalMinW and actual height TotalMinH of the measuring range are calculated as follows: TotalMinW=2×MinRang×tg(LA / 2)=400mm×tg(32.0°)=249.95mm>RealW=38mm TotalMinH=2×MinRang×tg(VA / 2)=400mm×tg(20.5°)=149.55mm>RealH=60mm In the above formula, LA and VA are the horizontal and vertical field angles of the camera 5, LA=64°, VA=41°; RealW and RealH are the width and height of the grabbing hole, respectively; the above calculation shows that the width of the measuring range TotalMinW> the width of the grabbing hole, and the height of the measuring range TotalMinH> the height of the grabbing hole; therefore, when Distance=MinRang, the image Img can include the grabbing hole; When MaxRang=1000mm, the actual width TotalMaxW and actual height TotalMaxH of the measuring range and the actual pixel points RealPnX in the X direction and RealPnY in the Y direction are calculated as follows: TotalMaxW=2×MaxRang×tg(LA / 2)=2000mm×tg(32.0°)=1249.74mm TotalMaxH=2×MaxRang×tg(VA / 2)=2000mm×tg(20.5°)=747.77mm RealPnX=RealW×W / TotalMaxW=38mm×1280 / 1249.74mm=38.92>30 RealPnY=RealH×H / TotalMaxH=60mm×800 / 747.77mm=64.19>50 In the above formula, W and H are the width and height of the image Img, respectively; TotalMaxW and TotalMaxH are the actual width and actual height of the object corresponding to the image Img when MaxRang=1000mm, respectively; RealPnX and RealPnY are the actual pixel points in the X direction and the actual pixel points in the Y direction of the grabbing hole when MaxRang=1000mm, respectively; because the actual pixel points are the smallest at the maximum distance, it is only necessary to calculate the actual pixel points when the maximum distance MaxRang=1000mm; According to the above calculation, when Distance=MaxRang, the actual pixel points RealPnX of the grabbing hole in the image Img in the X direction are greater than 30 and the actual pixel points RealPnY of the Y direction are greater than 50; because the image processing algorithm needs ≥10 pixels to identify an edge; therefore, the actual pixel points in the X direction and the actual pixel points in the Y direction both satisfy the image processing algorithm to identify the edge line in the X direction and the edge line in the Y direction, and further, it can be known that this range satisfies the image processing algorithm to identify the grabbing hole; Step A.2, acquiring image data of the grabbing hole, controlling the robot 2 to change its posture so that the camera 5 meets the posture condition, and obtaining the insertion movement amount; Step A.2.1, obtaining image data of the grabbing hole, and controlling the robot 2 to change the posture so that the camera 5 meets the posture condition; The steps of acquiring the image data of the grabbing hole and controlling the robot 2 to change the posture so that the camera 5 meets the posture condition are as follows; Step A.2.1.1, making the camera 5 meet the measurement conditions; In this embodiment, the camera 5 has met the measurement conditions; Step A.2.1.2, acquiring image data of the grabbing hole, and controlling the robot 2 to change its posture so that the camera optical axis 6 is perpendicular to the surface of the grabbing hole; The steps of acquiring the image data of the grab hole and controlling the robot 2 to change the posture so that the camera optical axis 6 is perpendicular to the surface of the grab hole are as follows: Step A.2.1.2.1, acquiring image data of the grabbing hole, and obtaining depth data of the surface of the grabbing hole; The steps of acquiring the image data of the grab hole and obtaining the depth data of the surface of the grab hole are as follows: Acquire image data of the grabbing hole, and obtain the minimum circumscribed rectangle GRect of the grabbing hole from the image data; According to the minimum circumscribed rectangle GRect, two measurement areas TopRect and DnRect are obtained; wherein TopRect and DnRect are both areas with a width of rw and a height of rh, and are both areas parallel to the GRect, and TopRect and DnRect are respectively located at the top and bottom of the rectangle GRect; wherein, rh=ToFRectH∧DnRectH=DnRectH; rw=GRectW, GRectW is the width of the rectangle GRect; ToFRectH and DnRectH are respectively the heights of the top side of the rectangle GRect and the bottom side of the rectangle GRect in the image Img; as shown in the attached figure Figure 6 As shown; Measure the mean depth value DeepTv from camera 5 to the area TopRect, and measure the mean depth value DeepDv from camera 5 to the area DnRect; wherein the depth value of each pixel point in the area TopRect and the depth value of each pixel point in the area DnRect are both taken from the set DeepAreaSet of depth values ​​of the surface area of ​​the capture hole; Step A.2.1.2.2, use formula (1) to calculate the average depth value AveDeepV of the surface of the grab hole, AveDeepV=(DeepTv+DeepDv) / 2 (1) Step A.2.1.2.3, obtaining image data of the grabbing hole, and controlling the robot 2 to change its posture so that the center point GP of the grabbing hole is located at the center point of the image ImgC; The steps of acquiring the image data of the grabbing hole and controlling the robot 2 to change the posture so that the center point GP of the grabbing hole is located at the center point ImgC of the image are as follows: Acquire image data of the grabbing hole, and obtain the center point GP (IGx, IGy) of the grabbing hole in the image Img from the image data; The specific steps of controlling the robot 2 to change its posture so that the point GP (IGx, IGy) moves to the point ImgC (ImgCx, ImgCy) are as follows: When the distance from the camera 5 to the surface of the gripping hole is AveDeepV, the point GP (IGx, IGy) is moved to the point ImgC (ImgCx, ImgCy), and the movement amount DCy of the point RCP in the Y direction and the movement amount DCz in the Z direction are calculated by the following formula: DCy=AveDeepV×tg(LA / 2)×(IGx-ImgCx) / W DCz=AveDeepV×tg(VA / 2)×(IGy-ImgCy) / H In the formula, the processing device controls the robot 2 to move the point RCP in the Y direction by DCy and in the Z direction by DCz, and if DCy≥0, the Y direction moves in the positive Y direction, otherwise it moves in the negative Y direction; if DCz≥0, the Z direction moves in the positive Z direction, otherwise it moves in the negative Z direction; after the movement is completed, the point GP moves to the point ImgC; the movement of the following points is realized in a similar way; When the robot 2 moves so that the point GP is located at the center point ImgC of the image, the point RCP (x2, y2, z2, a2, b2, c2) changes to RCP (x3, y3, z3, a3, b3, c3), and the point OP (x2, y2, OPz2) changes to OP (x3, y3, OPz3); wherein a3=a2, b3=b2, c3=c2; as shown in the attached figure Figure 4 As shown; Step A.2.1.2.4, calculate the surface X-direction depth difference Dx of the grab hole using formula (2); Dx=DeepTv-DeepDv (2) Step A.2.1.2.5. Use formula (3) to calculate the angle σ between the surface of the gripping hole and the Z axis of the robot 2 coordinate system. σ = tg -1 (Dx / Dz) (3) (3) Where Dz = GRectH + rh, Dz is the height difference between the center point of region TopRect and the center point of region DnRect; Step A.2.1.2.6, if |σ|<MinWhirlAngle, go to step A.2.1.2.7, otherwise go to step A.2.1.2.8; wherein || is an absolute value operation; MinWhirlAngle is the minimum rotation of the b-axis in the robot 2; Step A.2.1.2.7, set σ = 0; Step A.2.1.2.8, use formula (4) to calculate the X-direction offset Wx and Z-direction offset Wz of point OP caused by the rotation of point RCP by angle σ, Wx=dz×Sinσ Wz=dz×(1-Cosσ) (4) Step A.2.1.2.9. Use formula (5) to calculate the movement distance Kxz of point RCP when it moves from point RCP(x2, y2, z2, a2, b2, c2) to point RCP(x3, y3, z3, a3, b3, c3). Kxz = AveDeepV × Sinσ (5) Step A.2.1.2.10: Use formula (6) to calculate the corresponding movement amount Kx in the X direction and the movement amount Kz in the Z direction. Kx=Kxz×Sinσ Kz=Kxz×Cosσ (6) Step A.2.1.2.11, the processing device controls the robot 2 to rotate the Euler angle b axis in the point RCP counterclockwise by an angle σ, and controls the point RCP to move KWx in the positive X-axis direction and KWz in the positive Z-axis direction; wherein, counterclockwise refers to counterclockwise with respect to the positive Z-axis, and if σ=0, no actual rotation is performed; KWx=int[0.5+Kx-Wx], KWz=int[0.5+Kz+Wz]; int[] is an integer operation, and adding 0.5 to int[] can minimize the quantization error of the integer operation; Step A.2.1.2.12, after the robot 2 rotates and moves, the camera optical axis 6 is perpendicular to the surface of the grab hole (and at this time the point GP (IGx, IGy) is located at the center point ImgC (ImgCx, ImgCy) of the image Img); the point RCP (x3, y3, z3, a3, b3, c3) changes to RCP (x4, y4, z4, a4, b4, c4), and the point OP (x3, y3, OPz3) changes to OP (x4, y4, OPz4); wherein, x4 = x3 + KWx, y4 = y3, z4 = z3 + KWz, a4 = a3, b4 = b3 + σ, c4 = c3; as shown in the attached figure Figure 5 As shown; Step A.2.2, acquiring image data of the grabbing hole and obtaining an insertion movement amount; the insertion movement amount is the movement amount IntoMoveXv in the X direction, the movement amount IntoMoveYv in the Y direction, and the movement amount IntoMoveZv in the Z direction when the robot 2 moves to a position where the tapered connector 10 can be connected to the fixing sleeve 9; The steps of acquiring the image data of the grabbing hole and obtaining the insertion movement amount are as follows: Step A.2.2.1, calculate the coordinate value of point GP in the robot coordinate system; The steps to calculate the coordinate value of point GP in the coordinate system of robot 2 are as follows: Step A.2.2.1.1. Use formula (7) to calculate the X-direction movement MoveXv and Z-direction movement MoveZv of robot 2 when point RCP (x4, y4, z4, a4, b4, c4) moves to the point OP. MoveRXv=dz×Sinσ MoveRZv=dz×Cosσ (7) (7)In the formula, dz is the Z-direction offset from point OP to point RCP; Step A.2.2.1.2, use formula (8) to calculate the X-axis coordinate value Gx, Y-axis coordinate value Gy, and Z-axis coordinate value Gz of the center point GP of the gripping hole in the coordinate system of robot 2, Gx=x4+MoveRXv+AveDeepV×Cosσ Gy=y4 Gz=z4+MoveRZv-AveDeepV×Sinσ (8) Step A.2.3, calculate the coordinate value of point EP; The steps to calculate the coordinate value of point EP are as follows: Step A.2.3.1. Use formula (9) to calculate the X-direction movement MoveEXv and Z-direction movement MoveEZv of robot 2 when point EP moves to the RCP. MoveEXv=EPZv×Sinσ MoveEZv=EPZv×(1-Cosσ) (9) (9) where point EP is the center point of the front end of the tapered connector 10; EPZv is the Z-direction distance from point EP to point RCP; Step A.2.3.2, use formula (10) to calculate the X-axis coordinate value Ex, Y-axis coordinate value Ey, and Z-axis coordinate value Ez of point EP, Ex=x4+EPXv-MoveEXv Ey=y4 Ez=z4+EPZv+MoveEZv (10) (10)In the formula, EPXv is the X-direction distance from point EP to point RCP; Step A.2.3.3, calculate the movement distance from point EP to point GP; Use formula (11) to calculate the X-direction movement amount Mx, Y-direction movement amount My, and Z-direction movement amount Mz from point EP to point GP. Mx=Gx-Ex My=Gy-Ey Mz=Gz-Ez (11) Step A.2.3.4, use formula (12) to calculate the X-direction movement Ix, Y-direction movement Iy, and Z-direction movement Iz of point GP when the depth of insertion from point GP into the grab hole is IntoDeep, Ix=IntoDeep×Cosσ Iy=0 Iz=-IntoDeep×Sinσ (12) In formula (12), when the front end of the conical connector 10 is inserted into the gripping hole from point GP to a depth value of IntoDeep, the sliding rod 25 is inserted between the upper convex slider 16 and the lower convex slider 18, and the upper convex slider 16 extends upward from the conical seat 26 and is connected to the upper groove 17 in the fixing sleeve 9, while the lower convex slider 18 extends downward from the conical seat 26 and is connected to the lower groove 19 in the fixing sleeve 9; Step A.2.3.5, use formula (13) to calculate the insertion movement amount IntoMoveXv, IntoMoveYv, IntoMoveZv, IntoMoveXv=Mx+Ix IntoMoveYv=My IntoMoveZv=Mz+Iz (13) (13) IntoMoveXv, IntoMoveYv, and IntoMoveZv are the movement amounts of robot 2 in the X direction, the movement amounts in the Y direction, and the movement amounts in the Z direction, respectively, when the point RCP (x4, y4, z4, a4, b4, c4) changes to RCP (x5, y5, z5, a5, b5, c5); Step A.3, automatically grab the refueling gun 7 according to the insertion movement amount.

[0042] In embodiment 1, the method described is implemented by an automatic grasping system; the automatic grasping system includes a processing device, an input device, a display, a camera, a robot, a fixing component, a pneumatic device, a refueling gun, a refueling gun stand, and a power supply; the automatic grasping system also includes a fixing sleeve and a conical connector; wherein, The processing device is connected to the control circuits in the input device, display, camera, robot, and pneumatic device; the processing device is an industrial computer with a hard disk that meets the use requirements and a Windows operating system and application software installed on the hard disk; the application software includes at least the following programs: an image acquisition program, a picture shooting program, an image processing program, a depth measurement program, a solenoid valve control program, a robot 2 control program, and a network communication program; the image processing program includes at least an image capture program, a color image conversion program to a grayscale image program, a gradient algorithm program, an edge point acquisition program, a filtering algorithm program, a maximum contour algorithm program, a capture hole contour acquisition program, a filling algorithm program, a capture hole minimum circumscribed rectangle acquisition program, a region center point acquisition program, and a pixel depth value acquisition program in the region; the solenoid valve control program is a program for controlling the solenoid valve to open or close; the robot 2 control program is a program for changing the posture of the robot 2 by controlling the robot 2; The fixing assembly includes a fixing plate 15, a camera fixing plate 4, a fixing bottom plate 13, a fixing bar 34, and fixing screws; the fixing plate 15 is fixedly connected to the robot 2 by using four fixing screws to pass through the through holes on the fixing plate 15 from the front to fix it on the plane at the end of the robot 2; The camera 5 outputs its color video and depth video to the processing device through its USB interface using an explosion-proof USB extension cable. The camera 5 has a color resolution ≥ 920,000 pixels, a depth resolution ≥ 920,000 pixels, a measurement depth range of 200mm-2000mm, and is fixed to the end of the robot 2 with a fixing plate 15 and a camera fixing plate 4 in a fixing assembly. The depth camera has an explosion-proof grade of ExdIIBT4. When the depth camera 5 is installed, the deviation (dx, dy, dz) between the point OP (OPx, OPy, OPz) of the camera 5 and the center point RCP (x, y, z, a, b, c) of the end of the robot 2 in the coordinate system of the robot 2 is determined; the X-direction deviation value dx=OPx-x, the Y-direction deviation value dy=OPy-y, and the Z-direction deviation value dz=OPz- z; in Example 1, dx=0, dy=0, dz=100mm; so OP(OPx, OPy, OPz)=OP(x, y, OPz); the camera 5 is fixedly connected to the fixing plate 15, and the connection method is to use two fixing screws from the back of the camera fixing plate 4 through the through holes thereon to fix the camera fixing plate 4 to the back of the fixing plate 15, and use two fixing screws from the back of the camera fixing plate 4 through the through holes thereon to fix the camera 5 to the front of the camera fixing plate 4; the installation of the camera 5 also satisfies the following conditions: when the Euler angle a=b=c=0 in the point RCP(x, y, z, a, b, c), the camera optical axis 6 is parallel to the X-axis of the robot coordinate system and the horizontal direction of the camera 5 collecting images is parallel to the XOY plane of the robot coordinate system; as shown in the attached figure Figure 4 As shown; the front and back mentioned refer to the left and right sides in the figure; The explosion-proof grade of the robot 2 is ExdIIBT4. The robot 2 is composed of a robot body including an integrated controller, a robot teaching pendant, a power adapter and cables, and a network interface; the power adapter and cables provide the power required by the robot 2; the network interface is an interface for network communication between the robot 2 and the processing equipment, and the network interface is connected to the network port of the processing equipment with a network cable; the robot teaching pendant is a controller for manually controlling the position and posture of the robot 2; the robot 2 is a robot that meets the use requirements and is equipped with a camera 5 at its end; as shown in the attached Figure 4 With attached Figure 5 As shown; The power supply is connected to the display and the camera and provides the required DC power to the display and the camera; The fixing sleeve 9 has a gripping hole; the upper and lower parts of the gripping hole have a quadrangular pyramidal groove; when the refueling gun 7 is placed on the oil gun rack 8, the four fixing baffles 27 at the four corners of the fixing sleeve 9 are inserted into the four fixing grooves 37 on the oil gun rack 8; the upper and lower parts refer to the upper and lower parts in the figure respectively; The conical connector 10 includes a conical seat 26, a sliding rod 25, an upper convex slider 16, a lower convex slider 18, a spring 20, a screw 21 and a screw 23 for fixing the spring, and a nut 22 and a nut 24 for fixing the spring; the conical connector 10 is fixedly connected to the robot 2, and the connection method is that the conical connector 10 is fixedly connected to the fixed base plate 13, the fixed base plate 13 is fixedly connected to the fixed plate 15, and the fixed plate 15 The conical connector 10 is fixedly connected to the robot 2; the conical connector 10 is fixedly connected to the fixed base plate 13 by using two fixing screws from the bottom of the fixed base plate 13 through the through holes thereon to fix the bottom of the conical connector 10 to the front end of the fixed base plate 13, and two fixing screws 35 are used to fix the upper ends of the two fixing strips 34 to the upper sides of the two sides of the conical connector 10, and two fixing screws are used to fix the lower ends of the two fixing strips 34 to the fixed base plate 13 respectively; the fixed base plate 13 is fixedly connected to the fixed plate 15 by using three fixing screws from one side of the rear of the fixed plate 15 through the through holes thereon to fix one rear side surface of the fixed base plate 13 to the fixed plate 15, and three fixing screws are used from the other side of the rear of the fixed plate 15 through the through holes thereon to fix the other rear side surface of the fixed base plate 13 to the fixed plate 15; The cylinder 12 in the pneumatic device is fixedly connected to the fixing plate 15, and is fixed to the fixing plate 15 by passing four screws through the fixing holes thereon; the sliding rod 25 is fixedly connected to the piston rod 11, and is fixedly connected to the screw rod and the fastening nut at the front end of the piston rod 11 through the thread at the rear end of the sliding rod 25; The upper convex slider 16 and the lower convex slider 18 are both placed in the conical seat 26 and are fixedly connected to the upper end of the spring 20 and the lower end of the spring 20 respectively; The oil gun rack 8 is used to place the oil gun 7; the oil gun rack 8 is fixed on the fixing surface 36 of the oil gun rack 8; the installation of the oil gun rack 8 also satisfies the condition that when the oil gun 7 is in an ideal shape, the surface of the grabbing hole in the fixing sleeve 9 is perpendicular to the ground 14; The pneumatic device comprises a compressor, an air intake pipe, a control circuit, a solenoid valve, an air pipe F, an air pipe B, a piston rod 11, and a cylinder 12; wherein the compressor is connected to the air intake pipe; the air intake pipe is connected to the air intake port of the solenoid valve, and the air outlet F of the solenoid valve is connected to the air pipe F; the air pipe F is connected to the air joint at the front end of the cylinder 12; the air outlet B of the solenoid valve is connected to the air pipe B; the air pipe B is connected to the air joint at the rear end of the cylinder 12; the solenoid valve is a two-position five-way solenoid valve; the processing device controls the solenoid valve through the control circuit to supply air to the rear end of the cylinder 12 so that the piston rod 11 extends forward, or controls the solenoid valve through the control circuit to supply air to the front end of the cylinder 12 so that the piston rod 11 retracts backward; The processing equipment, input equipment, display, and power supply are all placed indoors; Figure 1 shown.

[0043] The steps of obtaining the image data of the grab hole according to the present invention are as follows: In Example 1, the steps of obtaining the image data of the grabbing hole are as follows: Step B.1, the processing device captures a frame of image Img and a frame of depth image DeepImg from the video image output by the camera 5; the image Img refers to a color image; Step B.2, identifying the grabbing hole from the image using an image processing algorithm; The steps of identifying the gripping holes from the image using an image processing algorithm are as follows: Step B.2.1, obtain the area FaceRect of the capture hole in the image; The steps to obtain the FaceRect region of the capture hole in the image are as follows: Formula (14) is used to calculate the actual width TotalDW and actual height TotalDH corresponding to the image Img captured by the camera 5 when the distance is Distance, TotalDW=2×Distance×tg(LA / 2) TotalDH=2×Distance×tg(VA / 2) (14) Step B.2.2: According to the actual width RealW and actual height RealH of the grabbing hole, use formula (15) to calculate the width GRectW and height GRectH of the grabbing hole in the image Img. GRectW=W×RealW / TotalDW GRectH=H×RealH / TotalDH (15) (15)W and H are the width and height of the image Img, respectively; Formula (16) is used to calculate the width FRectW and height FRectH of the surface area FaceRect of the grabbing hole. FRectW=GRectW×ExpW FRectH=GRectH×ExpH (16) (16) Where ExpW and ExpH are the proportional coefficients of the set expansion width and the set expansion height, respectively; 1.1≤ExpW≤1.3, 1.1≤ExpH≤1.2; Step B.2.3, use formula (17) to calculate the boundary value of the region FaceRect, FLeft=Max(0,ImgCx-FRectW / 2), FRight=Min(W-1,ImgCx+FRectW / 2) FDown=Min(H-1,ImgCy+FRectH / 2), FTop=Max(0,ImgCy-FRectH / 2) (17) (17) In the formula, Max() is the maximum value operation, Min() is the minimum value operation; FLeft, FRight, FDown, and FTop are the left boundary value, right boundary value, lower boundary value, and upper boundary value of the region FaceRect, respectively; Step B.3, obtaining image data of the grabbing hole according to the identified grabbing hole; the image data of the grabbing hole at least includes the outline Outline of the grabbing hole, the area GRect of the grabbing hole, the center point GP of the grabbing hole area, and the set DeepAreaSet of depth values ​​of each pixel point in the surface area FaceRect of the grabbing hole; The steps for obtaining the image data of the grabbing hole according to the identified grabbing hole are as follows: Step B.3.1, the processing device captures a frame of image Img and a frame of depth image DeepImg from the video image output by the camera 5; the image Img refers to a color image; Step B.3.2, grayscale the color image Img to obtain a grayscale image GrayImg; Step B.3.3, within the region FaceRect, use a filtering algorithm (such as a Gaussian filtering algorithm) to perform noise reduction filtering on the grayscale image GrayImg; Step B.3.4, using an edge detection algorithm (such as a Canny edge detection algorithm) to obtain edge points in the area FaceRect of the grayscale image GrayImg, where the edge points refer to the edge points in the outline image OutlineImg; Step B.3.5, using a filtering algorithm (such as a Gaussian filtering algorithm) to delete isolated edge points in the area FaceRect of the outline image OutlineImg, where an isolated edge point refers to an edge point that has no edge points in the 8 neighborhood of the point; Step B.3.6, in the region FaceRect, using a convex hull algorithm (such as a Convex Hull algorithm) to connect the edge points in the outline graph OutlineImg to obtain a set of outline points of the capture hole Outline; Step B.3.7, using the left edge point, right edge point, bottom edge point, and top edge point in the set of outline points Outline to respectively calculate the left straight line, right straight line, bottom straight line, and top straight line of the grabbing hole; using edge points to calculate a straight line refers to using the least squares algorithm to calculate a straight line using edge points; Step B.3.8, using the left straight line, the right straight line, the bottom straight line, and the top straight line to calculate the four vertices of the grabbing hole; Step B.3.9, using the four vertices of the grabbing hole to obtain the minimum circumscribed rectangle GRect of the grabbing hole; Step B.3.10, using the four vertices of the grabbing hole to calculate the center point GP of GRect; Step B.3.11, use the depth image DeepImg to obtain a set DeepAreaSet of depth values ​​for each pixel in the area FaceRect.

[0044] The steps of automatically grabbing the refueling gun according to the insertion movement amount are as follows: In embodiment 1, the steps of automatically grabbing the refueling gun according to the insertion movement amount are as follows: According to the insertion movement amount, the robot 2 is controlled to move so that the tapered connector 10 is inserted into the grab hole. This step specifically controls the robot 2 to move from point RCP (x4, y4, z4, a4, b4, c4) to point RCP (x5, y5, z5, a5, b5, c5); wherein, x5=x4+IntoMoveXv, y5=y4+IntoMoveYv, z5=z4+IntoMoveZv, a5=a4, b5=b4, c5=c4; and if IntoMoveXv≥0, the robot 2 moves in the positive X direction, otherwise it moves in the negative X direction; if IntoMoveYv≥0, the robot 2 moves in the positive Y direction, otherwise it moves in the negative Y direction; if IntoMoveZv≥0, the robot 2 moves in the positive Z direction, otherwise it moves in the negative Z direction; The piston rod 11 in the pneumatic device is controlled to extend, so that the upper convex slider 16 and the lower convex slider 18 in the conical connector 10 extend out of the conical seat 26 in the conical connector 10, and the upper convex slider 16 is connected to the upper groove 17 in the fixing sleeve 9, and the lower convex slider 18 is connected to the lower groove 19 in the fixing sleeve 9, so as to realize the connection between the robot 2 and the refueling gun 7; wherein the connection refers to the connection of the four surfaces of the quadrangular pyramid in a tight fit; The robot 2 is controlled to move upward by Kh mm so that the refueling gun 7 is taken out from the oil gun rack 8; Kh mm>H1 mm; H1 mm is the height of the fixing groove 37 of the oil gun rack 8.

[0045] Embodiment 1, the fixing sleeve 9 is fixedly connected to the refueling gun 7; the fixing sleeve 9 has a grab hole; the grab hole is in the shape of a concave quadrangular pyramid, and the shape and angle of the concave quadrangular pyramid are the same as the shape and angle of the conical seat 26 in the conical connector; an upper groove 17 is provided on the upper side of the concave quadrangular pyramid, and a lower groove 19 is provided on the lower side of the concave quadrangular pyramid; the shapes of the upper groove 17 and the lower groove 19 are both quadrangular pyramids; The conical connector 10 comprises a conical seat 26, a sliding rod 25, an upper convex slider 16, a lower convex slider 18, a spring 20, a screw 21 for fixing the upper end of the spring, a screw 23 for fixing the lower end of the spring, a nut 22 for fixing the upper end of the spring, and a nut 24 for fixing the lower end of the spring; wherein, The conical seat 26 has a quadrangular pyramidal shape, and contains the upper convex slider 16, the lower convex slider 18, the screws 21 and 23 for fixing the spring, and the nuts 22 and 24 for fixing the spring. Since the conical seat 26 has a quadrangular pyramidal shape, the horizontal positioning range of the conical seat 26 is the difference between the front and rear ends of the horizontal cone of the conical shape, and the vertical positioning range is the difference between the front and rear ends of the vertical cone of the conical shape. This positioning range is much larger than the positioning range of the circular connector. The sliding rod 25 is fixedly connected to the piston rod 11 in the pneumatic device; the upper and lower surfaces of the front of the sliding rod 25 are conical; wherein the angle of the upper cone of the front end of the sliding rod 25 is the same as the angle of the lower portion of the upper convex slider 16, and the angle of the lower cone of the front end of the sliding rod 25 is the same as the angle of the upper portion of the lower convex slider 18; this taper makes it easier for the sliding rod 25 to be inserted between the upper convex slider 16 and the lower convex slider 18, and because when the sliding rod 25 is inserted, the convex slider is subjected to a horizontal force as well as a vertical force, making it easier for the convex slider to extend outward; The shape of the upper convex slider 16 is a quadrangular pyramid; the shape of the lower convex slider 18 is a quadrangular pyramid; the upper convex slider 16 is fixedly connected to the upper end of the spring 20, and the lower convex slider 18 is fixedly connected to the lower end of the spring 20; the fixing screw 30 fixes the upper part 28 of the upper convex slider 16 with the lower part 29; the shape and angle of the quadrangular pyramid on the upper convex slider 16 are the same as the shape and angle of the upper groove 17; the shape and angle of the quadrangular pyramid below the lower convex slider 17 are the same as the shape and angle of the lower groove 19; the fixing screw 33 fixes the upper part 32 of the lower convex slider 18 with the lower part 31; when the sliding rod 25 is not inserted into the upper convex slider 16 and the lower convex slider 18, the upper convex slider 16 and the lower convex slider 18 are both within the conical seat 26; when the sliding rod 25 is inserted between the upper convex slider 16 and the lower convex slider 18, the upper convex slider 16 extends out of the conical seat 26 and is connected to the upper groove 17 in the fixing sleeve 9, and the lower convex slider 18 extends out of the conical seat 26 and is connected to the lower groove 19 in the fixing sleeve 9; wherein, the connection refers to a tight-fitting connection of the four faces of the quadrangular pyramid; since the shape of the quadrangular pyramid of the convex slider and the angle of the cone are the same as the shape of the quadrangular pyramid of the groove and the angle of the cone, and the connection between the convex slider and the groove is a tight-fitting connection, the six degrees of freedom of the conical connector 10 are all stably fixed, thereby realizing a stable and reliable connection between the robot 2 and the refueling gun 7.

[0046] The working principle of the above technical solution is: use a depth camera to collect video images, use image processing to identify the grabbing hole and obtain its image data, control the robot to change its posture, insert the conical connector into the grabbing hole, and then control the pneumatic device to insert the sliding rod in the conical connector between the upper convex slider and the lower convex slider, so that the upper convex slider and the lower convex slider extend from the conical seat, and are tightly fitted with the upper groove and the lower groove in the fixing sleeve respectively, and the shape and angle of the four-cornered pyramid of the convex slider are the same as the shape and angle of the four-cornered pyramid of the groove, so as to realize stable and reliable connection between the robot and the refueling gun; the method is implemented by an automatic grasping system; the automatic grasping system includes processing equipment, input equipment, display, camera, robot, fixing assembly, pneumatic device, refueling gun, refueling gun rack, power supply, fixing sleeve, and conical connector.

[0047] The beneficial effects of the above technical solution are: using a depth camera to capture video images, using image processing to identify the grab hole connected to the refueling gun and obtain its image data, the image data including the outline of the grab hole, the center point of the grab hole area, and the depth value of each pixel point in the surface area of ​​the grab hole; the technical solution uses three-dimensional vision to implement measurement and has the characteristics of large measurement range, accurate measurement, large grabbing range, stable and reliable grabbing, large grabbing load, convenient and reliable use, good practicality, and high stability; the products of this technical solution can be widely used in automatic refueling at gas stations, with good economic benefits and broad market prospects.

[0048] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for automatically grabbing a refueling gun based on three-dimensional vision, wherein the method is implemented by an automatic grabbing system, wherein the automatic grabbing system includes a processing device, an input device, a display, a camera, a robot, a fixing component, a pneumatic device, a refueling gun, a refueling gun stand, and a power supply, wherein: The automatic grabbing system further comprises a fixing sleeve and a conical connector, and the steps of the method are as follows: Step A.1, control the robot to change its posture so that the camera meets the measurement conditions; the measurement conditions include that the image contains the grabbing hole, and the distance from the camera to the refueling gun is within a preset range; the image refers to the image captured by the camera; the grabbing hole refers to the grabbing hole in the fixing sleeve; the fixing sleeve is a fixing sleeve fixedly connected to the refueling gun; the preset range is the range in which the image can contain the grabbing hole and the image processing algorithm can identify the grabbing hole; The steps of controlling the robot to change its posture so that the camera meets the measurement conditions are as follows: Controlling the robot to change its posture so that the image includes a grasping hole; The steps of controlling the robot to change its posture so that the image includes the grasping hole are as follows: Controlling the robot to change its posture so that the camera optical axis is parallel to the X axis of the robot coordinate system and the horizontal direction of the image is parallel to the XOY plane of the robot coordinate system; Controlling the robot to change its posture so that the image captured by the camera includes the grabbing hole; Controlling the robot to change its posture so that the distance from the camera to the refueling gun is within a preset range; Step A.2, acquiring image data of the grabbing hole, controlling the robot to change its posture, making the camera meet the posture condition, and obtaining the insertion movement amount; the posture condition includes that the camera meets the measurement condition, and the camera optical axis is perpendicular to the surface of the grabbing hole; the insertion movement amount is the movement amount of the robot to make the tapered connector in a position to be connected to the fixing sleeve; the surface of the grabbing hole is the surface of the fixing sleeve containing the grabbing hole; The steps of obtaining the image data of the grabbing hole, controlling the robot to change the posture so that the camera meets the posture condition, and obtaining the insertion movement amount are as follows: Acquire image data of the grabbing hole, and control the robot to change its posture so that the camera meets the posture condition; Acquire image data of the grab hole and obtain the insertion movement amount; Step A.3, automatically grab the refueling gun according to the insertion movement amount.

2. The method for automatically grabbing a refueling gun based on three-dimensional vision according to claim 1 is characterized in that: The steps of obtaining the image data of the grab hole are as follows: The processing device captures a frame of image and a frame of depth image from the video image output by the camera; the image is a color image; the camera is a depth camera; identifying a gripping hole from the image using an image processing algorithm; The image data of the grabbing hole is obtained according to the identified grabbing hole; the image data of the grabbing hole at least includes the contour of the grabbing hole, the area of ​​the grabbing hole, the center point of the grabbing hole area, and the depth value of each pixel point in the surface area of ​​the grabbing hole.

3. The method for automatically grabbing a refueling gun based on three-dimensional vision according to claim 1 is characterized in that: The steps of automatically grabbing the refueling gun according to the insertion movement amount are as follows: According to the insertion movement amount, controlling the robot to move so that the tapered connector is inserted into the grabbing hole; Control the piston rod in the pneumatic device to extend, so that the upper convex slider and the lower convex slider in the conical connector extend out of the conical seat in the conical connector, and the upper convex slider is connected to the upper groove in the fixed sleeve, and the lower convex slider is connected to the lower groove in the fixed sleeve, so as to realize the connection between the robot and the refueling gun; The robot is controlled to move so that the refueling gun is taken out from the refueling gun rack.

4. The method for automatically grabbing a refueling gun based on three-dimensional vision according to claim 1 is characterized in that: The fixing sleeve is fixedly connected to the refueling gun; the fixing sleeve has a grabbing hole; the grabbing hole is in the shape of a concave quadrangular pyramid, and the shape and angle of the concave quadrangular pyramid are the same as the shape and angle of the conical seat in the conical connector; an upper groove is provided on the upper side of the concave quadrangular pyramid, and a lower groove is provided on the lower side of the concave quadrangular pyramid; the shapes of the upper groove and the lower groove are both quadrangular pyramids; The conical connector comprises a conical seat, a sliding rod, an upper convex slider, a lower convex slider, a spring, a screw for fixing the spring, and a nut for fixing the spring; wherein, The conical seat has a quadrangular cone shape, and contains the upper convex slider, the lower convex slider, the spring, the screw for fixing the spring, and the nut for fixing the spring; The sliding rod is fixedly connected to the piston rod in the pneumatic device; the upper and lower surfaces of the front of the sliding rod are conical; The shape of the upper convex slider is a quadrangular pyramid; the shape of the lower convex slider is a quadrangular pyramid; the upper convex slider is fixedly connected to the upper end of the spring, and the lower convex slider is fixedly connected to the lower end of the spring; the shape and angle of the quadrangular pyramid above the upper convex slider are the same as the shape and angle of the upper groove; the shape and angle of the quadrangular pyramid below the lower convex slider are the same as the shape and angle of the lower groove; when the sliding rod is not inserted between the upper convex slider and the lower convex slider, the upper convex slider and the lower convex slider are both in the conical seat; when the sliding rod is inserted between the upper convex slider and the lower convex slider, the upper convex slider extends out of the conical seat and is connected with the upper groove in the fixing sleeve, and the lower convex slider extends out of the conical seat and is connected with the lower groove in the fixing sleeve; wherein, the connection refers to a tight fit connection of the four surfaces of the quadrangular pyramid.

Images