A method for estimating the pose of the fuel cap of an automatic refueling robot

Abstract

The present invention discloses a method for estimating the posture of a fuel cap of an automatic refueling robot, which is applied to the technical field of automatic refueling of fuel vehicles. A solution combining passive binocular stereo vision and visual servoing is proposed. Passive binocular stereo vision is used to perceive environmental information, and stereo matching technology is used to obtain the posture information of the fuel cap grasping point. In addition, a visual servoing method is added. The passive binocular camera is responsible for the "coarse positioning" part of the early perception task, that is, the process of calculating the posture information of the grasping point through the binocular stereo matching method. The visual servo performs "fine positioning" on the basis of "coarse positioning", that is, the process of guiding the robot arm to the grasping point position through the visual servo technology, to ensure that the robot arm moves to the fuel cap grasping point position so that the fuel cap can be unscrewed and the fuel gun can be inserted for refueling.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic refueling of fuel vehicles, and more particularly to a method for estimating the position and posture of a fuel tank cap of an automatic refueling robot. Background Art

[0002] At present, most gas stations rely on gas station workers to refuel manually, which requires a lot of manpower, material and financial costs, and there are problems such as improper operation by gas station workers, which may cause safety accidents. Therefore, the development of an automatic refueling robot has broad application prospects.

[0003] At present, the existing solutions in the field of fuel cap perception and posture estimation can be roughly divided into three categories: (1) The first category is to obtain three-dimensional information by using expensive equipment such as laser scanners to scan the area near the fuel cap. This method is expensive and difficult to meet the explosion-proof regulations of gas stations; (2) The second category is to match the vehicle information in the database by the owner's active entry of vehicle information. This method can only collect vehicle models existing in the database and requires the owner to spend a lot of time entering vehicle information before refueling, which is not flexible; (3) The third category is to use a monocular industrial camera to guide the robotic arm to move to the vicinity of the fuel cap through the relative position relationship of the fuel cap in the camera's field of view. Due to the lack of depth information, this method can only adopt visual servoing, and the perception real-time performance is poor and the accuracy is low.

[0004] Therefore, how to provide a method for estimating the position and posture of the fuel cap of an automatic refueling robot that meets the high-precision requirements of automatic refueling perception is an urgent problem that those skilled in the art need to solve. Summary of the Invention

[0005] In view of this, the present invention provides a method for estimating the position and posture of the fuel cap of an automatic refueling robot. Deep learning technology is used to realize intelligent perception in automatic refueling scenarios, eliminating the need for tedious steps such as manual information entry and vehicle library matching in advance. A solution combining binocular stereo matching and visual servoing is adopted to ensure the high-precision requirements of automatic refueling perception.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A method for estimating the position and posture of a fuel cap of an automatic refueling robot, comprising the following steps:

[0008] The main control board establishes a TCP connection with the robotic arm, initializes the binocular camera connection, loads the camera's internal parameters and hand-eye calibration parameters, and initializes the recognition program;

[0009] Obtain at least two images of the area of ​​interest, identify the fuel tank cover in each image, and calculate the position information of the suction point located at the center of the fuel tank cover. The main control board sends this position information to the robotic arm, which then sucks the cover open.

[0010] Turn on the bar light source (turning on the bar light source provides stable lighting conditions, obtains high-quality image acquisition results, and improves the stability of image recognition). Acquire at least two images of the second region of interest, identify the inner cover handle in each image, and calculate the position information of the inner cover handle grasping point. The main control board sends the position information to the robotic arm, and the robotic arm moves the gripper to the grasping point.

[0011] Determine the relative position relationship between the gripping point of the robotic arm's gripper and the inner cover of the refueling tank. When the gripper of the robotic arm is in the same direction as the inner cover handle, the robotic arm unscrews the inner cover of the refueling tank for refueling.

[0012] Through the above technical solution, the technical effect of the present invention is that a solution combining passive binocular stereo vision and visual servoing is proposed. Active binocular vision is different from passive binocular vision. It mainly refers to the visual imaging process that uses the round-trip time of optical signals such as laser or infrared light to perceive the depth information of the environment. Gas stations are generally located in open and unobstructed scenes due to explosion-proof requirements. Therefore, the ambient light interference is extremely large. The active binocular vision using optical signals is easily interfered by the ambient light and has weak environmental perception ability. Therefore, the present invention uses passive binocular stereo vision to perceive environmental information and uses stereo matching technology to obtain the posture information of the gas cap suction point; the passive binocular camera uses Using the traditional camera imaging principle to obtain images within the field of view, it is difficult to ensure the accuracy of target imaging for cameras with limited resolution. At the same time, there are matching errors in the stereo matching process, making it difficult to obtain the true position information of the target. Therefore, the present invention incorporates a visual servoing method. The passive binocular camera is responsible for the "coarse positioning" part of the early perception task, that is, the process of calculating the position information of the grasping point through the binocular stereo matching method. Visual servoing performs "fine positioning" on the basis of "coarse positioning", that is, the process of guiding the robotic arm to move to the grasping point position through visual servoing technology, ensuring that the robotic arm moves to the grasping point position of the fuel cap so that the fuel cap can be unscrewed and the fuel gun can be inserted for refueling.

[0013] Preferably, in the above-mentioned method for estimating the posture of the fuel cap of an automatic refueling robot, the robotic arm is a six-axis robotic arm, and the binocular camera is installed at the fourth axis.

[0014] Preferably, in the above-mentioned method for estimating the posture of the fuel cap of an automatic refueling robot, the specific steps of loading the hand-eye calibration parameters are as follows:

[0015] Use the DH parameters provided by the robotic arm to solve the pose data of the fourth axis end and obtain the hand-eye calibration matrix B between the camera and the fourth axis end;

[0016] Use coordinate transformation to obtain the position transformation relationship of the camera in the robot arm base coordinate system;

[0017]

[0018] in is the pose matrix of the camera in the manipulator base coordinate system, is the pose matrix of the fourth-axis end of the manipulator in the manipulator base coordinate system, It is the pose matrix of the camera in the coordinate system of the fourth axis end of the manipulator.

[0019] Preferably, in the above-mentioned method for estimating the posture of the fuel tank cap of an automatic refueling robot, the specific steps of calculating the posture information of the suction point located at the center of the fuel tank outer cover are as follows:

[0020] Determine the center point of the fuel tank cover as feature point P; identify the fuel tank cover, fit a closed contour of the fuel tank cover, and obtain the center point of the closed contour, which is the center point of the fuel tank cover. Apply this method to both the left and right images, then perform point matching to obtain the world coordinates of the point.

[0021] The center point of the fuel tank cover is imaged as P in the left and right cameras of the binocular camera respectively. L 、P R , where P L The x-axis coordinate of the point is X L , P R The x-axis coordinate of the point is X R ;

[0022] The parallax is d = |X L -X R |;O L With O R represents the optical center of the two cameras. The line connecting the optical centers is called the baseline, and its distance is b; f is the focal length of the two cameras; the distance between point P and the baseline is It is called the depth information of point P;

[0023] use Z=Z is calculated to obtain the world coordinates (X, Y, Z) of point P;

[0024] Where x is the x-axis coordinate of the imaging point of point P in the camera, y is the y-axis coordinate of the imaging point of point P in the camera, f is the focal length of the camera, x0 is the x-axis offset value of the camera's pixel coordinate system, and y0 is the y-axis offset value of the camera's pixel coordinate system.

[0025] Preferably, in the above-mentioned method for estimating the posture of the fuel cap of an automatic refueling robot, the specific steps of calculating the posture information of the inner cover handle grasping point are as follows:

[0026] First, identify the inner lid handle in the left and right images. Detect two parallel lines on the inner lid handle area. Select the center point of the two parallel lines in the left view. Then, based on the principle of binocular camera stereo matching, select the corresponding matching point in the right view.

[0027] The overlap between the edge and the parallel line is determined by detecting the edges near the parallel line. If the overlap between the edge and the parallel line pixels is high, the parallel line is considered accurate enough and the matching point on the parallel line is selected. Otherwise, the parallel line is moved closer to the edge line. If the overlap is greater than the threshold, the move is considered successful and the corresponding point on the parallel line after moving closer is selected. (The distance between the corresponding pixel points on the edge and the parallel line is calculated. If the distance is less than the set threshold, the "corresponding points" are considered "same points". The more "same points" there are, the higher the overlap.)

[0028] The matching points are imaged as P in the left and right cameras of the binocular camera respectively. L 、P R , where P L The x-axis coordinate of the point is X L , P R The x-axis coordinate of the point is X R ;

[0029] The parallax is d = |X L -X R |;O L With O R represents the optical center of the two cameras. The line connecting the optical centers is called the baseline, and its distance is b; f is the focal length of the two cameras; the distance between point P and the baseline is It is called the depth information of point P;

[0030] use Z=Z is calculated to obtain the world coordinates (X, Y, Z) of point P;

[0031] Where x is the x-axis coordinate of the imaging point of point P in the camera, y is the y-axis coordinate of the imaging point of point P in the camera, f is the focal length of the camera, x0 is the x-axis offset value of the camera's pixel coordinate system, and y0 is the y-axis offset value of the camera's pixel coordinate system.

[0032] Preferably, in the above-mentioned method for estimating the position and posture of the fuel tank cap of an automatic refueling robot, the specific steps of identifying the outer cover of the fuel tank and identifying the inner cover handle are as follows:

[0033] Construct a dataset and use the ZED camera to capture 1,000 images of the fuel tank cover and the inner fuel tank cover. Then use the LabelMe software to annotate the positions of the fuel tank cover and the inner fuel tank cover.

[0034] For model training, select a recognition model such as YOLOv4, YOLOv5, Nanodet, or MobileNetSSD. For inner lid handle recognition, select a network model such as Yolact, Yolact++, or Mask RCNN. Use the labeled dataset to train the model network, enabling it to learn to intelligently recognize the outer and inner lid handles of the fuel tank.

[0035] Model deployment: Use OpenCV deep learning module, NCNN, and TensorRT network inference framework to deploy the model on the Jetson embedded control board, and use the onboard GPU for inference acceleration.

[0036] It can be seen from the above technical solution that compared with the existing technology, the present invention discloses a method for estimating the posture of the fuel cap of an automatic refueling robot, which uses deep learning technology to realize intelligent perception in the automatic refueling scenario, without the need for tedious steps such as manual entry of information in advance and vehicle library matching; it adopts a solution combining binocular stereo matching and visual servoing to ensure the high precision requirements of automatic refueling perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0038] Figure 1 is a flow chart of the method of the present invention;

[0039] Figure 2 This is a schematic diagram of hand-eye calibration of the present invention;

[0040] Figure 3 This is a flow chart of the hand-eye calibration of the present invention;

[0041] Figure 4 This is a flow chart of the method for stereo matching of the center point of the fuel tank cover of the present invention;

[0042] Figure 5 This is a flow chart of the three-dimensional matching method of the fuel tank inner cover handle of the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] The embodiment of the present invention discloses a method for estimating the position and posture of the fuel cap of an automatic refueling robot. Figure 1 As shown, the specific steps include:

[0045] The S101 main control board establishes a TCP connection with the robotic arm, initializes the binocular camera connection, loads the camera's internal parameters and hand-eye calibration parameters, and initializes the recognition program;

[0046] S102: Acquire at least two images of the area of ​​interest, identify the fuel tank cover in each image, calculate the position information of the suction point located at the center of the fuel tank cover, and the main control board sends the position information to the robotic arm, which then sucks the cover open.

[0047] S103: Turn on the bar light source, obtain at least two images of the second area of ​​interest, identify the inner cover handle in each image, calculate the position information of the inner cover handle grasping point, and send the position information to the robot arm, which moves the gripper to the grasping point.

[0048] S104 determines the relative position relationship between the gripping point of the manipulator arm's gripper and the inner cover of the refueling tank. When the gripping direction of the manipulator arm's gripper is consistent with the inner cover handle, the manipulator arm unscrews the inner cover of the refueling tank to refuel.

[0049] Through the above technical solution, this embodiment proposes a solution combining passive binocular stereo vision with visual servoing. Active binocular vision is different from passive binocular vision. It mainly refers to the visual imaging process that uses the round-trip time of optical signals such as laser or infrared light to perceive the depth information of the environment. Gas stations are generally located in open and unobstructed scenes due to explosion-proof requirements. Therefore, the ambient light interference is extremely large. The active binocular vision using optical signals is easily interfered by the ambient light and has weak environmental perception ability. Therefore, the present invention uses passive binocular stereo vision to perceive environmental information and uses stereo matching technology to obtain the posture information of the gas cap grasping point; the passive binocular camera uses traditional The camera imaging principle obtains the image within the field of view. For cameras with limited resolution, it is difficult to ensure the accuracy of target imaging. At the same time, there are matching errors in the stereo matching process, making it difficult to obtain the true position information of the target. Therefore, the present invention adds a visual servoing method. The passive binocular camera is responsible for the "coarse positioning" part of the early perception task, that is, the process of calculating the position information of the grasping point through the binocular stereo matching method. The visual servo performs "fine positioning" on the basis of "coarse positioning", that is, the process of guiding the robotic arm to move to the grasping point position through visual servo technology, ensuring that the robotic arm moves to the grasping point position of the fuel cap so that the fuel cap can be unscrewed and the fuel gun can be inserted for refueling.

[0050] In order to further optimize the above technical solution, the robotic arm is a six-axis robotic arm, and the binocular camera is installed on the fourth axis.

[0051] It should be noted that: the six-axis robot arm, as the name implies, has six rotatable joints, and the rotation of the six joint axes is driven directly by six servo motors through reducers, synchronous pulleys, etc. Six axes are marked from J1 to J6, representing the first axis, the second axis...the sixth axis of the six-axis robot arm. Generally speaking, tools such as cameras and grippers are installed at the end of the sixth axis (that is, the very end of the robot arm). However, in the automatic refueling scenario, the refueling cap needs to be unscrewed at the very end, so the sixth axis of the robot arm will carry the gripper to rotate multiple times to unscrew the refueling cap. If the camera is installed on the sixth axis of the robot arm, the camera power cord and data cable will be entangled. Therefore, in this embodiment, the camera is installed on the fourth axis to avoid the occurrence of wire entanglement problems.

[0052] Furthermore, it's important to understand that after the binocular camera is mounted on the fourth axis, the binocular camera and the sixth axis are not fixed. This is because the binocular camera on the fourth axis is fixed to the fourth axis, but the fifth and sixth axes rotate, so the relative position of the binocular camera and the fifth and sixth axes will change. This problem of the binocular camera and the sixth axis not being fixed will affect the hand-eye calibration task.

[0053] Specifically, the binocular camera is mounted on the fourth axis of the robotic arm. The binocular camera moves with the movement of the robotic arm. Therefore, it can be obtained that the camera and the fourth axis are fixed, so the relative position of the camera and the end of the fourth axis is determined. The pose matrix (including translation and rotation) of the binocular camera and the end of the fourth axis of the robotic arm can be obtained through hand-eye calibration. Therefore, the premise of hand-eye calibration is that the camera and the calibrated axis are fixed and there is no change in pose. Hand-eye calibration uses a checkerboard calibration plate, which is placed in a fixed position.

[0054] First, let me introduce Figure 2 The meaning of each matrix in is:

[0055] A: The position of the fourth axis end of the robot arm in the robot arm base coordinate system can be calculated using the DH parameters of the robot arm.

[0056] B: The position of the binocular camera in the fourth-axis end coordinate system of the robot arm.

[0057] C: The position of the binocular camera in the calibration plate coordinate system can be obtained through camera calibration.

[0058] D: The pose of the camera in the robot arm base coordinate system.

[0059] The task of the hand-eye calibration of the present invention is to obtain the pose transformation matrix B between the binocular camera and the fourth axis end of the robotic arm, such as Figure 3 As shown in the figure, the robot carries a camera to collect the image of the checkerboard calibration plate in the camera, and at the same time obtains the end pose data of the fourth axis of the robot (i.e. Figure 2 The A matrix in the image is used to recapture the image and obtain the four-axis pose data of the robot arm. Repeat the acquisition 10 times to obtain 10 sets of images and four-axis pose data of the robot arm. Use 10 sets of checkerboard images to calibrate the camera and obtain 10 sets of external parameters, that is, 10 sets Figure 2 The C matrix in . Using the correspondence between 10 sets of A matrices and C matrices, we can get the following formula, and use OpenCV to solve and get the hand-eye calibration matrix B;

[0060]

[0061] In this embodiment, the specific steps of using the coordinate transformation relationship to calculate the camera's position and posture in the base coordinate system are as follows:

[0062] Use the DH parameters provided by the robotic arm to solve the pose data of the fourth axis end and obtain the hand-eye calibration matrix B between the camera and the fourth axis end;

[0063] Use coordinate transformation to obtain the position transformation relationship of the camera in the robot arm base coordinate system;

[0064]

[0065] in is the pose matrix of the camera in the manipulator base coordinate system, is the pose matrix of the fourth-axis end of the manipulator in the manipulator base coordinate system, It is the pose matrix of the camera in the coordinate system of the fourth axis end of the manipulator.

[0066] In order to further optimize the above technical solution, Figure 4 As shown in the figure, the specific steps for calculating the pose information of the suction point located at the center of the fuel tank cover are as follows:

[0067] Determine the center point of the fuel tank cover as feature point P; identify the fuel tank cover, fit a closed contour of the fuel tank cover, and obtain the center point of the closed contour, which is the center point of the fuel tank cover. Apply this method to both the left and right images, then perform point matching to obtain the world coordinates of the point.

[0068] The center point of the fuel tank cover is imaged as P in the left and right cameras of the binocular camera respectively. L 、P R , where P L The x-axis coordinate of the point is X L , P R The x-axis coordinate of the point is X R ;

[0069] The parallax is d = |X L -X R |;O L With O R represents the optical center of the two cameras. The line connecting the optical centers is called the baseline, and its distance is b; f is the focal length of the two cameras; the distance between point P and the baseline is It is called the depth information of point P;

[0070] use Z=Z is calculated to obtain the world coordinates (X, Y, Z) of point P;

[0071] Where x is the x-axis coordinate of the imaging point of point P in the camera, y is the y-axis coordinate of the imaging point of point P in the camera, f is the focal length of the camera, x0 is the x-axis offset value of the camera's pixel coordinate system, and y0 is the y-axis offset value of the camera's pixel coordinate system.

[0072] In order to further optimize the above technical solution, Figure 5 As shown in the figure, the specific steps for calculating the pose information of the inner cover handle grasping point are as follows:

[0073] First, identify the inner lid handle in the left and right images. Detect two parallel lines on the inner lid handle area. Select the center point of the two parallel lines in the left view. Then, based on the principle of binocular camera stereo matching, select the corresponding matching point in the right view.

[0074] The overlap between the edge and the parallel line is determined by detecting the edges near the parallel line. If the overlap between the edge and the parallel line pixels is high, the parallel line is considered accurate enough and the matching point on the parallel line is selected. Otherwise, the parallel line is moved closer to the edge line. If the overlap is greater than the threshold, the move is considered successful and the corresponding point on the parallel line after moving closer is selected. (The distance between the corresponding pixel points on the edge and the parallel line is calculated. If the distance is less than the set threshold, the "corresponding points" are considered "same points". The more "same points" there are, the higher the overlap.)

[0075] The matching points are imaged as P in the left and right cameras of the binocular camera respectively. L 、P R , where P L The x-axis coordinate of the point is X L , P R The x-axis coordinate of the point is X R ;

[0076] The parallax is d = |X L -X R |;O L With O R represents the optical center of the two cameras. The line connecting the optical centers is called the baseline, and its distance is b; f is the focal length of the two cameras; the distance between point P and the baseline is It is called the depth information of point P;

[0077] use Z=Z is calculated to obtain the world coordinates (X, Y, Z) of point P;

[0078] Where x is the x-axis coordinate of the imaging point of point P in the camera, y is the y-axis coordinate of the imaging point of point P in the camera, f is the focal length of the camera, x0 is the x-axis offset value of the camera's pixel coordinate system, and y0 is the y-axis offset value of the camera's pixel coordinate system.

[0079] In order to further optimize the above technical solution, the specific steps for identifying the fuel tank outer cover and the inner cover handle are as follows:

[0080] Construct a dataset and use the ZED camera to capture 1,000 images of the fuel tank cover and the inner fuel tank cover. Then use the LabelMe software to annotate the positions of the fuel tank cover and the inner fuel tank cover.

[0081] For model training, select one of the following recognition models: YOLOv4, YOLOv5, Nanodet, or MobileNetSSD. For inner lid handle recognition, select one of the following network models: Yolact, Yolact++, or Mask RCNN. Use the labeled dataset to train the model network, enabling it to learn to intelligently recognize the outer and inner lid handles of the fuel tank.

[0082] Model deployment: Use OpenCV deep learning module, NCNN, and TensorRT network inference framework to deploy the model on the Jetson embedded control board, and use the onboard GPU for inference acceleration.

[0083] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0084] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for estimating the position and posture of a fuel cap of an automatic refueling robot, characterized in that: The specific steps include: The main control board establishes a TCP connection with the robotic arm, initializes the binocular camera connection, loads the camera's internal parameters and hand-eye calibration parameters, and initializes the recognition program; the robotic arm is a six-axis robotic arm, and the binocular camera is installed on the fourth axis; Obtain at least two images of the area of ​​interest, identify the fuel tank cover in each image, and calculate the position information of the suction point located at the center of the fuel tank cover. The main control board sends this position information to the robotic arm, which then sucks the cover open. Acquire at least two images of the second region of interest, identify the inner lid handle in each image, calculate the pose information of the inner lid handle grasping point, and the main control board sends the pose information to the robotic arm, which moves the gripper to the grasping point. The specific steps for calculating the pose information of the inner lid handle grasping point are as follows: first, identify the inner lid handle in the left and right images, detect two parallel lines on the inner lid handle in the inner lid handle area, select the center point of the two parallel lines in the left view, and then select the corresponding matching point in the right view based on the stereo matching principle of the binocular camera; By detecting the edges near the parallel lines, the overlap between the edge lines and the parallel lines is determined. If the overlap between the edge and the parallel line pixels is high, the parallel lines are considered accurate enough and the matching points on the parallel lines are taken. Otherwise, the parallel lines are moved closer to the edge lines. If the overlap is greater than the threshold, the move is considered successful and the corresponding points on the parallel lines after moving closer are taken. Determine the relative position relationship between the gripping point of the robotic arm's gripper and the inner cover of the refueling tank. When the gripper of the robotic arm is in the same direction as the inner cover handle, the robotic arm unscrews the inner cover of the refueling tank for refueling.

2. The method for estimating the position and posture of the fuel cap of an automatic refueling robot according to claim 1, characterized in that: The specific steps of loading the hand-eye calibration parameters are as follows: Use the DH parameters provided by the robotic arm to solve the pose data of the fourth axis end and obtain the hand-eye calibration matrix B between the camera and the fourth axis end; Use coordinate transformation to obtain the position transformation relationship of the camera in the robot arm base coordinate system; in is the pose matrix of the camera in the manipulator base coordinate system, is the pose matrix of the fourth-axis end of the manipulator in the manipulator base coordinate system, It is the pose matrix of the camera in the coordinate system of the fourth axis end of the manipulator.

3. The method for estimating the position and posture of the fuel cap of an automatic refueling robot according to claim 1, It is characterized by: The specific steps to calculate the pose information of the suction point located at the center of the fuel tank cover are as follows: Determine the center point of the fuel tank cover as the feature point P; The center point of the fuel tank cover is imaged as P in the left and right cameras of the binocular camera respectively. L 、P R , where P L The x-axis coordinate of the point is X L , P R The x-axis coordinate of the point is X R ; The parallax is d = |X L -X R |; The distance between point P and the baseline is It is called the depth information of point P; b is the baseline distance between the optical centers of the left and right cameras; use Z=Z is calculated to obtain the world coordinates (X, Y, Z) of point P, where x is the x-axis coordinate of the imaging point of point P in the camera, y is the y-axis coordinate of the imaging point of point P in the camera, f is the focal length of the camera, x0 is the x-axis offset value of the camera's pixel coordinate system, and y0 is the y-axis offset value of the camera's pixel coordinate system.

4. The method for estimating the position and posture of the fuel cap of an automatic refueling robot according to claim 1, characterized in that: The specific steps for calculating the matching point coordinates are as follows: The matching point P' is imaged as P' in the left and right cameras of the binocular camera respectively. L , P′ R , where P′ L The x-axis coordinate of the point is X′ L , P′ R The x-axis coordinate of the point is X′ R ; The parallax is d′=|X′ L -X′ R |; The distance between point P′ and the baseline is It is called the depth information of point P'; b is the baseline distance between the optical centers of the left and right cameras; use Z′=Z′ is calculated to obtain the world coordinates (X′, Y′, Z′) of point P′; where x′ is the x-axis coordinate of the imaging point of point P′ in the camera, y′ is the y-axis coordinate of the imaging point of point P′ in the camera, f is the focal length of the camera, x0 is the x-axis offset value of the camera's pixel coordinate system, and y0 is the y-axis offset value of the camera's pixel coordinate system.

5. The method for estimating the position and posture of the fuel cap of an automatic refueling robot according to claim 1, characterized in that: The specific steps to identify the fuel tank outer cover and the inner cover handle are as follows: Construct a dataset and use the ZED camera to capture 1,000 images of the fuel tank cover and the inner fuel tank cover. Then use the LabelMe software to annotate the positions of the fuel tank cover and the inner fuel tank cover. For model training, a recognition model is selected, such as YOLOv4, YOLOv5, Nanodet, or MobileNetSSD. For inner lid handle recognition, a network model such as Yolact, Yolact++, or Mask RCNN is selected. The model network is trained using a labeled dataset to enable the model network to learn to intelligently recognize the outer and inner lid handles of the fuel tank. Model deployment: Use OpenCV deep learning module, NCNN, and TensorRT network inference framework to deploy the model on the Jetson embedded control board, and use the onboard GPU for inference acceleration.

Images