Tank opening point pose estimation method based on refueling robot

Abstract

The application discloses a fuel tank cover opening point pose estimation method based on a refueling robot, and aims to solve the problem that the existing technology is affected by the vehicle body angle and light, and the pose estimation has a large error. The passive binocular photographing mode is adopted to avoid the problem that the existing active binocular photographing mode is easily disturbed by environmental light in an outdoor scene and the fuel tank cover data is difficult to obtain due to black reflection. The initial photographing position and the vehicle body fuel tank side multi-angle photographing position are adopted, so that the problems of visual blind area and unclear edge in single-angle photographing are avoided. Through point cloud fusion and point cloud reduction, the superposition of multi-angle effective point clouds and the elimination of invalid data are realized, so that the pose calculation is more accurate. The RANSAC plane fitting and point cloud mean can effectively eliminate the interference of point cloud noise, and the fuel tank cover opening point pose is efficiently and stably obtained.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of automatic refueling of energy, in particular to an oil tank cover opening pose estimation method based on a refueling robot. BACKGROUND

[0002] Automobile refueling stations are an important part of the transportation industry. Currently, two refueling modes are commonly used in refueling stations: one is operated by full-time refueling personnel, which requires corresponding labor costs; the other is a self-service refueling mode, in which the driver operates the oil gun to add fuel. Since the driver is generally unfamiliar with the refueling machine and lacks relevant operation professional knowledge, it is easy to cause certain oil leakage and pose a safety hazard.

[0003] In recent years, intelligent robots have developed rapidly due to the development and combination of artificial intelligence and robot technology. Intelligent production modes dominated by robots gradually replace traditional production modes dominated by manpower. As a result, refueling stations have developed a new service mode - automatic refueling by refueling robots. The refueling robot integrates control systems, robot technology and sensor technology, and can complete the recognition and spatial positioning of the automobile fuel tank cover, fuel tank cap and fuel tank mouth, and complete the refueling process.

[0004] One of the key technologies of the refueling robot is the pose estimation of the vehicle fuel tank cover. The pose estimation of the fuel tank cover known to the inventors includes the following schemes: (1) a binocular camera is used to take pictures of the vehicle, and the fuel tank cover pose is calculated through feature matching; (2) a grating structured light camera is used to scan the three-dimensional data of the vehicle, and then the pose is calculated; (3) a line laser is used to scan the three-dimensional data of the vehicle body, and then the fuel tank cover pose is calculated.

[0005] However, the inventors have found that the above-mentioned technologies have the following technical problems in the process of implementing the technical solutions in the embodiments of the application: (1) the scheme based on a binocular camera is easily disturbed by the vehicle body angle, paint reflection, etc., which affects the pose estimation due to inaccurate binocular matching; (2) the scheme based on grating structured light is affected by strong light outdoors and cannot obtain complete three-dimensional data, which affects the pose estimation; (3) the scheme based on line laser is easily affected by the reflection of black shiny paint, which cannot obtain accurate three-dimensional data, which affects the pose estimation.

[0006] The information disclosed in this BACKGROUND section is only for the purpose of enriching the understanding of the background of the present disclosure and should not be considered as acknowledging or implying in any form that this information constitutes prior art known to those skilled in the art. SUMMARY

[0007] In view of at least one of the above technical problems, the present disclosure provides an oil tank cover opening point pose estimation method based on a refueling robot, aiming to solve the problem that the existing technology is affected by the vehicle body angle and light, and there is a large error in pose estimation.

[0008] According to one aspect of the present disclosure, an oil tank cover opening point pose estimation method for an automatic refueling robot is provided, comprising the following steps:

[0009] (1) An initial oil tank cover image of a vehicle parked at a corresponding vehicle parking position in a refueling area is collected by a binocular camera arranged at a position corresponding to a mechanical arm of the refueling robot;

[0010] (2) The oil tank cover image is sequentially subjected to region segmentation, edge extraction and edge screening processing to obtain left and right edge contour maps of the oil tank cover;

[0011] (3) The left and right edge contour maps of the oil tank cover are sequentially subjected to stereo correction, matching cost calculation, disparity calculation and 3D point cloud calculation to obtain an oil tank cover point cloud PC_Cam, and the oil tank cover opening pose is calculated:

[0012] (31) Plane fitting point cloud PC_Cam, three point positions in the point cloud PC_Cam are selected, and the plane equation is calculated: ;

[0013] (32) The algebraic distance of all points in PC_Cam to the plane equation is calculated ;

[0014] (33) A threshold value is selected, and the points corresponding to are recorded as in-model sample points, and the points corresponding to are recorded as out-of-model sample points, and the number of in-model sample points is counted; wherein ;

[0015] (34) The steps (31) and (32) are repeated, and the model parameters corresponding to the plane with the largest number of in-model sample points are selected as the final oil tank cover plane parameters, and the parameters correspond to the plane point cloud PC_Cap with a normal vector of ;

[0016] (35) Let the oil tank cover direction vector be , and define equal to the y-axis of the camera coordinate system, and ; get the mutually orthogonal oil tank cover direction vectors ;

[0017] (36) Along Find the minimum and maximum values in the point cloud PC_Cap and maximum values ; determine the relationship between the opening position of the fuel tank cover and the direction according to the vehicle type, if the opening position of the fuel tank cover is in the positive direction, then the opening point ,

[0018]

[0019]

[0020]

[0021] if the opening position of the fuel tank cover is in the negative direction, then:

[0022]

[0023]

[0024]

[0025] (37) Calculate the pose of the opening point of the fuel tank cover according to the direction vector of the fuel tank cover and the opening point ; wherein ; ;

[0026] (4) According to the pose of the opening point of the fuel tank cover , move the binocular camera a certain distance to the side of the vehicle body, and collect images at different positions on the side of the vehicle body, and record the flange pose corresponding to the shooting position;

[0027] (5) Process each image collected in step (4) according to steps (2)-(3) to obtain each point cloud, then perform point cloud fusion and point cloud reduction to obtain point cloud , and calculate the final opening point pose according to steps (35)-(37).

[0028] In some embodiments of the present disclosure, in the step (1), after camera calibration of the binocular camera, the binocular camera is installed at the end of a six-axis mechanical arm and hand-eye calibration is performed between the binocular camera and the mechanical arm; and the binocular camera is arranged at the middle position of the vehicle parking position, and the distance from the vehicle parking position is any value in the range of 1000mm-1500mm.

[0029] ​​​In some embodiments of the present disclosure, in the step (2), the region segmentation employs any one of Faster R-CNN, YOLO, and R-FCN network model to identify and segment the fuel tank cap image, to obtain the small images capImgL and capImgR of the fuel tank cap; and the edge extraction employs Canny edge extraction method to extract the left and right edge contour lines edgeL and edgeR of the fuel tank cap.

[0030] In some embodiments of the present disclosure, in the step (2), the edge screening includes the following steps:

[0031] (21) traversing the edge contour lines edgeL and edgeR, and calculating the connected domain area S of each edge contour line;

[0032] (22) sorting to obtain the maximum value of the connected domain area ;

[0033] (23) defining a threshold value ;

[0034] (24) deleting the connected domain with an area less than the threshold value N, to obtain the edge contour contourL and contourR.

[0035] In some embodiments of the present disclosure, in the step (34), the steps (31) and (32) are repeated more than 1000 times.

[0036] In some embodiments of the present disclosure, in the step (37), the method further includes coordinate system conversion of the fuel tank cap opening point pose : defining the base coordinate system of the mechanical arm as the system base coordinate system, then , wherein is the fuel tank cap opening point pose in the system base coordinate system; is the conversion relationship from the flange of the mechanical arm to the system base coordinate system; is the conversion relationship between the binocular camera and the flange of the mechanical arm.

[0037] In some embodiments of the present disclosure, in the step (4), the binocular camera moves 500 mm along the z-axis positive direction of the fuel tank cap opening point pose, and image acquisition is performed at three positions corresponding to the front, middle and rear of the vehicle body fuel tank respectively, and the conversion relationship of the flange pose corresponding to each shooting position relative to the base coordinate is , , .

[0038] In some embodiments of the present disclosure, in the step (5), the three groups of images corresponding to the front, middle and rear of the body oil tank are respectively converted into plane point clouds PC_cap1, PC_cap2 and PC_cap3 after the processing in the step (3), and the point cloud fusion comprises the following steps:

[0039] (511) respectively obtaining the coordinate conversion relationship of the photographing position 1 corresponding to the front, the photographing position 3 corresponding to the rear to the photographing position 2 corresponding to the middle part of the vehicle 、 :

[0040] ;

[0041] ;

[0042] Among them, 、 、 respectively correspond to the conversion relationship between the binocular camera and the mechanical arm flange at different image acquisition positions;

[0043] (512) converting PC_cap1 and PC_cap3 to the coordinate system of PC_cap2 according to the coordinate conversion relationship 、 ;

[0044] (513) merging each plane point cloud into a point cloud .

[0045] In some embodiments of the present disclosure, in the step (5), the point cloud reduction comprises the following steps:

[0046] (521) setting a matrix , respectively obtaining the mean value by column, and letting ;

[0047] (522) obtaining the covariance matrix Cov of the matrix , wherein n is the number of three-dimensional points in the point cloud; ;

[0048] (523) calculating the eigenvalue and eigenvector of cov;

[0049] (524) sorting the eigenvalues in descending order, selecting the largest k, and taking the k eigenvectors corresponding to the k largest eigenvalues as row vectors to form an eigenvector matrix P;

[0050] (525) converting the point cloud into a two-dimensional space Y constructed by the eigenvector matrix P, i.e. ;

[0051] (526)After the Y is three-dimensionally processed .

[0052] In some embodiments of the present disclosure, in the step (524), the value of k is 20% of the number of feature points satisfying the condition.

[0053] The one or more technical solutions provided in the embodiments of the present application have at least any of the following technical effects or advantages:

[0054] 1. The passive binocular photographing mode is adopted, which avoids the problem that the existing active binocular photographing mode needs to use a structured light source such as a grating or a line laser to irradiate the measured object, thereby causing environmental light interference in an outdoor scene and difficulty in obtaining data of a black reflective fuel tank cover, effectively improving the anti-interference of environmental light and the reliability of data acquisition, and providing accurate data support for pose estimation.

[0055] 2. The three-dimensional data of the vehicle fuel tank cover is obtained from different angles by using the initial photographing position and the multi-angle photographing position of the vehicle body fuel tank side, thereby avoiding the problems of visual blind area and unclear edge caused by photographing from a single angle, and improving the robustness of the fuel tank cover detection.

[0056] 3. The superposition of effective point clouds from multiple angles and the elimination of invalid data are realized by point cloud fusion and point cloud reduction, which maximizes the retention of effective information from each angle and deletes invalid information, making the pose calculation more accurate.

[0057] 4. The RANSAC plane fitting and point cloud mean can effectively eliminate the interference of point cloud noise, and efficiently and stably obtain the fuel tank cover opening point pose. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 The figure is a flowchart of the fuel tank opening point pose estimation method in an embodiment of the present application.

[0059] Figure 2 The figure is a schematic diagram of the fuel tank cover region segmentation in an embodiment of the present application.

[0060] Figure 3 The figure is a schematic diagram of the fuel tank cover edge extraction in an embodiment of the present application.

[0061] Figure 4 The figure is a schematic diagram of the multi-angle photographing point position in an embodiment of the present application.

[0062] Wherein, 1 is the initial photographing position, 21 is the photographing position 1, 22 is the photographing position 2, and 23 is the photographing position 3. DETAILED DESCRIPTION

[0063] The programs involved or relied on in the following embodiments are all conventional programs or simple programs in the technical field, and those skilled in the art can make routine selection or adaptive adjustment according to the specific application scenarios.

[0064] In order to better understand the technical solutions of the present application, the above technical solutions will be described in detail below in combination with the drawings of the specification and specific embodiments.

[0065] The example discloses a tank cover opening pose estimation method based on a refueling robot, referring to Figure 1 , comprising the following steps:

[0066] (1) An initial tank cover image of a vehicle parked in a corresponding parking position in a refueling area is collected by a binocular camera arranged at a position corresponding to a mechanical arm of the refueling robot.

[0067] In this embodiment, the binocular camera is fixed at the end of the six-axis mechanical arm, and the photographing position of the binocular camera is controlled by the mechanical arm. Due to optical imaging problems, the camera has pillow-shaped distortion and barrel-shaped distortion, the pixel size of the camera is not in a proportional relationship with the actual object size, and there is a pose deviation between the installation position of the camera and the actual imaging plane. Therefore, the camera is calibrated before use, and hand-eye calibration is performed with the mechanical arm to obtain the conversion relationship of the coordinates between the pixel coordinate system in the camera image and the three-dimensional coordinate system of the mechanical arm, so as to accurately solve the three-dimensional coordinates of the target in the mechanical arm coordinate space through the pixel coordinates and the distance to the target. In this example, in order to enable the binocular camera to effectively capture the tank image of the vehicle, and considering that there is a difference between the parking positions of each vehicle at the refueling vehicle parking position, the initial position of the binocular camera is adjusted to be arranged at the middle position of the vehicle parking position, and the distance between the initial position of the binocular camera and the vehicle parking position is 1300mm. In other embodiments, the distance between the initial position of the binocular camera and the vehicle parking position is any value in the range of 1000mm-1500mm. After the refueling robot detects that the vehicle is parked and stable in the parking area, the binocular camera is started to take pictures and obtain the initial tank cover image.

[0068] (2) The tank cover image is sequentially subjected to region segmentation, edge extraction, and edge screening processing to obtain the left and right edge contour map of the tank cover.

[0069] Since the picture taken by the binocular camera contains a large number of non-tank cover regions, in order to avoid the influence of non-tank cover regions on pose determination, in this example, the Faster R-CNN model is used for region segmentation of the initial tank cover image, and only the tank cover region in the image is retained. In other embodiments, any one of the YOLO and R-FCN network models is used to identify and segment the tank cover image. Since the binocular camera takes pictures through left and right two cameras, two images are obtained at the same time, referring toFigure 2 Therefore, the initial fuel tank cap image is regionally segmented to obtain the fuel tank cap sub-images capImgL and capImgR.

[0070] The fueling robot cannot directly observe the specific position of the fuel tank cap from the fuel tank cap sub-images capImgL and capImgR, and therefore, further processing is required on the fuel tank cap sub-images capImgL and capImgR so that the fueling robot can identify the edge of the fuel tank cap therefrom. In this example, the Canny edge extraction method is used to extract the fuel tank profile lines in the fuel tank cap sub-images capImgL and capImgR, respectively. After image noise reduction by a Gaussian filter, calculation of the gradient and direction of each pixel, non-maximum suppression, and setting of high and low thresholds to screen out binary images, the points with large changes in grayscale values in the images are detected, and these points are fitted into lines. The collection of lines is the edge of the image, as shown in FIG. 3. Figure 3 The left and right edge profile lines edgeL and edgeR of the fuel tank cap are further extracted from the fuel tank cap sub-images capImgL and capImgR.

[0071] Since the left and right edge profile lines edgeL and edgeR of the fuel tank cap are a collection of many lines, including false profile lines of the fuel tank cap caused by the influence of light and reflection, it is necessary to screen each profile line to exclude the interference of external environmental light. In this example, the edge screening includes the following steps:

[0072] (21) The connected domain area S of each edge profile line is calculated by traversing the edge profile lines edgeL and edgeR. Since the left and right edge profile lines edgeL and edgeR of the fuel tank cap extracted by the edge extraction include a plurality of edge profile lines, there are false profiles caused by the interference of external environmental light. Considering that the area between the true profile lines on both sides of the fuel tank cap is determined, the connected domain area S between each edge profile line and other edge profile lines is calculated by traversing each edge profile line edgeL and edgeR.

[0073] (22) The areas of each connected domain are sorted to obtain the maximum value of the connected domain area .

[0074] (23) The threshold value is defined.

[0075] (24) Considering the difference between the connected domain between the false profile line and the remaining profile lines and the connected domain between the true left and right profile lines of the fuel tank cap, each profile line is screened by setting the threshold value , and the connected domains with an area less than the threshold value N are deleted, thereby obtaining the edge profiles contourL and contourR.

[0076] (3) The left and right edge profile images of the fuel tank cap are sequentially subjected to stereo correction, matching cost calculation, disparity calculation, and 3D point cloud calculation to obtain the fuel tank cap point cloud PC_Cam, and the fuel tank cap opening pose is calculated.

[0077] Considering that the two camera image planes of the binocular camera are not completely coplanar, thereby interfering with the image disparity calculation, in order to make the subsequent stereo matching more reliable and the calculation more feasible, the left and right edge profile images of the fuel tank cap are subjected to stereo correction to make the left and right edge profile images of the fuel tank cap coplanar.

[0078] Since there is a certain disparity between the two images formed by the left and right cameras of the binocular camera, in order to measure the correlation between the candidate pixels and the matching pixels in the two images, matching cost calculation is required. In the left and right binocular images, the pixels with larger disparity are closer to the camera, and the pixels with smaller disparity are farther away from the camera. Therefore, according to the matching cost calculation, the horizontal distance between the two matching fast center pixels is calculated, the depth information of each pixel is obtained according to the corresponding relationship between the depth and the disparity, and the 3D point cloud PC_Cam of the fuel tank cap is calculated accordingly.

[0079] Then, according to the 3D point cloud PC_Cam data of the fuel tank cap, the fuel tank cap opening pose is calculated by the following steps:

[0080] (31) Plane fitting point cloud PC_Cam; according to the principle of determining a plane by three points, three points are selected in the point cloud PC_Cam, and the plane equation is calculated: .

[0081] (32) Calculate the algebraic distance of all other points in the point cloud PC_Cam to the above plane equation . .

[0082] (33) Select a threshold , and record the points corresponding to as in-sample points and the points corresponding to as out-of-sample points, and count the number of in-sample points; wherein , in this example, the threshold is selected as .

[0083] (34) Repeat steps (31) and (32) to select the model parameters corresponding to the plane with the most in-sample points as the final fuel tank cap plane parameters, which correspond to the plane point cloud PC_Cap with a normal of ; in this example, steps (31) and (32) are repeated 1000 times, and in some other embodiments, the iteration of steps (31) and (32) is performed more than 1000 times.

[0084] (35) Set the oil tank cover direction vector as , and define equal to the y-axis of the camera coordinate system, through ; get the mutually orthogonal oil tank cover direction vector .

[0085] (36) Find the minimum value and the maximum value in the point cloud PC_Cap along the direction; the refueling robot compares the detected vehicle type with the vehicle type database to determine the direction relationship between the longitudinal coordinate of the most suitable grabbing position of the different oil tank covers in the longitudinal direction and , if it is in the positive direction of , then the opening point ,

[0086]

[0087]

[0088]

[0089] if it is in the negative direction of :

[0090]

[0091]

[0092] .

[0093] (37) Calculate the oil tank cover opening point pose according to the oil tank cover direction vector and the opening point .

[0094] In addition, since the coordinates of the flange at the end of the mechanical arm can be directly read and the accuracy can be guaranteed, in this example, the oil tank cover opening point pose is subjected to coordinate conversion, and the base coordinate system of the mechanical arm is defined as the system base coordinate system, then , wherein is the oil tank cover opening point pose in the system base coordinate system; is the conversion relationship from the flange of the mechanical arm to the system base coordinate system; is the conversion relationship between the binocular camera and the flange of the mechanical arm; thus, the pose of the oil tank cover opening point of the refueling robot based on the flange of the mechanical arm is obtained.

[0095] (4) According to the oil tank cover opening point pose , the binocular camera is moved a distance to be close to the vehicle body, and image acquisition is performed at different positions on the oil tank side of the vehicle body, and the flange pose corresponding to the photographing position is recorded.

[0096] To avoid the problems of visual blind area or unclear edge in the initial oil tank cover image, in this example, the oil tank cover is photographed from multiple angles to obtain more comprehensive three-dimensional data of the vehicle oil tank cover, thereby improving the robustness of oil tank cover detection. Specifically, in this embodiment, according to the calculated oil tank cover opening point pose , the binocular camera is moved 500 mm along the positive direction of the z-axis of the oil tank cover opening point pose, that is, moved in the direction close to the vehicle body, see Figure 4 , and image acquisition is performed at three corresponding positions on the front, middle and rear of the oil tank side of the vehicle body, that is, photographing position 1, photographing position 2 and photographing position 3, and the transformation relationship of the flange pose corresponding to each photographing position relative to the base coordinate is recorded 、 、 . In other embodiments, at least three images are collected at different positions on the oil tank side of the vehicle body.

[0097] (5) The images collected in step (4) are respectively processed according to steps (2)-(3) to obtain point clouds, and then the point clouds are fused and reduced to obtain point cloud , and the final opening point pose is calculated according to steps (35)-(37) .

[0098] The three groups of images corresponding to the front, middle and rear of the oil tank side of the vehicle are respectively processed according to steps (2)-(3) to obtain plane point clouds PC_cap1, PC_cap2 and PC_cap3. To make full use of the effective information of each point cloud and make the images complement each other, point cloud fusion is performed, which includes the following steps:

[0099] (511) The coordinate transformation relationships of photographing position 1 corresponding to the front of the vehicle, photographing position 3 corresponding to the rear of the vehicle, and photographing position 2 corresponding to the middle of the vehicle are respectively calculated 、 :

[0100] ;

[0101] ;

[0102] Among them 、 、 respectively correspond to the transformation relationship between the binocular camera and the flange of the mechanical arm at different image collection positions.

[0103] (512) According to the coordinate transformation relationships 、 PC_cap1 and PC_cap3 are converted to the coordinate system of PC_cap2 respectively, and corresponding results are obtained , .

[0104] (513) merge the planar point clouds into a point cloud .

[0105] Thus, the merging of the point clouds is achieved, but there are erroneous point cloud data in the point cloud data formed by different images. Therefore, in this example, part of the erroneous data is removed by point cloud reduction relying on the common part of the three sets of point cloud data, which includes the following steps:

[0106] (521) set the matrix , and calculate the mean value of each column , let

[0107] (522) calculate the covariance matrix Cov of the matrix , where n is the number of three-dimensional points in the point cloud;

[0108] (523) calculate the eigenvalues and of cov, and the corresponding eigenvectors.

[0109] (524) sort the eigenvalues in descending order, select the largest k, and take the corresponding k eigenvectors as row vectors to form the eigenvector matrix P; in this example, the value of k is selected to be 20% of the number of feature points that meet the condition.

[0110] (525) convert the point cloud into the two-dimensional space Y constructed by the eigenvector matrix P, that is , thus realizing the dimension reduction processing of the data by principal component analysis, that is, converting the three-dimensional space data into two-dimensional plane data, and retaining the main features of the tank cover, so as to calculate the opening point pose.

[0111] (526) according to the corresponding feature points in the left and right images and the internal and external parameters of the binocular camera, calculate the spatial three-dimensional coordinates according to the parallax, that is, obtain .

[0112] Although some preferred embodiments of the present application have been described, those skilled in the art can make further changes and modifications to these embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0113] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the inventive spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for estimating the pose of a fuel tank opening point based on a refueling robot, characterized by, Comprise the following steps: (1) through the setting in the oiling robot corresponding mechanical arm position binocular camera, to parking in the corresponding vehicle parking area in the region of the vehicle, initial fuel tank cover image acquisition; (2) the fuel tank cover image is sequentially carried out region segmentation, edge extraction, edge screening processing, and the left and right edge contour of fuel tank cover is obtained; (3) the left and right edge contour of fuel tank cover is sequentially carried out stereo correction, matching cost calculation, disparity calculation, 3D point cloud calculation, and the fuel tank cover point cloud PC_Cam is obtained, and the fuel tank cover opening position is calculated: (31) plane fitting point cloud PC_Cam, optional three point in point cloud PC_Cam, calculate its plane equation: Ax+By+Cz=0;Wherein, (A, B, C) is the normal vector of plane fitting point cloud PC_Cam, (x, y, z) represents the point in point cloud PC_Cam; (32) Calculate the algebraic distance of all remaining points in PC_Cam to the plane equation x i , y i , z i ) to the plane equation d i | Ax i +By i + Cz i |; (33) Selecting a threshold value d th , record the corresponding d i d th The point is the in-model sample point, and the corresponding d i d th The point is the out-of-model sample point, and the number of in-model sample points is counted; wherein d th <0.5;​​ (34) repeating the step (31) and step (32), selecting the model parameters corresponding to the plane with the largest number of sample points in the model as the final tank cover plane parameters, which correspond to the normal of the plane point cloud PC_Cap. (35) Set the oil tank cover direction vector as , and define equal to the y-axis of the camera coordinate system, through ; get the mutually orthogonal oil tank cover direction vector ; (36) along the direction in the point cloud PC_Cap to find a minimum value P min and a maximum value P max ; according to the relationship between the opening position of the fuel tank cover and the direction, if the opening position of the fuel tank cover is located in the positive direction, the opening point P cap (0, 0, 0) P cap_x , P cap_y , P cap_z )​​ ； ； ； If the oil filler cap open position is located in the negative direction, then: ； ； ； wherein P min_x and P max_x represent the x-coordinates of the minima P min and the maxima P max respectively; P min_y and P max_y represent the y-coordinates of the minima P min and the maxima P max respectively; P min_z and P max_z represent the z-coordinates of the minima P min and the maxima P max respectively. (37) The tank flap direction vector according to the tank flap opening point and the tank flap opening point P cap calculating the tank flap opening point pose RT cap ; wherein ; (4) According to the oil tank cover opening point pose RT cap , the binocular camera is moved by a certain distance to be close to the vehicle body, image collection is performed at different positions of the vehicle body oil tank side, and the flange pose corresponding to the shooting position is recorded. (5) The images collected in step (4) are respectively processed by the steps (2)-(3), and after obtaining each point cloud, the point cloud fusion and point cloud reduction are performed to obtain the point cloud PC_final The final cover opening point pose is calculated according to the steps (35)-(37) RT final .

2. The oil tank uncapping point pose estimation method according to claim 1, characterized in that, In the step (1), the camera calibration is carried out on the binocular camera, which is installed at the end of the six-axis mechanical arm, and the hand-eye calibration is carried out between the binocular camera and the mechanical arm;And the binocular camera is arranged at the middle position of the vehicle parking position, and the distance from the vehicle parking position is any value in 1000mm-1500mm.

3. The oil tank uncapping point pose estimation method according to claim 1, characterized in that, In the step (2), the region segmentation adopts any one of Faster R-CNN, YOLO, R-FCN network model to identify and segment the fuel tank cover image, and the fuel tank cover small image capImgL and capImgR are obtained;The edge extraction adopts Canny edge extraction method, and the left and right edge contour lines edgeL and edgeR of fuel tank cover are obtained.

4. The oil tank uncapping point pose estimation method according to claim 3, characterized in that, In the step (2), the edge screening includes the following steps: (21) traverse the edge contour line edgeL and edgeR, calculate the connected domain area S of each edge contour line; (22) ranking to obtain a maximum value of the area of the connected domain S max ; (23) defining a threshold S th =0.5×S max ; (24) delete the connected domain with an area less than the threshold value N, and obtain the edge contour contourL and contourR.

5. The oil tank uncapping point pose estimation method of claim 1, wherein, In the step (34), the number of times of repeating the steps (31) and (32) is greater than 1000.

6. The oil tank uncapping point pose estimation method according to claim 1, characterized in that, In the step (37), the oil tank cover opening point pose is also included RT cap Coordinate system conversion: define the base coordinate system of the mechanical arm as the system base coordinate system, then Wherein, RT target The oil tank cover opening point pose in the system base coordinate system; The conversion relationship from the mechanical arm flange to the system base coordinate system; The conversion relationship between the binocular camera and the mechanical arm flange.

7. The oil tank uncapping point pose estimation method of claim 1, wherein, In the step (4), the binocular camera is moved 500 mm along the z-axis positive direction of the oil tank cover opening point pose, and image collection is performed at three positions corresponding to the front, middle and rear of the vehicle body oil tank, respectively. The conversion relationship of the flange pose H1, H2 and H3 corresponding to each photographing position relative to the base coordinate is , , .

8. The oil tank uncapping point pose estimation method of claim 7, wherein, In the step (5), three groups of images corresponding to the front, middle and rear of the vehicle body fuel tank are obtained respectively according to the step (3), and the point cloud fusion includes the following steps: (511) respectively obtain coordinate conversion relationships of the front of the vehicle corresponding to the photographing position 1, the rear of the vehicle corresponding to the photographing position 3, and the middle of the vehicle corresponding to the photographing position 2 , : ； ； wherein, , , respectively correspond to the conversion relationship between the binocular camera and the flange of the robot arm at different image acquisition positions, C1 represents the camera coordinate system corresponding to the first shooting position, C2 represents the camera coordinate system corresponding to the first shooting position, C3 represents the camera coordinate system corresponding to the first shooting position, H1 represents the flange coordinate system corresponding to the first shooting position, H2 represents the flange coordinate system corresponding to the second shooting position, and H3 represents the flange coordinate system corresponding to the third shooting position. (512) According to the coordinate conversion relationship , , convert PC_cap1 and PC_cap3 to the coordinate system of PC_cap2 respectively. (513) merge each plane point cloud into point cloud 。 9. The oil tank uncapping point pose estimation method of claim 8, wherein, In the step (5), the point cloud reduction includes the following steps: (521) Set matrix PC_capN [ x i y i z i ], take mean of each column X ave , Y ave , Z ave , let ; (522) Covariance matrix PC_new of the n 3D points in the point cloud; ; (523) calculate the eigenvalue and eigenvector of cov; (524) sort the eigenvalues in descending order, select the largest k, and take the corresponding k eigenvectors as row vectors to form the eigenvector matrix P; (525) Projecting the point cloud PC_new into the two-dimensional space Y constructed by the eigenvector matrix P, i.e. Y=P•PC_new ; (526)After the Y is three-dimensionally processed, the following is obtained PC_final .

10. The oil tank uncapping point pose estimation method of claim 9, wherein, In the step (524), the value of k is 20% of the number of feature points satisfying the condition.

Images