Positioning method and device for outer cover of automobile fuel tank and refueling system

Abstract

The invention discloses a method and a device for positioning an outer cover of an automobile fuel tank and a refueling system. The method and the device are used for realizing high-precision and high-stability positioning of the outer cover of the fuel tank in an automatic refueling process. The position and posture of the wheel hub are estimated through the deep learning algorithm and the binocular vision technology, and the posture information of the fuel tank outer cover relative to the base coordinate system of the automatic refueling robot is efficiently calculated in combination with the posture information between the fuel tank outer cover and the vehicle body coordinate system. And the positioning robustness and precision under complex working conditions such as illumination variation, oil stain coverage and vehicle type difference are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and device for positioning the outer cap of a car fuel tank for automatic refueling, as well as an automatic refueling system. Background Technology

[0002] Currently, most car refueling processes are performed manually, which is time-consuming and labor-intensive. Furthermore, the inability of humans to maintain high levels of concentration for extended periods increases the risk of accidents due to improper operation. In harsh environments, manual operation is unsuitable. As demands for improved working conditions increase, manufacturers are increasingly exploring and adopting new technologies. Robot research and development has been ongoing for many years, and its results have been applied to actual production. Currently, research is underway to introduce robotic technology into car refueling to achieve automated refueling, and a vision system to guide the robot is essential. For robotics, achieving high-precision positioning of the fuel tank cap in complex environments remains a challenging problem.

[0003] Chinese Patent Publication No. CN114644315A discloses a device and method for precisely positioning a fuel tank cap in an automatic refueling process. This device and method involves pre-setting brightly colored reference points around the refueling area and locating the fuel tank cap's position using these reference points and the vehicle's feature points. This unmanned refueling method requires pre-setting reference points and taking multiple images before refueling, resulting in a large processing load, complex operation, and reduced efficiency.

[0004] Chinese Patent Publication No. CN118154836A discloses a fuel tank cap positioning model training method, fuel tank cap positioning method, and apparatus. This method and apparatus construct a dataset by collecting sample images containing different scenes, vehicle body colors, and fuel tank cap shapes. The YOLOv3 algorithm is used to train a model, enabling it to accurately predict the position of the fuel tank cap, i.e., its bounding box. The accuracy of this approach is highly dependent on the comprehensiveness of the training dataset. If novel, unusually shaped, severely occluded, or damaged fuel tank caps not included in the training set appear, the recognition accuracy of the trained model will significantly decrease. Furthermore, this method is sensitive to changes in lighting conditions, such as strong reflections, shadows, and low light at night, which severely affects the accuracy and stability of positioning when used in complex outdoor lighting environments.

[0005] In summary, the limitations of traditional vision methods in terms of planar information and the bottleneck in stereo matching efficiency constitute the core challenges of automatic positioning technology for automotive fuel tank caps. These issues not only restrict the performance indicators of automated systems but also directly affect the safety and reliability in practical applications. Therefore, it is urgent to achieve high-precision and high-stability positioning of fuel tank caps during automatic refueling through innovative methods. Summary of the Invention

[0006] This invention provides a method for positioning the outer cover of a car fuel tank, which enables high-precision and high-stability positioning of the fuel tank outer cover during automatic refueling.

[0007] This invention provides a method for positioning an outer cover of a car fuel tank, comprising: Step A: Two parallel cameras simultaneously acquire images of the wheel hubs of the vehicle to be refueled, the wheels closest to the fuel tank cover. Step B: Process the wheel hub image based on the trained wheel hub localization model and output the wheel hub localization information, which includes the minimum bounding rectangle of the wheel hub boundary and the target mask of the wheel hub region; Step C: Based on the calibration parameters of the two cameras, perform parallax calculation and stereo information recovery on the wheel hub positioning information to obtain the three-dimensional pose information of the wheel hub in the camera coordinate system, and obtain the three-dimensional pose information of the vehicle body in the camera coordinate system with the wheel hub center as the origin based on the three-dimensional pose information of the wheel hub in the camera coordinate system. Step D: Based on the vehicle model to be refueled, query the stored 3D pose information of the vehicle model to obtain the relative pose information of the fuel tank cap opening point relative to the vehicle coordinate system. Step E: Combining the pose transformation relationship between the vehicle coordinate system and the camera coordinate system, and the hand-eye calibration relationship between the camera coordinate system and the automatic refueling robot base coordinate system, calculate the position and attitude information of the fuel tank cover opening point in the automatic refueling robot base coordinate system, and obtain the fuel tank cover positioning information of the vehicle to be refueled.

[0008] In a preferred embodiment, based on the position and attitude information in the base coordinate system of the automated refueling robot, the automated refueling robot drives the robotic arm to perform the opening operation of the outer fuel tank cover. More preferably, a third camera can be installed on the robotic arm to image and locate the inner fuel tank cover, and after the outer fuel tank cover is opened, the inner fuel tank cover opening and closing device on the robotic arm can be guided to complete the opening and closing operation of the inner fuel tank cover.

[0009] In a preferred implementation, the trained wheel hub positioning model is obtained through the following steps: Step 101: Acquire sample images with multiple scenes, multiple orientations, multiple vehicle body colors, multiple lighting conditions, and multiple wheel rim shapes; Step 102: Preprocess the sample image; preprocessing includes at least one of translation, scaling and brightness adjustment operations; Step 103: Label the wheel hub positions in the sample images to generate a dataset including wheel hub positioning information; Step 104: Train the wheel hub positioning model using the dataset until the prediction results of the wheel hub positioning model for wheel hub positioning information meet the preset convergence conditions.

[0010] In a preferred embodiment, before labeling the hub position of the sample image in step 103, the method further includes: performing distortion correction on the sample image based on the distortion coefficient in the camera calibration parameters.

[0011] In a preferred implementation, the wheel hub positioning model is trained based on algorithms such as Mask-RCNN or YOLO series, and outputs the minimum bounding rectangle of the wheel hub boundary and the target mask of the wheel hub.

[0012] In a preferred embodiment, the two cameras arranged in parallel are specifically binocular cameras.

[0013] In a preferred embodiment, acquiring wheel hub images includes: obtaining two wheel hub images with parallax; and performing distortion correction and stereo correction on the two wheel hub images respectively using camera calibration parameters.

[0014] In a preferred embodiment, obtaining the three-dimensional pose information of the wheel hub in the camera coordinate system includes: Ellipse fitting is performed based on the edge points of the target mask in the hub region to calculate the disparity of the hub center points in the two images. Based on the baseline and intrinsic parameters of the binocular camera, the three-dimensional coordinates of the hub center point relative to the camera coordinate system are recovered; Using binocular vision technology, the three-dimensional coordinates of the target mask edge points in the wheel hub area are recovered, and spatial plane fitting is performed on these three-dimensional coordinate points to obtain the normal vector of the wheel hub surface relative to the camera coordinate system. Based on this, the rotation matrix of the vehicle coordinate system relative to the camera coordinate system is calculated.

[0015] Based on the method for positioning an outer cap of a car fuel tank provided in the embodiments of the present invention, and based on the same inventive concept, the embodiments of the present invention also provide a device for positioning an outer cap of an automatic refueling fuel tank, comprising: Two cameras, fixedly installed on the ground of the refueling island and set in parallel, simultaneously acquire images of the wheel hubs of the vehicles to be refueled, the wheels closest to the fuel tank cover, and transmit the wheel hub images to the processor unit. The processor unit stores the trained wheel hub positioning model, vehicle model information, and pose database of the fuel tank cover opening point relative to the vehicle body coordinate system, and obtains the fuel tank cover positioning information of the vehicle to be refueled by a method for positioning the fuel tank cover according to an embodiment of the present invention.

[0016] In a preferred embodiment, the two cameras arranged in parallel are passive binocular cameras, binocular speckle cameras, or binocular structured light cameras, and are equipped with point light sources for nighttime illumination.

[0017] In a preferred embodiment, the device further includes an automatic refueling robot, which drives a robotic arm to perform the opening or closing operation of the fuel tank cover based on positioning information.

[0018] In a preferred embodiment, the robotic arm of the automatic refueling robot is equipped with a third camera to image and locate the inner cover of the fuel tank, and guide the inner cover opening and closing device on the robotic arm to complete the opening or closing operation of the inner cover of the fuel tank.

[0019] Based on the positioning method and apparatus for an automobile fuel tank cover provided in the embodiments of the present invention, and based on the same inventive concept, the embodiments of the present invention also provide an unmanned refueling system, including: An oil delivery device for refueling via a refueling nozzle; an embodiment of the present invention provides a positioning device for an outer cover of a car fuel tank; The automated refueling robot has a fuel tank cover opening and closing device at the front end of its robotic arm. This device is used to open and close the fuel tank cover based on the position information of the fuel tank cover obtained by the positioning device. The robotic arm then moves the refueling nozzle to the target position for refueling.

[0020] A preferred embodiment includes a third binocular camera mounted on a robotic arm and an inner cover opening and closing device. The third binocular camera is used to accurately position the inner cover of the fuel tank and guide the inner cover opening and closing device to complete the opening and closing operation of the inner cover of the fuel tank.

[0021] The beneficial effects of this invention include: compared with conventional techniques for direct visual positioning of the fuel tank cover, this invention estimates the wheel hub position and orientation using deep learning algorithms and binocular vision technology. Combined with the pose information between the fuel tank cover and the vehicle body coordinate system, it efficiently calculates the pose information of the fuel tank cover relative to the robot coordinate system. This effectively improves the robustness and accuracy of positioning under complex conditions such as changes in lighting, oil stains, and vehicle model differences, laying the foundation for subsequent unmanned vehicle refueling operations. Furthermore, unmanned refueling can reduce labor costs, manual labor, and work risks, improving the efficiency of vehicle refueling and promoting the further development of industrial automation. Attached Figure Description

[0022] Figure 1 This is a method for positioning the outer cover of the fuel tank according to an embodiment of the present invention; Figure 2 This is a wheel hub positioning model training method according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the definition of a coordinate system; Figure 4 This is a schematic diagram illustrating the calculation of the rotation angle between the vehicle coordinate system and the camera coordinate system; Figure 5This is a schematic diagram of the unmanned refueling system provided in an embodiment of the present invention. Detailed Implementation

[0023] The following is in conjunction with the appendix Figure 1 The specific implementation method of the positioning method for an automobile fuel tank cover provided in the embodiments of the present invention will be described in detail.

[0024] This invention provides a method for positioning an outer cover of a car fuel tank, comprising: Step A: Two parallel cameras simultaneously acquire images of the wheel hubs of the vehicle to be refueled, the wheels closest to the fuel tank cover. Step B: Process the wheel hub image based on the trained wheel hub localization model and output the wheel hub localization information, which includes the minimum bounding rectangle of the wheel hub boundary and the target mask of the wheel hub region; Step C: Based on the calibration parameters of the two cameras, perform parallax calculation and stereo information recovery on the wheel hub positioning information to obtain the three-dimensional pose information of the wheel hub in the camera coordinate system, and obtain the three-dimensional pose information of the vehicle body in the camera coordinate system with the wheel hub center as the origin based on the three-dimensional pose information of the wheel hub. Step D: Based on the vehicle model to be refueled, query the stored 3D pose information of the vehicle model to obtain the relative pose information of the fuel tank cap opening point relative to the vehicle coordinate system. Step E: Combining the pose transformation relationship between the vehicle coordinate system and the camera coordinate system, and the hand-eye calibration relationship between the camera coordinate system and the automatic refueling robot base coordinate system, calculate the position and attitude information of the fuel tank cover opening point in the automatic refueling robot base coordinate system, and obtain the fuel tank cover positioning information of the vehicle to be refueled.

[0025] To achieve automated refueling, accurately determining the precise pose of the fuel tank cap opening point is essential. However, existing solutions using planar image processing algorithms cannot accurately identify and locate the fuel tank cap. This is because car fuel tank caps typically exhibit a variety of colors, shapes, weak textures, few features, strong reflectivity, and environmental reflections, posing a significant challenge to image recognition-based fuel tank cap identification and localization. The main reasons include: firstly, the diversity of vehicle colors and fuel tank cap shapes; secondly, the randomness of vehicle parking positions and angles; and thirdly, the variability of usage scenarios due to weather and other factors. To accurately obtain the pose information of the fuel tank outer cover, this invention provides a method for locating the opening point of the fuel tank outer cover. This method uses a binocular system calibration and correction, along with a deep learning target detection and positioning algorithm, to precisely locate the center and pose of the rear wheel hub of a vehicle. Then, it utilizes the relative pose of the fuel tank outer cover opening point with respect to the vehicle's coordinate system with the hub center as the origin to complete the precise positioning of the fuel tank outer cover opening point. This provides pose information and a basis for the refueling robot to open the outer and inner fuel tank covers and operate the fuel nozzle for automatic refueling.

[0026] The process of locating the opening point of the fuel tank cover mainly includes offline calibration of two cameras, training of the wheel hub positioning model, and the online execution steps mentioned above.

[0027] The steps are explained in detail below: The calibration steps involve two cameras, which can be stereo cameras. The calibration method can employ Zhang Zhengyou's calibration method, which calibrates the stereo cameras based on multiple sets of images captured by the stereo cameras on the calibration board, obtaining parameters including the intrinsic parameters of the left and right cameras, the distortion coefficients of the left and right cameras, and the rotation and translation matrices between the left and right cameras.

[0028] Camera calibration essentially involves obtaining the direct transformation relationship between the world coordinate system (which is the robot's base coordinate system in this invention), the camera coordinate system, and the pixel coordinate system. Zhang Zhengyou's calibration method can be used to first calibrate and correct distortion in a single camera. After completing the camera calibration and obtaining the intrinsic parameters of the single camera, a binocular system model is established. Due to errors in the manufacturing and installation of the binocular system, stereo calibration is required to solve for the mutual pose relationship between the left and right cameras, transforming it into an ideal binocular system, thus obtaining the three-dimensional information of spatial points.

[0029] For example, 15 sets of checkerboard images can be acquired. In order to establish the correspondence between the object and the coordinates of each point on the imaging plane, Zhang Zhengyou's calibration method is used to calibrate the two cameras respectively. Based on the 15 sets of checkerboard images acquired from different angles, feature points in the images are detected, the camera intrinsic and extrinsic parameters under ideal distortion-free conditions are solved, and the maximum likelihood method is used to estimate and improve the accuracy. The least squares method is used to calculate the actual radial distortion coefficient. Finally, the intrinsic parameters, extrinsic parameters, and distortion coefficients are combined, and the maximum likelihood method is used to optimize the estimation and improve the accuracy. The camera intrinsic and extrinsic parameter matrices, distortion coefficients, and reprojection errors are calculated to make the calibration results meet the accuracy requirements.

[0030] As attached Figure 2 The training steps for the wheel hub positioning model shown are as follows: Step 101: Acquire sample images with multiple scenes, multiple orientations, multiple vehicle body colors, multiple lighting conditions, and multiple wheel rim shapes; Step 102: Preprocess the sample image, including at least one of translation, scaling and brightness adjustment.

[0031] Step 103: Label the hub positions in the sample images to generate a dataset including hub position information. This mainly involves adding bounding boxes of specified shapes, such as rectangles, circles, and ellipses, to the hub positions in the sample images based on their shapes. Before performing this step, it is preferable to perform distortion correction on the acquired sample images. This correction is done based on the distortion coefficients in the determined camera calibration parameters. Distortion correction of the sample images avoids image deformation and errors caused by different shooting angles, making the trained model's predictions of the target more accurate.

[0032] Step 104: Add the labeled sample images and wheel hub position information to the dataset; use the dataset to train the wheel hub positioning model until the prediction results of the wheel hub positioning model for the wheel hub positioning information meet the preset convergence conditions, and obtain the trained wheel hub positioning model.

[0033] In some embodiments, the wheel hub recognition and localization model can be a target localization and instance segmentation model represented by Mask-RCNN or YOLO series algorithms (such as YOLOv8 / V11), and the output is the minimum bounding rectangle of the wheel hub boundary and the target mask of the wheel hub region.

[0034] To achieve the positioning of the fuel tank cover, hand-eye calibration between the two cameras and the robot is also required.

[0035] At the gas station, two cameras can be binocular cameras, specifically passive binocular cameras, binocular speckle cameras, or binocular structured light cameras, equipped with point light sources for nighttime illumination. The binocular cameras are fixedly mounted on a refueling island, forming an "eye-outside-hand" positional relationship with the automated refueling robot. Before normal use, the rotational and translational transformation relationship between the binocular camera coordinate system and the automated refueling robot's base coordinate system needs to be calibrated. This process, known as "eye-outside-hand" calibration, is a mature and well-known technology and will not be elaborated upon here.

[0036] The following describes a more detailed embodiment of the fuel tank outer cover positioning method.

[0037] Step A1: Simultaneously acquire two images of the car's rear wheel hubs with parallax, captured by a binocular camera; Step A2: Based on the distortion coefficients in the pre-determined binocular camera calibration parameters, perform distortion correction on the acquired image of the wheel hub to be located. Distortion correction makes the image information more accurate and avoids various errors caused by image deformation or differences in shooting angle. Step A3: Using the relative pose parameters of the left and right cameras obtained from the binocular camera calibration, perform stereo correction on the two distortion-corrected wheel hub images. This step can be accomplished using the most common Bouguet epipolar correction method, which is a mature and well-known technology. Step B1: Input the two stereo-corrected wheel hub images into the trained wheel hub localization model and output the wheel hub localization information. The localization information output by the model is the position information of the wheel hub in the image pixel coordinate system. The output wheel hub localization information includes the minimum bounding rectangle of the wheel hub boundary and the target mask of the wheel hub region, and the edge information of the wheel hub region mask is detected accordingly. Step C1: Based on the pre-determined calibration parameters of the binocular camera, the hand-eye calibration parameters between the binocular camera and the base coordinate system of the automatic refueling robot, and the edge information of the wheel hub area mask, perform three-dimensional information restoration and coordinate system transformation on the wheel hub center and the wheel hub mask edge to obtain the position and attitude information of the wheel hub in the camera coordinate system and the base coordinate system of the automatic refueling robot, thereby determining the rotation and translation transformation relationship between the vehicle coordinate system with the wheel hub center as the origin and the camera coordinate system.

[0038] Reference Appendix Figure 3 As shown, in order to further illustrate the oil tank outer cover positioning method described in this embodiment, the definitions of the several coordinate systems involved are described as follows.

[0039] The base coordinate system of the automated refueling robot—a Cartesian coordinate system based on the robot's mounting base, used to describe the robot's body motion, with { The three axes represent the axes; Camera coordinate system—with the optical center of the left camera of the stereo camera as the origin, the horizontal axis is the Y-axis (positive to the right), the vertical axis is the X-axis (positive upwards), and the optical axis is the Z-axis (positive in the direction pointing to the target). The three axes represent the axes; Vehicle coordinate system – with the rear wheel hub center closest to the fuel tank cap as the origin, the vehicle's direction as the Y-axis (positive forward), the vehicle's lateral direction as the Z-axis (positive inward), and the X-axis vertically upward as positive. The three axes represent this.

[0040] Rotation and translation relationship between the camera coordinate system and the base coordinate system of the automated refueling robot The parameters determined through the aforementioned robot hand-eye calibration process are known parameters of the system.

[0041] The position of the fuel tank cap opening point in the vehicle coordinate system is denoted as { },in The coordinates are the position. The direction vector of the opening point in the vehicle coordinate system is known information and can be obtained from the stored vehicle model information.

[0042] Rotation and translation relationship between the vehicle coordinate system and the camera coordinate system Since these are unknowns, they need to be solved in real time during the operation.

[0043] In this embodiment, the translation vector of the vehicle coordinate system relative to the camera coordinate system is... The calculation process is described below.

[0044] Assume that the number of mask edge points for the wheel hub region located in the left eye image is k, which are { }, the number of n edge points n of the mask for the wheel hub region located in the right eye image, respectively { To accurately determine the center position of the wheel hub, considering the deformation caused by the tilt angle, we assume the outer contour of the wheel hub is elliptical. We use the least squares method to fit the ellipse. After obtaining the foci of the fitted ellipse of the wheel hub mask in the left and right images respectively, we calculate the center point of the left and right ellipses. },{ Then calculate the horizontal disparity at the center point of the ellipse: Formula (1) Next, based on the intrinsic parameters of the left camera obtained during the calibration phase and the baseline length between the optical centers of the left and right cameras, the three-dimensional coordinates of the wheel hub center relative to the coordinate system of the left camera's optical center can be calculated. This is the translation vector of the vehicle coordinate system relative to the camera coordinate system. .

[0045] For calculation Discrete sampling is performed on the fitted left and right ellipses obtained by the above method to obtain the sampling point sets of the left and right ellipses { },{ The number of sampling points are respectively and .

[0046] Then, by searching for matching points of the same name in the left and right sampling point sets, calculating disparity, and restoring stereo information, the three-dimensional coordinates of the wheel hub edge sampling points relative to the camera coordinate system are obtained {( Next, the least squares method is used to perform plane fitting on this three-dimensional coordinate point set to obtain the normal vector of the fitted plane on the hub surface. This is the vehicle coordinate system. The direction vector of the axis in the camera coordinate system.

[0047] As attached Figure 4 As shown, this embodiment can calculate the vehicle coordinate system relative to the camera coordinate system. The rotation angle of the shaft is Formula (2) Vehicle coordinate system relative to camera coordinate system The rotation angle of the shaft is Formula (3) The rotation angles mentioned above are positive counterclockwise. Therefore, the vehicle coordinate system relative to the camera coordinate system... shaft and The rotation matrices of the axes are as follows: Formula (4) Formula (5) In practical applications, it is assumed that the gas station is along The axis remains horizontal, without any rotation. The assumption of axis rotation is reasonable considering the construction requirements of gas stations. This allows us to calculate the rotation matrix of the vehicle coordinate system relative to the camera coordinate system. Formula (6) Step D1: Extract vehicle model information from license plate information to obtain the pose information of the fuel tank cap opening point in the vehicle coordinate system. }; Step E1: Based on the pose information obtained in step D1 and the rotation and translation relationship between the vehicle coordinate system and the camera coordinate system calculated through the preceding steps... And the rotation and translation relationship between the camera coordinate system and the base coordinate system of the automatic refueling robot obtained by hand-eye calibration. The position and orientation of the fuel tank cap opening point in the base coordinate system of the automatic refueling robot are calculated. The calculation process is as follows: The coordinate position of the fuel tank cap opening point relative to the camera coordinate system { } and attitude vector { The formula for calculating} is: Formula (7) Formula (8) The formulas for the position and pose transformation between the camera coordinate system and the base coordinate system of the automated refueling robot are as follows: Formula (9) Formula (10) The position of the opening point of the fuel tank cover in the base coordinate system of the automatic refueling robot can be calculated using formulas (7) to (10). } and attitude information { }, obtain the location information of the fuel tank cover of the vehicle to be refueled; Step F: Based on the position of the fuel tank cap opening point in the automatic refueling robot's base coordinate system { } and attitude information { The automatic refueling robot's robotic arm and refueling nozzle are driven to the opening point of the fuel tank's outer cover to perform the opening and closing of the outer and inner covers of the fuel tank and the refueling operation.

[0048] The method described in this embodiment acquires sample wheel hub images, labels the wheel hub positions and information in the sample wheel hub images, and constructs a dataset including sample wheel hub images and wheel hub position information. This dataset is then used to train a wheel hub positioning model, ensuring that the model's parameters meet preset convergence conditions, resulting in a well-trained wheel hub positioning model. Due to the use of a large amount of sample data and the clear and stable wheel hub feature information, a positioning model that meets the requirements of prior polygon bounding box parameters is trained, enabling the trained model to accurately locate the wheel hub position and wheel hub edge information. For images to be located using a binocular camera, based on the located wheel hub position, the position of the wheel hub, including depth information, can be determined in both the camera coordinate system and the automatic refueling robot's base coordinate system. This means the wheel hub's pose can be accurately identified, not just its planar position on a certain plane. Even if the vehicle's parking position shifts left or right, tilts, or the vehicle width changes, the accurate position and orientation of the wheel hub and the fuel tank cap opening point can still be accurately located, providing technical support for automatic refueling and ensuring its smooth operation.

[0049] Based on the embodiment of the fuel tank outer cover positioning method of the present invention, a corresponding embodiment of a fuel tank outer cover positioning device is provided, which is used to obtain the position and orientation information of the target fuel tank outer cover of the vehicle to be refueled before refueling. The device includes: Two cameras, fixedly installed on the ground of the refueling island and set in parallel, simultaneously acquire images of the wheel hubs of the vehicles to be refueled, the wheels closest to the fuel tank cover, and transmit the wheel hub images to the processor unit. The processor unit stores the trained wheel hub positioning model, vehicle model information, and pose database of the fuel tank cover opening point relative to the vehicle body coordinate system, and obtains the fuel tank cover positioning information of the vehicle to be refueled according to the fuel tank cover positioning method provided in the above embodiment.

[0050] To better enable big data processing, a dedicated computing coprocessor can be configured for the processor unit to execute the positioning method provided in the embodiments above. The computing coprocessor can be selected and configured as needed, depending on the response time requirements.

[0051] The two cameras, set in parallel, are passive binocular cameras, binocular speckle cameras, or binocular structured light cameras, and are equipped with point light sources for nighttime illumination.

[0052] The positioning device also includes an automated refueling robot. This robot drives a robotic arm to open or close the outer cap of the fuel tank based on positioning information. A third camera can also be installed on the robotic arm to image and locate the inner cap of the fuel tank, guiding the inner cap opening / closing device on the robotic arm to complete the opening or closing operation. After the inner cap is opened, the robotic arm can grab the refueling nozzle and move it to the filling port to refuel. Alternatively, the robotic arm can carry the refueling nozzle and move it after opening the inner cap to refuel. The positioning device can be integrated into the automated refueling robot. As an information processing device, it can exist independently or be integrated with or placed within other equipment; there are no restrictions on this.

[0053] Based on the fuel tank cover positioning method and apparatus of the present invention, an embodiment of an unmanned refueling system is provided, which is used to automatically locate the fuel tank cover of the vehicle to be refueled, and automatically open or close the fuel tank cover, and complete the fuel filling.

[0054] As attached Figure 5 As shown, embodiments of the present invention also provide an unmanned refueling system. After the vehicle 1 to be refueled arrives at the refueling station, the system determines the side of the vehicle with its fuel tank cap based on the vehicle information, guides the vehicle into the refueling lane, and ensures that the side of the vehicle with the fuel tank cap faces the positioning device 4.

[0055] The unmanned refueling system includes a positioning device 4, which uses two parallel cameras 41 to take pictures of the wheel hub 3 of the vehicle to be refueled 1 near the fuel tank cover 2, and obtains the positioning information of the fuel tank cover 2 according to the above positioning method based on the obtained wheel hub images; an automatic refueling robot 5, whose robotic arm 51 is equipped with a fuel tank cover opening and closing device 61 at its front end, which is used to perform the opening and closing operation of the fuel tank cover 2 according to the fuel tank cover pose information obtained by the positioning device 4; the robotic arm 51 moves the refueling nozzle to the target position for refueling.

[0056] A third binocular camera 52 and an inner cover opening and closing device can be installed on the robotic arm 51. The third binocular camera is used to accurately position the inner cover of the fuel tank and guide the inner cover opening and closing device to complete the opening and closing operation of the inner cover of the fuel tank.

[0057] After the fuel tank outer cover positioning device 4 obtains the position and orientation information of the target fuel tank cover 2 of the vehicle 1 to be refueled, the fuel tank outer cover opening and closing device 61 on the robotic arm 51 of the automatic refueling robot 5 opens the target fuel tank outer cover 2 at the position indicated by the position information; then, the third binocular camera 52 performs binocular imaging on the fuel tank inner cover and identifies the opening point of the fuel tank inner cover; the fuel tank inner cover opening and closing device opens the fuel tank inner cover; then the robotic arm grabs the refueling nozzle to refuel the vehicle 1 to be refueled; after the vehicle 1 to be refueled is refueled, the robotic arm 51 removes the refueling nozzle; the fuel tank cover opening and closing device closes the target fuel tank inner cover and outer cover.

[0058] The fuel tank cap positioning method, device, and refueling system provided in this invention, by using the image to be positioned captured by a binocular camera, can locate the position of the wheel hub in the camera coordinate system and the base coordinate system of the automatic refueling robot, including depth information, based on the located wheel hub position. That is, it can accurately identify the wheel hub's pose, not just its planar position on a certain plane. Even if the vehicle's parking position is lateral, tilted, or the vehicle width changes, it can still accurately locate the precise position and posture of the wheel hub and the fuel tank cap opening point, thereby more accurately guiding the automatic refueling operation and enabling the automatic refueling to proceed smoothly.

[0059] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for positioning an outer cover of a car fuel tank, characterized in that: Step A: Two parallel cameras simultaneously acquire images of the wheel hubs of the vehicle to be refueled, the wheels closest to the fuel tank cover. Step B: Process the wheel hub image based on the trained wheel hub positioning model and output wheel hub positioning information, which includes the minimum bounding rectangle of the wheel hub boundary and the target mask of the wheel hub region; Step C: Based on the calibration parameters of the two cameras, perform parallax calculation and stereo information recovery on the wheel hub positioning information to obtain the three-dimensional pose information of the wheel hub in the camera coordinate system, and obtain the three-dimensional pose information of the vehicle body in the camera coordinate system with the wheel hub center as the origin based on the three-dimensional pose information of the wheel hub in the camera coordinate system. Step D: Based on the vehicle model to be refueled, query the stored vehicle model's three-dimensional pose information to obtain the relative pose information of the fuel tank cap opening point relative to the vehicle body coordinate system. Step E: Combining the pose transformation relationship between the vehicle coordinate system and the camera coordinate system, and the hand-eye calibration relationship between the camera coordinate system and the automatic refueling robot base coordinate system, calculate the position and attitude information of the fuel tank cover opening point in the automatic refueling robot base coordinate system to obtain the fuel tank cover positioning information of the vehicle to be refueled.

2. The method according to claim 1, characterized in that, Based on the position and attitude information in the base coordinate system of the automatic refueling robot, the automatic refueling robot drives the robotic arm to perform the opening operation of the fuel tank cover.

3. The method according to claim 1, characterized in that, The trained wheel hub positioning model is obtained through the following steps: Step 101: Acquire sample images with multiple scenes, multiple orientations, multiple vehicle body colors, multiple lighting conditions, and multiple wheel rim shapes; Step 102: Preprocess the sample image; the preprocessing includes at least one operation among translation, scaling, and brightness adjustment; Step 103: Annotate the wheel hub positions in the sample images to generate a dataset including wheel hub positioning information; Step 104: Train the wheel hub positioning model using the dataset until the prediction results of the wheel hub positioning model for wheel hub positioning information meet the preset convergence conditions.

4. The method according to claim 3, characterized in that, Before labeling the hub position of the sample image in step 103, the process also includes: performing distortion correction on the sample image based on the distortion coefficient in the camera calibration parameters.

5. The method according to claim 1 or 3, characterized in that, The wheel hub positioning model is trained based on algorithms such as Mask-RCNN or YOLO series, and outputs the minimum bounding rectangle of the wheel hub boundary and the target mask of the wheel hub.

6. The method according to claim 1, characterized in that, The two cameras set in parallel are specifically binocular cameras.

7. The method according to claim 1, characterized in that, The acquisition of the wheel hub image includes: Obtain two wheel hub images that exhibit parallax; The two wheel hub images were subjected to distortion correction and stereo correction using camera calibration parameters.

8. The method according to claim 1, characterized in that, The obtained three-dimensional pose information of the wheel hub in the camera coordinate system includes: Ellipse fitting is performed based on the edge points of the target mask in the hub region to calculate the disparity of the hub center points in the two images. Based on the camera baseline and intrinsic parameters, recover the three-dimensional coordinates of the hub center point relative to the camera coordinate system; Spatial plane fitting is performed on the three-dimensional coordinates of the edge points of the target mask in the wheel hub area to obtain the normal vector of the wheel hub surface relative to the camera coordinate system, and the rotation matrix of the vehicle coordinate system relative to the camera coordinate system is calculated accordingly.

9. The method according to claim 2, characterized in that, After opening the outer cover of the fuel tank, a third camera mounted on the robotic arm is used to image and position the inner cover of the fuel tank, guiding the inner cover opening and closing device on the robotic arm to complete the opening or closing operation of the inner cover of the fuel tank.

10. A device for positioning the outer cover of an automatic refueling tank, characterized in that, The device includes: Two cameras, fixedly installed on the ground of the refueling island and arranged in parallel, simultaneously acquire images of the wheel hubs of the vehicles to be refueled, the wheels closest to the fuel tank cover, and transmit the wheel hub images to the processor unit. The processor unit stores the trained wheel hub positioning model, vehicle model information, and the pose database of the fuel tank cover opening point relative to the vehicle body coordinate system, and obtains the fuel tank cover positioning information of the vehicle to be refueled according to the method described in claim 1.

Images