Fuel filler orientation determination method, apparatus, control device, and readable storage medium

Abstract

The application relates to the technical field of positioning, and provides a fuel filler orientation determination method and device, a control equipment and a readable storage medium, the method comprises the following steps: acquiring an Aruco code image photographed by a binocular shooting device for each Aruco code; determining a first position of each Aruco code image in a camera coordinate system, wherein the camera coordinate system is a coordinate system with the binocular shooting device as a coordinate origin; translating each Aruco code from the corresponding first position to the direction of the center point of the fuel filler by a preset distance to obtain a second position of each Aruco code; and determining the position of the fuel filler according to the second positions of all the Aruco codes. The application realizes accurate positioning of the fuel filler while reducing the cost.

Description

Technical Field

[0001] This invention relates to the field of positioning technology, and more specifically, to a method, apparatus, control device, and readable storage medium for determining the location of a refueling port. Background Technology

[0002] The main purpose of automatic refueling services for agricultural machinery is to replenish fuel for agricultural machinery that is about to run out of fuel. In order to achieve automatic refueling, it is first necessary to be able to accurately locate the position of the refueling port of the agricultural machinery.

[0003] Most existing technologies use industrial-grade high-precision depth cameras, such as laser cameras, to obtain high-precision point cloud images of the refueling nozzle. The obtained point cloud images are then processed to obtain the precise location of the refueling nozzle. However, this technology requires extremely high-precision cameras, typically within 1 millimeter. The higher the precision, the higher the cost of the camera. Summary of the Invention

[0004] This invention provides a method, apparatus, control device, and readable storage medium for determining the location of a fuel filler nozzle. It employs a binocular imaging device, which achieves precise positioning of the fuel filler nozzle while reducing costs.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a method for determining the location of a fuel filler neck, applied to a control device. The control device is equipped with a binocular imaging device, and multiple Aruco codes are arranged on the edge of the fuel filler neck. The distance between each Aruco code and the center point of the fuel filler neck is within a preset range. The method includes: acquiring Aruco code images captured by the binocular imaging device for each Aruco code; determining a first position of each Aruco code image in a camera coordinate system, wherein the camera coordinate system is a coordinate system with the binocular imaging device as the origin; translating each Aruco code from its corresponding first position towards the center point of the fuel filler neck by a preset distance to obtain a second position of each Aruco code; and determining the location of the fuel filler neck based on the second positions of all the Aruco codes.

[0007] Specifically, the first position includes a first x-coordinate on the x-axis, a first y-coordinate on the y-axis, and a first z-coordinate on the z-axis; the second position includes a second x-coordinate on the x-axis, a second y-coordinate on the y-axis, and a second z-coordinate on the z-axis. The step of shifting each Aruco code from its corresponding first position toward the center point of the fuel filler neck by a preset distance to obtain the second position of each Aruco code includes:

[0008] The first y-coordinate of each Aruco code is translated along a preset direction of the y-axis to obtain the second y-coordinate of each Aruco code;

[0009] The second x-coordinate and second z-coordinate of each Aruco code are determined based on the relative positions of the second y-coordinate and the first y-coordinate of each Aruco code with the first x-coordinate and the first z-coordinate, respectively.

[0010] Specifically, the second position of each Aruco code is represented by the second coordinate of each Aruco code, and the step of determining the position of the filler neck based on the second positions of all the Aruco codes includes:

[0011] Calculate the distance between any two Aruco codes based on the second coordinates of any two Aruco codes among the plurality of Aruco codes;

[0012] If the distance between any two Aruco codes is less than a preset value, then the average value of the second coordinates of the multiple Aruco codes is taken as the position of the refueling nozzle.

[0013] Specifically, each Aruco code image corresponds to a depth map, and the step of determining the first position of each Aruco code image in the camera coordinate system includes:

[0014] Each Aruco code image is identified to obtain the pixel coordinates of the center point of each Aruco code image in the pixel coordinate system, wherein the pixel coordinate system is a coordinate system with a preset corner point of each Aruco code image as the origin.

[0015] In the depth map of each Aruco code, a depth value corresponding to the pixel coordinates of the center point of the image of each Aruco code is determined;

[0016] Based on the intrinsic parameters of the binocular imaging device and the depth value of the center point of each Aruco code image, determine the pixel coordinates of the center point of each Aruco code image in the camera coordinate system.

[0017] The coordinates of the center point of each Aruco code image in the camera coordinate system are taken as the first position of each Aruco code image in the camera coordinate system.

[0018] Specifically, the step of identifying each Aruco code image to obtain the pixel coordinates of the center point of each Aruco code image in the pixel coordinate system includes:

[0019] Each Aruco code image is identified to obtain the corner coordinates of the four corner points of each Aruco code image in the pixel coordinate system.

[0020] Based on the corner coordinates of each Aruco code image, determine the pixel coordinates of the center point of each Aruco code image in the pixel coordinate system.

[0021] Specifically, each Aruco code image corresponds to a depth map, and the method further includes:

[0022] Based on the depth map of each Aruco code image, point cloud data of each Aruco code image is generated;

[0023] Based on the point cloud data of each Aruco code image, the plane normal vector of each Aruco code image is obtained.

[0024] The average plane normal vector of all Aruco code images is taken as the inlet orientation of the fuel filler neck.

[0025] Specifically, the Aruco code includes a first Aruco code and two second Aruco codes, wherein the two second Aruco codes are symmetrically located on both sides of the first Aruco code, and the line connecting the center point of the first Aruco code and the center point of the fuel filler neck is perpendicular to the line connecting the center points of the two second Aruco codes.

[0026] Secondly, the present invention provides a fuel filler nozzle orientation determination device, applied to a control device. The control device is equipped with a binocular imaging device, and multiple Aruco codes are arranged on the edge of the fuel filler nozzle. The distance between each Aruco code and the center point of the fuel filler nozzle is within a preset range. The device includes: an acquisition module for acquiring Aruco code images captured by the binocular imaging device for each Aruco code; a determination module for determining a first position of each Aruco code image in a camera coordinate system, wherein the camera coordinate system is a coordinate system with the binocular imaging device as the origin; the determination module is further used to translate each Aruco code from its corresponding first position towards the center point of the fuel filler nozzle by a preset distance to obtain a second position of each Aruco code; the determination module is further used to determine the position of the fuel filler nozzle based on the second positions of all the Aruco codes.

[0027] Thirdly, the present invention provides a control device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method for determining the location of the refueling port.

[0028] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for determining the location of the refueling port.

[0029] Compared with existing technologies, this invention sets multiple Aruco codes on the edge of the fuel filler neck and uses a binocular camera to capture images of the Aruco codes. First, the first position of the Aruco code in the camera coordinate system is determined. Then, each Aruco code is translated a preset distance from its corresponding first position toward the center point of the fuel filler neck to obtain the second position of each Aruco code. The position of the fuel filler neck is finally determined based on the second positions of multiple Aruco codes. Since this invention can achieve precise positioning of the fuel filler neck without the need for a costly high-precision depth camera, it greatly reduces the implementation cost. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is an example diagram illustrating a specific setting method for the fuel filler cap and Aruco code provided in an embodiment of the present invention.

[0032] Figure 2 This is a flowchart illustrating a method for determining the location of a refueling port, as provided in an embodiment of the present invention.

[0033] Figure 3 A flowchart illustrating another method for determining the location of a refueling port provided in an embodiment of the present invention.

[0034] Figure 4 A flowchart illustrating another method for determining the location of a refueling port provided in an embodiment of the present invention.

[0035] Figure 5 An example diagram illustrating the translation of the center point of the Aruco code provided in an embodiment of the present invention.

[0036] Figure 6 A flowchart illustrating another method for determining the location of a refueling port provided in an embodiment of the present invention.

[0037] Figure 7 A flowchart illustrating another method for determining the location of a refueling port provided in an embodiment of the present invention.

[0038] Figure 8Example diagrams of point cloud images of the fuel filler neck in the z-axis and x-axis directions provided in embodiments of the present invention.

[0039] Figure 9 This is a block diagram illustrating a refueling port orientation determination device provided in an embodiment of the present invention.

[0040] Figure 10 This is a block diagram of a control device provided in an embodiment of the present invention.

[0041] Icons: 10-Control device; 11-Processor; 12-Memory; 13-Bus; 14-Peripheral interface; 20-Binocular camera device; 100-Fuel filler port location determination device; 110-Acquisition module; 120-Determination module. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0043] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0044] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0045] In the description of this invention, it should be noted that if terms such as "upper," "lower," "inner," or "outer" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of this invention is usually placed, they are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0046] Furthermore, the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0047] It should be noted that, where there is no conflict, the features in the embodiments of the present invention can be combined with each other.

[0048] For camera devices, high-precision depth cameras typically have an accuracy within 1 millimeter. These cameras can capture high-precision images for target object localization. However, high-precision depth cameras are also more expensive. In contrast, lower-precision camera devices, such as binocular cameras, typically have an accuracy of 2 to 3 millimeters, with an error generally within ±2 millimeters, and are much cheaper. However, these binocular cameras cannot effectively identify point clouds on curved or indistinct surfaces. Furthermore, the center of a fuel filler neck is hollow, lacking a flat surface, and the edges are usually curved. Therefore, directly using a low-cost binocular camera for fuel filler neck identification cannot meet the accuracy requirements for locating the fuel filler neck.

[0049] Aruco codes are a commonly used method for assisting in the location of target objects. They are established by placing Aruco codes on or near the target object and then identifying the captured image of the Aruco codes to aid in positioning. However, this method typically has a positional error of ±3 mm and a directional error of ±5 degrees. Therefore, directly using the positional and directional information from the Aruco codes cannot meet the accuracy requirements for locating the fuel filler cap.

[0050] In view of this, embodiments of the present invention provide a method, apparatus, control device, and readable storage medium for determining the location of a refueling port. By using Aruco codes captured by a binocular imaging device, the precise location of the refueling port is achieved while reducing costs. The following will describe it in detail.

[0051] Please refer to Figure 1 , Figure 1 This is an example diagram illustrating a specific configuration of the fuel filler cap and Aruco code provided in an embodiment of the present invention. Figure 1 In the process, three Aruco codes are set at the edge of the filler neck: one first Aruco code and two second Aruco codes. The two second Aruco codes are symmetrically arranged on both sides of the first Aruco code, and the distance between the three Aruco codes and the center point of the filler neck is within a preset range. The preset range can be an interval or a value. If the preset range is a value, it means that the two endpoints of the preset range are equal. For example, the distance is 120 mm. The preset range can be the measurement value after multiple measurements and calibrations using tools. Figure 1 In the first Aruco code, the angle between the central axis of the two second Aruco codes and the central axis of the first Aruco code is 25 degrees.

[0052] It should be noted that, Figure 1This is just one specific implementation method, not the only one. In actual application scenarios, the number of Aruco codes can be 2 or more than 3, and the included angle can also be set according to the actual application scenario, such as 20 degrees or 30 degrees.

[0053] This invention provides a method applicable to Figure 1 For the method of determining the location of the refueling port in the settings, please refer to [link / reference]. Figure 2 , Figure 2 This is a flowchart illustrating a method for determining the location of a refueling port according to an embodiment of the present invention. It should be noted that... Figure 2 The method described above is applicable to scenarios where the number of Aruco codes is greater than or equal to 2, and is also suitable for application scenarios at different angles. As long as the distances between multiple Aruco codes and the center point of the fuel filler neck are all within a preset range, the method includes the following steps:

[0054] Step S100: Acquire Aruco code images captured by the binocular imaging device for each Aruco code.

[0055] In this embodiment, the control device can be a robotic arm control device, which can be built into the robotic arm to control its movement, or it can be a separate control device that can communicate with the robotic arm to control its movement. The binocular camera device can be installed on the robotic arm and can communicate with the control device to capture Aruco images of each Aruco code on the edge of the refueling port when the robotic arm moves to a position close to the refueling port, so that the control device can acquire the image from the binocular camera device.

[0056] As another implementation, the binocular imaging device can capture an image of multiple Aruco codes and identify each Aruco code from the image.

[0057] Step S110: Determine the first position of each Aruco code image in the camera coordinate system, wherein the camera coordinate system is a coordinate system with the binocular imaging device as the origin.

[0058] In this embodiment, each Aruco code has a position in the pixel coordinate system of the image plane in the Aruco code image. This pixel coordinate system can be a coordinate system with the preset corner point of the Aruco code as the origin. This position needs to be converted into the position in the camera coordinate system according to the intrinsic parameters of the binocular imaging device.

[0059] Step S120: Shift each Aruco code from its corresponding first position toward the center point of the fuel filler nozzle by a preset distance to obtain the second position of each Aruco code.

[0060] In this embodiment, the center point of each Aruco code can be shifted a preset distance from its first position toward the center point of the fuel filler. Alternatively, a preset corner point or a preset point on a preset edge of each Aruco code, or any preset point within the Aruco code area, can be shifted a preset distance.

[0061] In this embodiment, the preset distance can be set according to the required moving point. For example, to move the center point of the Aruco code, the distance between the center point of the Aruco code and the center point of the fuel filler neck is first obtained through multiple measurements and calibrations, and then this distance is set as the preset distance. As another implementation method, if the corner point of the upper left position of the Aruco code is moved, the distance between the corner point of the upper left position of the Aruco code and the center point of the fuel filler neck is obtained through multiple measurements and calibrations, and then this distance is set as the preset distance.

[0062] Step S130: Determine the location of the filler neck based on the second position of all Aruco codes.

[0063] In this embodiment, since each Aruco code is within a preset range from the center of the filler neck, as each Aruco code moves toward the center of the filler neck, the distance between any two Aruco codes will inevitably become closer and closer to the position of the filler neck. Therefore, the position of the filler neck can be determined based on the second position of all Aruco codes.

[0064] The method provided in this embodiment of the invention sets multiple Aruco codes on the edge of the fuel filler neck, uses a binocular camera to capture images of the Aruco codes, first determines the first position of the Aruco code in the camera coordinate system, and then translates each Aruco code from its corresponding first position towards the center point of the fuel filler neck by a preset distance to obtain the second position of each Aruco code. The position of the fuel filler neck is finally determined based on the second positions of multiple Aruco codes. Since the precise positioning of the fuel filler neck can be achieved without using a high-precision depth camera, which is costly, the implementation cost is greatly reduced.

[0065] exist Figure 2 Based on this, embodiments of the present invention also provide a specific implementation method for determining the first position, please refer to... Figure 3 , Figure 3 This is a flowchart illustrating another method for determining the location of a refueling port provided in an embodiment of the present invention. Step S110 includes the following sub-steps:

[0066] Sub-step S1101: Recognize each Aruco code image to obtain the pixel coordinates of the center point of each Aruco code image in the pixel coordinate system, wherein the pixel coordinate system is a coordinate system with the preset corner point of each Aruco code image as the origin.

[0067] In this embodiment, existing tools or software can be used to identify each Aruco code image, for example, the aruco module in the opencv_contrib software can be used for identification.

[0068] In this embodiment, the preset corner point can be the corner point of the top left, bottom left, top right, or bottom right of each Aruco code.

[0069] In this embodiment, as a specific implementation, the step of determining the pixel coordinates of the center point of each Aruco code image can be:

[0070] First, each Aruco code image is identified to obtain the corner coordinates of the four corner points of each Aruco code image in the pixel coordinate system.

[0071] Secondly, based on the corner coordinates of each Aruco code image, the pixel coordinates of the center point of each Aruco code image in the pixel coordinate system are determined.

[0072] It should be noted that existing tools can also be used to identify each Aruco code image and directly obtain the pixel coordinates of the center point of each Aruco code image.

[0073] Sub-step S1102: Determine the depth value corresponding to the pixel coordinates of the center point of each Aruco code image in the depth map of each Aruco code.

[0074] In this embodiment, a binocular camera device can simultaneously acquire an image of each Aruco code and a depth map of each Aruco code. The depth map, also known as a distance image, is an image where the distance (depth) from the image acquisition device to each point in the scene is used as pixel values. For any Aruco code's depth map, each pixel corresponds to a depth value, which represents the distance between the pixel in the Aruco code and the binocular camera device.

[0075] Sub-step S1103: Based on the intrinsic parameters of the binocular imaging device and the depth value of the center point of each Aruco code image, determine the pixel coordinates of the center point of each Aruco code image in the camera coordinate system.

[0076] In this embodiment, the pixel coordinates of the center point of the Aruco code image are two-dimensional coordinates. The three-dimensional coordinates of the center point of the Aruco code image can be obtained by using the depth value and the intrinsic parameters of the binocular imaging device.

[0077] Sub-step S1104: The coordinates of the center point of each Aruco code image in the camera coordinate system are taken as the first position of each Aruco code image in the camera coordinate system.

[0078] In this embodiment, since the position of the center point is relatively less affected by the shooting angle and distortion, the coordinates of the center point of each Aruco code image in the camera coordinate system are used to represent the first position of each Aruco code image.

[0079] based on Figure 2 The present invention also provides a specific implementation method for obtaining the second position of each Aruco code, please refer to... Figure 4 , Figure 4 This is a flowchart illustrating another method for determining the location of a refueling port provided in an embodiment of the present invention. Step S120 includes the following sub-steps:

[0080] Sub-step S1201: The first y-coordinate of each Aruco code is translated along a preset direction of the y-axis to obtain the second y-coordinate of each Aruco code.

[0081] In this embodiment, both the first position and the second position can be represented by three-dimensional coordinates. The first position includes a first x-coordinate on the x-axis, a first y-coordinate on the y-axis, and a first z-coordinate on the z-axis. The second position includes a second x-coordinate on the x-axis, a second y-coordinate on the y-axis, and a second z-coordinate on the z-axis. The preset direction can be the positive direction of the y-axis.

[0082] To illustrate the translation process more clearly, this invention provides example diagrams of translation. Please refer to them. Figure 5 , Figure 5 An example diagram illustrating the translation of the center point of the Aruco code provided in an embodiment of the present invention. Figure 5 In the middle, the center point of the three Aruco codes ( Figure 5 The gray solid dots in the image are each translated a preset distance along the positive direction of their respective y-axis. Figure 5 In the Aruco code, the position of the solid center point is the first position before the movement, and the position of the white solid point is the second position after the movement.

[0083] Sub-step S1202: Determine the second x-coordinate and second z-coordinate of each Aruco code based on the second y-coordinate of each Aruco code and the relative positions of the first y-coordinate of each Aruco code with the first x-coordinate and the first z-coordinate, respectively.

[0084] In this embodiment, depending on the specific setting position of each Aruco code, the first x-coordinate before translation and the second x-coordinate after translation may be the same or different; similarly, the first z-coordinate before translation and the second z-coordinate after translation may be the same or different. For example... Figure 5In the Aruco code located in the middle, only the y-axis coordinate changes after translation, while the x-axis and z-axis coordinates remain unchanged. For the Aruco codes located on both sides, both the x-axis and z-axis coordinates change after translation. The specific changes can be determined by using the positions of the x-axis and z-axis relative to the y-axis before translation, and then determining the second x-coordinate and second z-coordinate of the x-axis and z-axis after translation.

[0085] The method provided in this embodiment of the invention ensures that the center point of each Aruco code can quickly approach the center point of the refueling nozzle in the same direction by translating the center point of each Aruco code along the positive y-axis.

[0086] In this embodiment, since the center point of each frame of Aruco code captured by the binocular camera device may fluctuate, and the y-axis direction of the Aruco code also varies slightly, the predicted center position of the fuel filler neck changes with each frame. This may lead to a slight error in the predicted center point of the fuel filler neck. To more accurately determine the position of the fuel filler neck, this embodiment of the invention... Figure 2 Based on this, a specific implementation method for determining the position of the fuel filler neck based on the second position of the Aruco code is provided. Please refer to [link / reference]. Figure 6 , Figure 6 This is a flowchart illustrating another method for determining the location of a refueling port provided in an embodiment of the present invention. Step S130 includes the following sub-steps:

[0087] Sub-step S1301: Calculate the distance between any two Aruco codes based on the second coordinates of any two Aruco codes among the multiple Aruco codes.

[0088] In this embodiment, the second position of each Aruco code is represented by the second coordinate of each Aruco code.

[0089] In sub-step S1302, if the distance between any two Aruco codes is less than a preset value, the average value of the second coordinates of multiple Aruco codes is taken as the position of the refueling port.

[0090] In this embodiment, if the Aruco code image captured by the binocular camera device is normal and subsequent recognition is normal, for normal translation within the error range, the distance between any two Aruco codes after the translation will inevitably be less than a preset value. Otherwise, it means that the shooting angle or image clarity of the binocular camera device cannot meet the requirements for correct recognition, resulting in a large deviation in the center point position of the recognized Aruco code. Ultimately, the center point of the Aruco code after translation does not approach the center point position of the fuel filler neck as expected. At this time, the position of the fuel filler neck obtained based on the second coordinates of multiple Aruco codes is incorrect or has a large deviation, and the result is incorrect. In this case, in order to obtain an accurate position of the fuel filler neck, the binocular camera device can be controlled to re-capture the image, and the above steps S100-S130 can be repeated to determine the position of the fuel filler neck. Of course, as a specific implementation, the above process can continue to be performed according to S100-S130 and its sub-steps to ensure that the final obtained position of the fuel filler neck is accurate.

[0091] In this embodiment, the second coordinate of each Aruco code is a three-dimensional coordinate, which can be represented by the second x-coordinate, the second y-coordinate, and the second z-coordinate. At this time, the average value of the second coordinates of multiple Aruco codes is obtained by averaging the second x-coordinate, the second y-coordinate, and the second z-coordinate of multiple Aruco codes respectively. The average value of the second x-coordinate, the average value of the second y-coordinate, and the average value of the second z-coordinate are the three average values, which are the positions of the refueling nozzle.

[0092] In this embodiment, after determining the location of the filler neck, in order to achieve automatic refueling, it is also necessary to determine the orientation of the filler neck. This embodiment also provides a method for determining the orientation of the filler neck; please refer to... Figure 7 , Figure 7 This is a flowchart illustrating another method for determining the location of a refueling port provided in an embodiment of the present invention. The method further includes the following steps:

[0093] Step S140: Generate point cloud data for each Aruco code image based on the depth map of each Aruco code image.

[0094] In this embodiment, point cloud data refers to a set of vectors in a three-dimensional coordinate system. These vectors are typically represented in the form of three-dimensional coordinates (X, Y, Z).

[0095] In this embodiment, as a specific implementation method, firstly, based on the depth map of each Aruco code image and the intrinsic parameters of the binocular camera device, the position of each pixel in the Aruco code image in the camera coordinate system, that is, the three-dimensional coordinates of each pixel, can be obtained. Based on the three-dimensional coordinates of each pixel in the Aruco code image, point cloud data of each Aruco code image can be generated. Figure 8This is an example diagram of the point cloud images of the fuel filler neck in the z-axis and x-axis directions provided in an embodiment of the present invention. Figure 8 (a) is the point cloud diagram of the fuel filler nozzle in the z-axis direction. Figure 8 (b) is the point cloud diagram of the refueling port in the x-axis direction.

[0096] Step S150: Based on the point cloud data of each Aruco code image, obtain the plane normal vector of each Aruco code image.

[0097] In this embodiment, the normal vector is the vector represented by a straight line perpendicular to the plane. Since there are countless straight lines in space that are perpendicular to a known plane, a plane has countless normal vectors (including two unit normal vectors). If a straight line is perpendicular to two intersecting straight lines in the plane, then the straight line is perpendicular to the plane.

[0098] In this example, the point cloud data of each Aruco code image includes the three-dimensional coordinates of each pixel in the Aruco code image. As one implementation method, the plane normal vector of each Aruco code image can be obtained by using the least squares fitting method based on the three-dimensional coordinates of each pixel in each Aruco code image. Alternatively, the plane normal vector of each Aruco code image can be calculated using existing MATLAB software.

[0099] Step S160: The average value of the plane normal vectors of all Aruco code images is used as the inlet orientation of the refueling nozzle.

[0100] In this embodiment, since the plane containing the Aruco code is horizontal to the plane of the fuel filler, the average value of the plane normal vector of all Aruco code images is the orientation of the fuel filler inlet.

[0101] It should be noted that, after multiple tests and verifications by the inventors, using three Aruco codes—one first Aruco code and two second Aruco codes—with the two second Aruco codes symmetrically positioned on either side of the first Aruco code, and the line connecting the center point of the first Aruco code to the center point of the fuel filler neck perpendicular to the line connecting the center points of the two second Aruco codes, and with each of the three Aruco codes 120 mm away from the center point of the fuel filler neck, and the angle between the central axes of the two second Aruco codes and the central axis of the first Aruco code both being 25 degrees, the fuel filler neck position identification achieved the following: with the Aruco code's own positional error being ±2 to ±3 mm, the positional error determined using the above method was controlled within ±1.5 mm, perfectly meeting the fuel filler neck identification accuracy requirements. Furthermore, by combining the y-axis direction of the Aruco code and the normal vector of the Aruco code surface point cloud, the fuel filler neck orientation identification achieved the following: with the Aruco code's own orientational error being ±5 degrees, the application's identification error for the fuel filler neck orientation was controlled within ±2 degrees. If the accuracy requirements of the refueling port are met, the cost of using the above method is approximately one-fifteenth of that of using a relatively inexpensive laser camera.

[0102] To perform the corresponding steps in the above embodiments and various possible implementations, an implementation of the fuel filler nozzle orientation determination device 100 is given below. Please refer to... Figure 9 , Figure 9 A block diagram of a fuel filler nozzle orientation determination device 100 provided in an embodiment of the present invention is shown. The fuel filler nozzle orientation determination device 100 is applied to a control device equipped with a binocular imaging device. Multiple Aruco codes are arranged along the edge of the fuel filler nozzle, and the distance between each Aruco code and the center point of the fuel filler nozzle is within a preset range. It should be noted that the basic principle and technical effects of the fuel filler nozzle orientation determination device 100 provided in this embodiment are the same as those in the above embodiments; however, for the sake of brevity, some details are not mentioned in this embodiment.

[0103] The refueling port orientation determination device 100 includes an acquisition module 110 and a determination module 120.

[0104] The acquisition module 110 is used to acquire Aruco code images captured by the binocular imaging device for each Aruco code.

[0105] The determination module 120 is used to determine the first position of each Aruco code image in the camera coordinate system, wherein the camera coordinate system is a coordinate system with the binocular imaging device as the origin.

[0106] The determining module 120 is also used to translate each Aruco code from its corresponding first position toward the center point of the refueling nozzle by a preset distance to obtain the second position of each Aruco code.

[0107] The determination module 120 is also used to determine the position of the filler neck based on the second position of all Aruco codes.

[0108] As a specific implementation, the first position includes the first x-coordinate of the x-axis, the first y-coordinate of the y-axis, and the first z-coordinate of the z-axis; the second position includes the second x-coordinate of the x-axis, the second y-coordinate of the y-axis, and the second z-coordinate of the z-axis. The determining module 120 is specifically used to: translate the first y-coordinate of each Aruco code along a preset direction of the y-axis to obtain the second y-coordinate of each Aruco code; and determine the second x-coordinate and the second z-coordinate of each Aruco code based on the second y-coordinate of each Aruco code and the relative positions of the first y-coordinate of each Aruco code with the first x-coordinate and the first z-coordinate, respectively.

[0109] As a specific implementation, the second position of each Aruco code is represented by the second coordinate of each Aruco code. The determining module 120 is specifically used to: calculate the distance between any two Aruco codes based on the second coordinates of any two Aruco codes among the multiple Aruco codes; if the distance between any two Aruco codes is less than a preset value, then the average value of the second coordinates of the multiple Aruco codes is taken as the position of the refueling nozzle.

[0110] As a specific implementation, each Aruco code image corresponds to a depth map. The determining module 120 is specifically used for: recognizing each Aruco code image to obtain the pixel coordinates of the center point of each Aruco code image in the pixel coordinate system, wherein the pixel coordinate system is a coordinate system with a preset corner point of each Aruco code image as the origin; determining the depth value corresponding to the pixel coordinates of the center point of each Aruco code image in the depth map of each Aruco code; determining the pixel coordinates of the center point of each Aruco code image in the camera coordinate system based on the intrinsic parameters of the binocular imaging device and the depth value of the center point of each Aruco code image; and taking the coordinates of the center point of each Aruco code image in the camera coordinate system as the first position of each Aruco code image in the camera coordinate system.

[0111] As a specific implementation, when determining each Aruco code image to obtain the pixel coordinates of the center point of each Aruco code image in the pixel coordinate system, the determining module 120 is specifically used to: determine each Aruco code image to obtain the corner coordinates of the four corner points of each Aruco code image in the pixel coordinate system; and determine the pixel coordinates of the center point of each Aruco code image in the pixel coordinate system based on the corner coordinates of each Aruco code image.

[0112] As a specific implementation, each Aruco code image corresponds to a depth map. The determining module 120 is further configured to: generate point cloud data for each Aruco code image based on the depth map of each Aruco code image; obtain the plane normal vector of each Aruco code image based on the point cloud data of each Aruco code image; and use the average value of the plane normal vectors of all Aruco code images as the inlet orientation of the fuel filler.

[0113] As a specific implementation, the Aruco code includes a first Aruco code and two second Aruco codes, wherein the two second Aruco codes are symmetrically located on both sides of the first Aruco code, and the line connecting the center point of the first Aruco code and the center point of the fuel filler neck is perpendicular to the line connecting the center points of the two second Aruco codes.

[0114] This invention also provides a block diagram of a control device 10 for performing the above method. Please refer to... Figure 10 , Figure 10 This is a block diagram of a control device 10 provided in an embodiment of the present invention. The control device 10 includes a processor 11, a memory 12, a bus 13, and a peripheral interface 14. The processor 11 and the memory 12 are connected via the bus 13, and the processor 11 communicates with the binocular camera device 20 via the peripheral interface 14.

[0115] Processor 11 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the hardware of processor 11 or through software instructions. Processor 11 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), an On-Premises Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0116] The memory 12 is used to store programs, such as the fuel filler location determination device 100 in this embodiment of the invention. The fuel filler location determination device 100 includes at least one software function module that can be stored in the memory 12 in the form of software or firmware. After receiving the execution instruction, the processor 11 executes the program to implement the fuel filler location determination method in this embodiment of the invention.

[0117] The memory 12 may include high-speed random access memory (RAM) or non-volatile memory. Optionally, the memory 12 may be a storage device built into the processor 11 or a storage device independent of the processor 11.

[0118] Bus 13 can be an ISA bus, PCI bus, or EISA bus, etc. Figure 10 It is indicated by a single double-headed arrow, but does not mean that there is only one bus or one type of bus.

[0119] The binocular camera device 20 can be a binocular camera, a binocular video camera, etc.

[0120] In this embodiment, as a specific application scenario, the control device can be mounted on or communicated with the robotic arm. The robotic arm can hold the refueling hose. After determining the location of the refueling port, the control device controls the robotic arm to move to the refueling port and insert the refueling hose into it, thus achieving automatic refueling. Of course, the control device can also be mounted on other mechanical devices used to control the movement of the refueling hose to achieve automatic refueling in the same way.

[0121] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for determining the location of the refueling port.

[0122] In summary, embodiments of the present invention provide a method, apparatus, control device, and readable storage medium for determining the location of a fuel filler nozzle. Applied to a control device equipped with a binocular imaging device, the edge of the fuel filler nozzle is provided with multiple Aruco codes, each Aruco code being within a preset distance from the center point of the fuel filler nozzle. The method includes: acquiring Aruco code images captured by the binocular imaging device for each Aruco code; determining a first position of each Aruco code image in a camera coordinate system, wherein the camera coordinate system is a coordinate system with the binocular imaging device as its origin; translating each Aruco code from its corresponding first position towards the center point of the fuel filler nozzle by a preset distance to obtain a second position of each Aruco code; and determining the location of the fuel filler nozzle based on the second positions of all Aruco codes.

[0123] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for determining the location of a fuel filler neck, characterized in that, The method, applied to a control device equipped with a binocular imaging device, includes the following: Multiple Aruco codes are arranged along the edge of the fuel filler neck, with each Aruco code's distance from the center point of the fuel filler neck falling within a preset range. Acquire Aruco code images captured by the binocular imaging device for each Aruco code; Determine the first position of each of the Aruco code images in the camera coordinate system, wherein the camera coordinate system is a coordinate system with the binocular imaging device as the origin; Each Aruco code is translated from its corresponding first position toward the center point of the fuel filler neck by a preset distance to obtain the second position of each Aruco code. As each Aruco code gets closer and closer to the center point of the fuel filler neck during the translation process, the distance between any two Aruco codes gets closer and closer. The location of the filler neck is determined based on the second position of all the Aruco codes.

2. The method for determining the location of the refueling port as described in claim 1, characterized in that, The first position includes a first x-coordinate on the x-axis, a first y-coordinate on the y-axis, and a first z-coordinate on the z-axis. The second position includes a second x-coordinate on the x-axis, a second y-coordinate on the y-axis, and a second z-coordinate on the z-axis. The step of shifting each Aruco code from its corresponding first position towards the center point of the fuel filler neck by a preset distance to obtain the second position of each Aruco code includes: The first y-coordinate of each Aruco code is translated along a preset direction of the y-axis to obtain the second y-coordinate of each Aruco code; The second x-coordinate and second z-coordinate of each Aruco code are determined based on the relative positions of the second y-coordinate and the first y-coordinate of each Aruco code with the first x-coordinate and the first z-coordinate, respectively.

3. The method for determining the location of the refueling port as described in claim 1, characterized in that, The second position of each of the Aruco codes is characterized by the second coordinates of each of the Aruco codes, and the step of determining the position of the filler neck based on the second positions of all the Aruco codes includes: Calculate the distance between any two Aruco codes based on the second coordinates of any two Aruco codes among the plurality of Aruco codes; If the distance between any two Aruco codes is less than a preset value, then the average value of the second coordinates of the multiple Aruco codes is taken as the position of the refueling nozzle.

4. The method for determining the location of the refueling port as described in claim 1, characterized in that, Each of the Aruco code images corresponds to a depth map, and the step of determining the first position of each of the Aruco code images in the camera coordinate system includes: Each Aruco code image is identified to obtain the pixel coordinates of the center point of each Aruco code image in the pixel coordinate system, wherein the pixel coordinate system is a coordinate system with a preset corner point of each Aruco code image as the origin. In the depth map of each Aruco code, a depth value corresponding to the pixel coordinates of the center point of the image of each Aruco code is determined; Based on the intrinsic parameters of the binocular imaging device and the depth value of the center point of each Aruco code image, determine the pixel coordinates of the center point of each Aruco code image in the camera coordinate system. The coordinates of the center point of each Aruco code image in the camera coordinate system are taken as the first position of each Aruco code image in the camera coordinate system.

5. The method for determining the location of the refueling port as described in claim 4, characterized in that, The step of identifying each Aruco code image to obtain the pixel coordinates of the center point of each Aruco code image in the pixel coordinate system includes: Each Aruco code image is identified to obtain the corner coordinates of the four corner points of each Aruco code image in the pixel coordinate system. Based on the corner coordinates of each Aruco code image, determine the pixel coordinates of the center point of each Aruco code image in the pixel coordinate system.

6. The method for determining the location of the refueling port as described in claim 1, characterized in that, Each Aruco code image corresponds to a depth map, and the method further includes: Based on the depth map of each Aruco code image, point cloud data of each Aruco code image is generated; Based on the point cloud data of each Aruco code image, the plane normal vector of each Aruco code image is obtained. The average plane normal vector of all Aruco code images is taken as the inlet orientation of the fuel filler neck.

7. The method for determining the location of the refueling port as described in claim 1, characterized in that, The Aruco code includes a first Aruco code and two second Aruco codes, wherein the two second Aruco codes are symmetrically located on both sides of the first Aruco code, and the line connecting the center point of the first Aruco code and the center point of the fuel filler neck is perpendicular to the line connecting the center points of the two second Aruco codes.

8. A device for determining the orientation of a refueling port, characterized in that, An application is made in a control device, the control device being equipped with a binocular imaging device, and multiple Aruco codes are set on the edge of the fuel filler neck, with the distance between each Aruco code and the center point of the fuel filler neck all within a preset range. The device includes: The acquisition module is used to acquire Aruco code images captured by the binocular imaging device for each Aruco code; The determination module is used to determine the first position of each of the Aruco code images in the camera coordinate system, wherein the camera coordinate system is a coordinate system with the binocular imaging device as the origin; The determining module is further configured to translate each Aruco code from its corresponding first position toward the center point of the fuel filler neck by a preset distance to obtain the second position of each Aruco code. As each Aruco code gets closer and closer to the center point of the fuel filler neck during the translation process, the distance between any two Aruco codes gets closer and closer. The determining module is further configured to determine the position of the refueling port based on the second position of all the Aruco codes.

9. A control device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and when the processor executes the computer program, it implements the method for determining the location of the refueling port as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for determining the location of the refueling port as described in any one of claims 1-7.

Images