Refueling system, refueling control method, refueling robot and storage medium

Abstract

The embodiment of the invention provides a refueling system, a refueling control method, a refueling robot and a storage medium. According to the method, a control device, an entering device, an information prompting device, a plurality of refueling robots and a plurality of refueling piles are included; the entering device is used for sending entering information to the control device; the control device is used for responding to the received entry information, sending first indication information to the information prompting device, and sending second indication information to a first refueling robot corresponding to the first refueling pile; the first indication information comprises identification information of the vehicle and identification information of the first refueling pile, and the second indication information is used for indicating the first refueling robot to move to a preset position; the information prompting device is used for prompting the vehicle to move to the first refueling pile; and the first refueling robot is used for refueling the vehicle by using the oil gun mounted on the first refueling pile. The method is used for achieving the effect of reducing the intelligent reconstruction cost of a traditional gas station.

Description

Technical Field

[0001] This application relates to the field of intelligent robots, and more particularly to a refueling system, a refueling control method, a refueling robot, and a storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent robots are being used more and more widely in production and daily life. Intelligent robots can achieve high precision, high efficiency, and automated continuous work.

[0003] In traditional gas stations, vehicles are guided to the refueling station manually, and refueling is done manually, resulting in low efficiency. Some solutions replace traditional gas station refueling piles with those equipped with robotic arms to achieve intelligent transformation, but this solution has high transformation costs. Summary of the Invention

[0004] This application provides a refueling system, a refueling control method, a refueling robot, and a storage medium, which can reduce the cost of intelligent transformation of traditional gas stations.

[0005] In a first aspect, embodiments of this application provide a refueling system, including: a control device, an entry device, an information prompting device, multiple refueling robots, and multiple refueling piles; the control device is connected to the entry device, the information prompting device, and the multiple refueling robots respectively.

[0006] The entry device is used to send entry information to the control device when a vehicle enters the site. The entry information includes the vehicle's identification information.

[0007] The control device is used to respond to receiving entry information, and if there is currently an idle first refueling pile, to send a first instruction message to the information prompting device, and to send a second instruction message to the first refueling robot corresponding to the first refueling pile; the first instruction message includes vehicle identification information and first refueling pile identification information, and the second instruction message is used to instruct the first refueling robot to move to a preset position;

[0008] The information prompting device is used to prompt the vehicle to move to the first refueling station according to the first instruction information;

[0009] The first refueling robot is used to refuel vehicles using a fuel nozzle installed on the first refueling station after the vehicle moves to the preset area of ​​the first refueling station.

[0010] In one possible implementation, the first refueling robot includes a robotic arm, which is used for:

[0011] After the vehicle moves to the preset area of ​​the first refueling station, a first image of the vehicle is captured, including the vehicle's fuel tank.

[0012] The pose data of the fuel tank relative to the first refueling robot are determined based on the first image;

[0013] Based on the pose data, the robotic arm is controlled to open the vehicle's fuel tank cap, and based on the fuel type, the robotic arm is controlled to grab the fuel nozzle corresponding to the fuel type on the first refueling station to refuel the vehicle.

[0014] In one possible implementation, the second indication information is further used to indicate the vehicle's first license plate number; the first refueling robot is further used to:

[0015] During the process of the vehicle moving to the preset area of ​​the first gas station, a second image of the vehicle is captured, and the second license plate number of the vehicle in the second image is obtained.

[0016] If the second license plate number matches the first license plate number, use the fuel nozzle installed on the first fuel station to refuel the vehicle.

[0017] In one possible implementation, the first refueling robot is used to: if the second license plate number matches the first license plate number, issue an inquiry request, the inquiry request being used to request the type and quantity of fuel to be added to the vehicle;

[0018] It receives a query response from the user and sends a refueling instruction to the first refueling station. The refueling instruction indicates the amount of fuel and the type of fuel.

[0019] In one possible implementation, the first refueling robot further includes a display screen that displays a first interface that displays an inquiry request.

[0020] The first refueling robot is used to respond to input operations on the first interface and send refueling instructions to the first refueling station.

[0021] In one possible implementation, the first refueling robot is also used to send a first notification message to the control device if the second license plate number is inconsistent with the first license plate number. The first notification message is used to indicate that the current vehicle is not the vehicle indicated by the control device.

[0022] The control device is also used to guide the current vehicle to another refueling station via an information prompting device in response to the first notification message.

[0023] In one possible implementation, the refueling system further includes a departure device connected to a control device;

[0024] The first refueling robot is also used to generate payment information based on the amount of fuel added to the vehicle after refueling is completed; display the payment information on the screen; and send a second notification message to the control device after confirming successful payment. The second notification message is used to indicate that refueling is complete and payment is successful.

[0025] The control device is also used to send a third notification message to the departure device;

[0026] The departure device is used to control the lifting of the exit barrier when a vehicle is detected to have arrived at the exit.

[0027] In one possible implementation, the refueling system further includes a departure device connected to a control device;

[0028] The first refueling robot is also used to send a fourth notification message to the control device after refueling is completed. The fourth notification message is used to indicate that the vehicle refueling is complete and the amount and type of fuel added to the vehicle.

[0029] The control device is also used to generate vehicle payment information based on the fourth notification message and send the vehicle payment information to the departure device.

[0030] The departure device is used to issue a prompt message when the vehicle arrives at the exit, prompting the user to make payment. After the user successfully makes payment, the device controls the gate at the exit to lift.

[0031] Secondly, embodiments of this application provide a refueling control method applied to a refueling robot of a refueling system as described in the first aspect and / or various possible implementations of the first aspect, wherein the refueling robot is equipped with a binocular camera; the refueling control method includes:

[0032] Acquire the first image of the vehicle from two shooting angles captured by the binocular camera. The first image includes the vehicle's fuel tank.

[0033] Extract the fuel tank feature points from the first image of the vehicle from both shooting perspectives;

[0034] Based on the camera parameters of the binocular camera, the fuel tank feature points in the first image of the vehicle from the two shooting perspectives are transformed into the world coordinate system to obtain the point cloud data of the fuel tank.

[0035] Three-dimensional reconstruction is performed based on the point cloud data of the fuel tank to determine the pose data of the fuel tank relative to the refueling robot.

[0036] The robotic arm of the refueling robot, controlled by pose data, opens the fuel tank cap of a vehicle.

[0037] After the robotic arm grabs the fuel nozzle, it is controlled to insert the fuel nozzle into the fuel tank to refuel the vehicle.

[0038] In one possible implementation, before extracting the fuel tank feature points from the first image of the vehicle from the two shooting perspectives, the method further includes:

[0039] The first images of the vehicle from two shooting perspectives are preprocessed, including at least one of the following: image filtering and image contrast correction.

[0040] Thirdly, embodiments of this application provide a refueling control device, comprising:

[0041] The acquisition module is used to acquire the first image of the vehicle from two shooting angles captured by the binocular camera. The first image includes the vehicle's fuel tank.

[0042] The processing module is used to extract fuel tank feature points from the first image of the vehicle from two shooting perspectives.

[0043] The processing module is also used to convert the fuel tank feature points in the first image of the vehicle from the two shooting perspectives to the world coordinate system according to the camera parameters of the binocular camera, and obtain the point cloud data of the fuel tank.

[0044] The processing module is also used to perform three-dimensional reconstruction based on the point cloud data of the fuel tank to determine the pose data of the fuel tank relative to the refueling robot.

[0045] The processing module is also used to control the robotic arm of the refueling robot to open the fuel tank cap of the vehicle based on the pose data.

[0046] The processing module is also used to control the robotic arm to insert the fuel nozzle into the fuel tank after the robotic arm grabs the fuel nozzle, so as to refuel the vehicle.

[0047] Fourthly, embodiments of this application provide a refueling robot, including: a binocular camera, a robotic arm, and a processor;

[0048] The binocular camera is used to capture a first image of the vehicle from two shooting angles after the vehicle moves to a preset area of ​​the gas station. The first image includes the vehicle's fuel tank.

[0049] The processor is used to execute the second aspect as described above and / or various possible implementations of the second aspect.

[0050] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the second aspect and / or various possible implementations of the second aspect.

[0051] Sixthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the second aspect and / or various possible implementations of the second aspect as described above.

[0052] The refueling system, refueling control method, refueling robot, and storage medium provided in this application embodiment, by configuring a control device, entry device, information prompting device, refueling robot, and refueling pile in the refueling system, and enabling these devices to communicate with each other, achieve intelligent identification and intelligent guidance of vehicle entry, realizing the intelligent transformation of traditional gas stations. The refueling robot, positioned at a preset location on the corresponding refueling pile, uses the fuel nozzle on the refueling pile to complete the refueling operation for the vehicle. This application embodiment retains the traditional refueling pile, using a mobile robot for refueling operations, avoiding large-scale replacement of traditional refueling piles, thereby reducing the cost of intelligent transformation of traditional gas stations. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0054] Figure 1 A schematic diagram of the refueling system provided in this application. Figure 1 ;

[0055] Figure 2 A schematic diagram of the control flow based on the refueling system provided in this application;

[0056] Figure 3 An example layout of a refueling robot and a refueling station. Figure 1 ;

[0057] Figure 4 An example layout of a refueling robot and a refueling station. Figure 2 ;

[0058] Figure 5 An example layout of a refueling robot and a refueling station. Figure 3 ;

[0059] Figure 6 An example interface diagram of a refueling robot;

[0060] Figure 7 A schematic diagram of the refueling system provided in this application. Figure 2 ;

[0061] Figure 8 A flowchart illustrating the refueling control method provided in this application;

[0062] Figure 9 A schematic diagram of the refueling control device provided in this application;

[0063] Figure 10 This is a structural schematic diagram of the refueling robot provided in this application.

[0064] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0065] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0066] With the rapid development of society and the economy, the number of cars is increasing daily, and correspondingly, the number of gas stations providing energy replenishment for cars is also increasing. In traditional gas stations, vehicles are guided to the refueling positions by humans, and refueling is done manually, which makes the refueling efficiency relatively low.

[0067] With the rapid development of artificial intelligence technology, intelligent robots are being used more and more widely in production and daily life. Intelligent robots can achieve high-precision, high-efficiency, and automated continuous operation. In some solutions, replacing traditional gas stations' refueling piles with those equipped with robotic arms enables intelligent transformation. However, this solution requires completely replacing the traditional gas stations' refueling piles with refueling robots, resulting in high costs for intelligent transformation.

[0068] The refueling system provided in this application introduces a mobile refueling robot into traditional gas stations, retains the traditional refueling piles, and equips them with intelligent devices such as control devices, entry devices, and information display devices. Through communication between these intelligent devices, the process of vehicles entering the gas station for refueling is automated, achieving intelligent transformation and improving the efficiency of vehicles entering the gas station for refueling. Furthermore, the above solution avoids large-scale replacement of traditional refueling piles, reducing the cost of intelligent transformation.

[0069] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0070] Figure 1 A schematic diagram of the refueling system provided in this application. Figure 1 ,like Figure 1As shown, the specific application scenario of this application is a refueling system consisting of a control device, an entry device, an information prompting device, multiple refueling robots, and multiple refueling piles, used to refuel a vehicle entering a gas station for refueling. As an example, the refueling system 10 includes: a control device 101, an entry device 102, an information prompting device 103, multiple refueling robots 104, and multiple refueling piles 105; the control device 101 is connected to the entry device 102, the information prompting device 103, and the multiple refueling robots 104.

[0071] As an example, gas station 105 and gas station robot 104 are in one-to-one correspondence. See below for details. Figure 3 The illustrated embodiment. In another example, one refueling station 105 can correspond to multiple refueling robots 104. See below for details. Figure 4 and Figure 5 The illustrated embodiment.

[0072] As an example, Figure 2 The schematic diagram of the control flow based on the refueling system provided in this application is as follows: Figure 2 As shown, the control flow includes:

[0073] Step 201. When a vehicle enters the site, the entry device 102 sends entry information to the control device 101. The entry information includes the vehicle's identification information.

[0074] For example, a user drives their vehicle into a gas station via entry device 102, preparing to refuel. Specifically, entry device 102 may include a gate; by controlling the raising of the gate, the vehicle enters the gas station. During this process, entry device 102 sends entry information to control device 101. The entry information indicates to control device 101 that a vehicle has entered the gas station to refuel. Further, this entry information includes vehicle identification information. The identification information is used to mark the vehicle, giving each vehicle unique identification information. Specifically, the form of the identification information is not limited. For example, the identification information may include: an identification code and / or the vehicle's license plate number.

[0075] As an example, the identification information may include an identification code that is a number generated based on the vehicle's entry sequence, used to indicate the vehicle's serial number at the current gas station. For instance, when a vehicle enters, the entry device 102 may print a paper ticket with the vehicle's serial number at the current gas station printed on it; alternatively, the vehicle's serial number at the current gas station may be displayed on the screen of the information display device 103.

[0076] As an example, the license plate number included in the identification information is the license plate number of the vehicle currently entering the gas station through the entry device 102. Specifically, the method by which the entry device 102 obtains the license plate number is not limited. For example, an image input device can be configured on the entry device 102 to obtain an image of the vehicle's license plate number, and the license plate number information can be extracted based on the image. The image input device can be a camera, and the method of extracting the license plate number information is not limited. For example, the license plate information can be extracted from the image using neural networks or Optical Character Recognition (OCR).

[0077] As an example, the identification information can also include information that combines the identification code and the license plate number, associating the identification code and the license plate number.

[0078] For example, a vehicle with license plate number 123456 enters a gas station through entry device 102, which can generate entry information. This entry information includes the vehicle's identification information, which can be a sequentially generated number "01", a license plate number "123456" recognized by an image input device, or a combination of the sequentially generated number "01" and the license plate number "123456" recognized by an image input device.

[0079] Step 202. In response to receiving the entry information, if there is currently an idle first refueling pile, the control device 101 sends a first instruction message to the information prompting device 103 and a second instruction message to the first refueling robot corresponding to the first refueling pile.

[0080] The first instruction information includes the vehicle's identification information and the first refueling station's identification information, while the second instruction information is used to instruct the first refueling robot to move to a preset position.

[0081] To illustrate with a scenario example, after receiving the entry information, the control device 101 will determine whether there is an available refueling pile 105.

[0082] If there is an available refueling pile 105 at the gas station, the control device 101 sends a first instruction to the information display device 103. This first instruction includes the vehicle's identification information and the identification information of the available first refueling pile, which is displayed to the vehicle via the information display device 103. Specifically, the control device 101 determines which refueling pile 105 is currently available and sends the corresponding identification information of that refueling pile 105 and the vehicle's identification information to the information display device 103.

[0083] For example, after a vehicle with license plate number 123456 enters the gas station through the entry device 102, the control device 101 determines that the first gas station is currently idle, and then sends the identification information "1051" of the first gas station and the vehicle's identification information to the information prompt device 103. The format of the vehicle's identification information can be determined by the description in step 201.

[0084] If there is an available refueling pile 105 at the gas station, the control device 101 sends a second instruction to the first refueling robot corresponding to the first refueling pile. This second instruction instructs the first refueling robot to move to a preset position. The preset position refers to the position pre-set by the refueling robot 104 before refueling begins. Once the refueling robot 104 moves to this preset position, it can proceed with the refueling operation. Specifically, the control device 101 determines which refueling pile 105 is currently available and sends a command to the refueling robot 104 that can move to that pile 105, causing the refueling robot 104 to move to the preset position and prepare to refuel the vehicle.

[0085] Step 203. The information prompting device 103 prompts the vehicle to move to the first refueling pile according to the first instruction information.

[0086] As an example, after receiving the first instruction information, the information prompting device 103 displays the first instruction information to the vehicle. The information prompting device 103 may include an image display device. The specific image display device is not limited; for example, it may be a light-emitting diode (LED) display, a liquid crystal display (LCD), or an organic light-emitting diode (OLED) display. The information prompting device 103 parses the first instruction information to obtain the vehicle's identification information and the identification information of the first gas station. It then displays the vehicle's identification information and the first gas station's identification information to the vehicle through the image display device. This guides the vehicle to move to the preset area of ​​the first gas station, preparing for refueling.

[0087] For example, when the information prompting device 103 receives the first instruction information and parses it to obtain the vehicle license plate number as "123456" and the currently available gas station 105 identifier as "1051", it displays the vehicle license plate number and the currently available gas station 105 identifier from the first instruction information on the image display device. For example, the displayed information could be: "Please proceed to the preset area of ​​gas station 1051 if your vehicle license plate number is 123456".

[0088] Optionally, the information prompting device 103 may also include an audio output device, which prompts the vehicle corresponding to the vehicle identification information in the first instruction information to proceed to the currently available gas station 105. The specific audio output device is not limited; for example, it could be a speaker.

[0089] Step 204. After the vehicle moves to the preset area of ​​the first refueling station, the first refueling robot uses the fuel nozzle installed on the first refueling station to refuel the vehicle.

[0090] Using scenarios as examples, Figure 3 An example layout of a refueling robot and a refueling station. Figure 1 .like Figure 3 As shown, taking two refueling piles and two refueling robots as an example, the refueling piles and refueling robots are in one-to-one correspondence. For example, if refueling pile number 1 is currently idle, refueling robot number 1 can use refueling pile number 1 to refuel vehicles in area A01. In this example, the information displayed by the corresponding information prompting device 103 through the image display device can be: "Please proceed to refueling area A01 if your vehicle license plate number is 123456".

[0091] Using scenarios as examples, Figure 4 An example layout of a refueling robot and a refueling station. Figure 2 .like Figure 4 As shown, this example uses two refueling stations and two refueling robots, with one refueling station corresponding to two refueling robots. The refueling robots can move between different refueling stations, with one robot corresponding to one refueling area. For example, if refueling station #1 is currently idle, refueling robot #1 can refuel vehicles in area A01 via refueling station #1, and refueling robot #2 can also refuel vehicles in area A02 via refueling station #1. Similarly, if refueling station #2 is currently idle, refueling robot #1 can refuel vehicles in area A01 via refueling station #2, and refueling robot #2 can also refuel vehicles in area A02 via refueling station #2.

[0092] Furthermore, Figure 5 An example layout of a refueling robot and a refueling station. Figure 3 .like Figure 5As shown, taking three refueling stations and four refueling robots as an example, in this example, one refueling station can correspond to multiple refueling robots. A track is configured between the refueling stations and the refueling areas. Refueling robots No. 1 and No. 2 can move within the first track. For example, if refueling station No. 1 is currently idle, refueling robot No. 1 can refuel vehicles in refueling area A01 and also refuel vehicles in refueling area A02 via refueling station No. 1; conversely, refueling robot No. 2 can refuel vehicles in refueling area A02 via refueling station No. 1 and also refuel vehicles in refueling area A01 via refueling station No. 1. The first track can be a closed capsule shape. Optionally, refueling station No. 2 has fuel nozzle pick-up and drop-off points on both sides, and refueling station No. 2 corresponds to four refueling robots. Understandably, when refueling station #2 is idle, refueling robot #1 or #2 can be instructed to refuel vehicles in refueling area A01 or A02, and refueling robot #3 or #4 can be instructed to refuel vehicles in refueling area B01 or B02.

[0093] It should be noted that, Figure 5 This is merely an example illustrating an embodiment of the solution; in practical applications, it can be modified according to the layout of the gas station. For example, in Figure 5 Based on the illustrated embodiment, the following can also be configured to the right of refueling pile No. 3: a third track; refueling areas C01 and C02; refueling robot No. 5; refueling robot No. 6; and refueling pile No. 4. For example, the following can also be configured below refueling area A02: a third track; refueling areas A03 and A04; refueling robot No. 5; refueling robot No. 6; and refueling pile No. 4 and refueling pile No. 5. The terms "up," "down," "left," and "right" used here are only for description based on the accompanying drawings; the layout in actual application should not be limited by the direction of the accompanying drawings.

[0094] It should also be noted that Figure 5 The track shown is for illustrative purposes only and its shape is not limited. The refueling robot 104 can also move between the refueling pile 105 and the refueling area by having a built-in route on its mobile device. The track can be a closed shape, such as a rectangle or a circle; or an open shape, such as a straight line or a curve.

[0095] Once a vehicle enters the corresponding refueling area, the refueling robot can refuel the vehicle. However, before refueling, the refueling robot 104 needs to confirm whether the vehicle entering the preset area of ​​the refueling station 105 is the vehicle indicated by the information prompting device 103. Therefore, in one example, the second indication information is also used to indicate the vehicle's first license plate number.

[0096] As the vehicle moves to the preset area of ​​the first refueling station, the first refueling robot captures a second image of the vehicle and obtains the second license plate number of the vehicle in the second image.

[0097] As can be seen from step 202, the second instruction information is sent by the control device 101 to the first refueling robot, instructing the first refueling robot to move to a preset position. In some embodiments, the second instruction information may further include the vehicle's first license plate number. The first license plate number may be the vehicle license plate number identified by the entry device 102 when the vehicle enters the gas station.

[0098] The first refueling robot waits at a preset position for vehicles to move to the preset area of ​​the first refueling station. When a vehicle enters the preset area of ​​the first refueling station, the robot captures an image of the vehicle and extracts the license plate number of the vehicle that entered the preset area of ​​the first refueling station from the captured image, which is then used as the second license plate number.

[0099] After obtaining the second license plate number, the first refueling robot determines whether the second license plate number matches the first license plate number. For example, if the second license plate number matches the first license plate number, the first refueling robot issues an inquiry request to obtain the type and quantity of fuel to be added to the vehicle. The first refueling robot receives the user's input inquiry response, which includes the user's input of the fuel quantity and fuel type. Subsequently, the first refueling robot sends a refueling instruction to the first refueling station, indicating the user's input of the fuel quantity and fuel type.

[0100] If the second license plate number matches the first license plate number, it indicates that the vehicle indicated by the information prompt device 103 has correctly entered the preset area of ​​the corresponding gas station 105. At this time, the first refueling robot needs to query the user of the vehicle for specific refueling information, which may include: the type of fuel to be refueled and the amount of fuel to be refueled. Therefore, after completing the license plate number matching, the first refueling robot can proactively issue an inquiry request, waiting for the user to input the specific refueling information into the first robot.

[0101] In one example, the first refueling robot may include a display screen. After the first refueling robot completes the license plate number matching, the display screen shows a first interface that displays an inquiry request. In response to an input operation on the first interface, the first refueling robot sends a refueling instruction to the first refueling station, the refueling instruction indicating the fuel type and fuel quantity.

[0102] Figure 6 This is an example interface diagram of a refueling robot. Figure 6As shown, in this example, the display screen can be a touchscreen. The first refueling robot displays a first interface 60 on the display screen, which shows multiple fuel types available for the user to select, such as... Figure 6 The interface 60 displays fuel categories such as "92 octane gasoline" and "95 octane gasoline." An input box 601 is also displayed; when the user clicks the input box 601, a numeric keypad 602 pops up. The user can input the fuel quantity using the numeric keypad 602. In response to the input operation on the first interface 60, the first refueling robot sends a refueling command to the first refueling station, indicating the fuel type and quantity entered by the user.

[0103] In another example, the multiple fuel category selection buttons and numeric keypad 602 in the first interface 60 can be physical buttons.

[0104] In another example, the first refueling robot may also include audio equipment (including a speaker and a microphone). After completing the license plate number matching, the first refueling robot can issue a voice prompt through the audio equipment stating, "Please state the type and quantity of fuel you wish to refuel with" (corresponding to the aforementioned inquiry request). The audio equipment receives the user's voice feedback (corresponding to the aforementioned inquiry response), recognizes the voice information through a voice recognition module, and obtains specific refueling information, such as fuel type and quantity. Subsequently, the first refueling robot sends a refueling instruction to the first refueling station, which instructs the user to specify the fuel quantity and type input via voice. Optionally, after recognizing the specific refueling information, the first refueling robot displays the refueling information to the user on a screen and asks for confirmation.

[0105] The following describes the actual process of the first refueling robot refueling the vehicle.

[0106] As an example, if the second license plate number matches the first license plate number, the first refueling robot uses the fuel nozzle installed on the first refueling station to refuel the vehicle.

[0107] Specifically, the first refueling robot includes a robotic arm. For example, the robotic arm of the first refueling robot can be a six-degree-of-freedom robotic arm.

[0108] As an example, the first refueling robot moves to the first refueling station based on the specific refueling information, controls the robotic arm to grab the fuel nozzle corresponding to the fuel type on the first refueling station according to the fuel type, and then returns to the refueling area corresponding to the first refueling station to refuel the vehicle.

[0109] After grabbing the corresponding fuel nozzle, the first refueling robot captures the first image of the vehicle after the vehicle moves to the preset area of ​​the first refueling station. The first image includes the vehicle's fuel tank.

[0110] The pose data of the fuel tank relative to the first refueling robot is determined based on the first image.

[0111] For example, the first refueling robot can be equipped with a binocular camera on its robotic arm to capture images of the vehicle's fuel tank, which serve as the first image. The pose of the fuel tank in space is then calculated based on this first image. The pose includes position and orientation. Position refers to the fuel tank's location in space, such as the spatial coordinates of the tank's center point in the world coordinate system. Orientation refers to the orientation of the fuel tank's plane in space, such as the RPY angles of the fuel tank plane, including roll (rotation around the x-axis), pitch (rotation around the y-axis), and yaw (rotation around the z-axis). From the vehicle's fuel tank's pose in space, the pose data of the fuel tank relative to the first refueling robot can be calculated. Specifically, the pose data is the pose data of the fuel tank relative to the end effector of the first refueling robot's robotic arm. The pose data can be in matrix form.

[0112] The robotic arm may include multiple actuators. For example, the robotic arm may include a first actuator and a second actuator. The first actuator controls the first actuator of the robotic arm to open the vehicle's fuel tank cap based on pose data. The second actuator controls the second actuator of the robotic arm to grasp the fuel nozzle corresponding to the fuel type on the first fuel station, and controls the second actuator of the robotic arm to insert the fuel nozzle into the fuel tank opening based on pose data, thereby refueling the vehicle.

[0113] In the aforementioned example, the case where the second license plate number is the same as the first license plate number is described. Based on the aforementioned example, if the second license plate number is not the same as the first license plate number, the first refueling robot sends a first notification message to the control device 101. The first notification message is used to indicate that the current vehicle is not the vehicle indicated by the control device 101.

[0114] In response to the first notification message, the control device 101 guides the current vehicle to another refueling station 105 via the information prompt device 103.

[0115] For example, if the second license plate number does not match the first license plate number, it indicates that the vehicle currently entering the preset area of ​​gas station 105 is not the vehicle indicated by information prompting device 103. For instance, the vehicle might have mistakenly entered the preset area of ​​another gas station 105, or the user might have misread the information displayed by information prompting device 103. In this case, the first refueling robot sends a first notification message to control device 101, indicating that the current vehicle is not the vehicle indicated by control device 101. Specifically, the first notification message indicates that the vehicle currently entering the preset area of ​​the first gas station is incorrect.

[0116] The control device 101 receives the first notification message and displays it through the information prompting device 103, thereby guiding the vehicle to another gas station 105. Optionally, when the information prompting device 103 displays the first notification message, it can use an audio output device to call staff to assist the vehicle in leaving the preset area of ​​the first gas station.

[0117] Based on any example, the refueling system 10 may further include a waiting area. The waiting area is used when, if no available refueling pile 105 is currently available, the control device 101 sends waiting information to the information prompting device 103. The information prompting device 103 responds to the waiting information by displaying it via an image output device and guiding vehicles to the waiting area. Optionally, the information prompting device 103 may update the waiting information and the currently available refueling piles 105 in real time. When an available refueling pile 105 is available, vehicles are guided sequentially to a preset area containing the available refueling piles 105.

[0118] The refueling system provided in this application embodiment is equipped with intelligent devices such as control devices, entry devices, and information prompting devices, and incorporates a mobile refueling robot. These intelligent devices enable intelligent recognition and guidance of vehicle entry. The mobile refueling robot, based on the user's refueling information, grabs the corresponding fuel nozzle and refuels the vehicle. This achieves intelligent transformation of traditional gas stations, improving refueling efficiency while avoiding large-scale replacement of traditional fuel piles, thereby reducing the cost of intelligent transformation of traditional gas stations.

[0119] After refueling, payment is required, and there are no restrictions on the payment method. For example, payment can be made at the first refueling robot. Figure 7 A schematic diagram of the refueling system provided in this application. Figure 2 ,like Figure 7 As shown, in this embodiment... Figure 1 Based on the embodiment, the refueling system 10 further includes: a departure device 106, which is connected to the control device 101.

[0120] After refueling is completed, the first refueling robot generates payment information based on the amount of fuel added to the vehicle and displays the payment information on the screen.

[0121] For example, after the first refueling robot completes refueling, it generates payment information based on the amount of fuel added to the vehicle. The amount of fuel added can be the specific amount of fuel entered by the user. In some special cases, if the actual amount of fuel added differs from the amount entered by the user, the actual amount added can be used to generate payment information. Specifically, the first refueling station sends the actual amount of fuel added to the first refueling robot, which then generates payment information based on that amount. As an example, the first refueling robot has a built-in unit price corresponding to different fuel types and calculates the payment information for this refueling based on the amount of fuel added. Furthermore, the payment information can be displayed on the first refueling robot's screen. The payment information can be a payment QR code, which the user scans to make payment.

[0122] After confirming successful payment, the first refueling robot sends a second notification message to the control device 101. This second notification message indicates that refueling is complete and payment was successful. For example, the second notification message includes vehicle identification information and payment completion information. It is understood that the control device 101 confirms that refueling has been completed and payment was successful upon receiving the second notification message.

[0123] The control device 101 sends a third notification message to the departure device 106. Specifically, after receiving the second notification message, the control device 101 generates a third notification message and sends it to the departure device 106. As an example, the third notification message includes the vehicle's identification information.

[0124] When the departure device 106 detects that a vehicle has reached the exit, it controls the lifting barrier at the exit to rise.

[0125] As an example, the departure device 106 may include an image input device that identifies the vehicle's license plate number when the vehicle arrives at the exit and compares it with the vehicle identification information in the third notification message. If the vehicle license plate number identified by the departure device 106 matches the vehicle identification information in the third notification message, the device controls the gate at the exit to lift, allowing the vehicle to leave.

[0126] Optionally, if the vehicle license plate number identified by the departure device 106 is inconsistent with the vehicle identification information in the third notification message, an error message is generated and sent to the control device 101. The control device 101 forwards the error message to the information prompting device 103, which then prompts staff to go to the exit for processing.

[0127] After refueling, payment can be made at the departure device 106 in addition to the first refueling robot. Alternatively, the refueling system 10 may also include a departure device 106 connected to the control device 101.

[0128] After refueling is completed, the first refueling robot sends a fourth notification message to the control device 101. The fourth notification message indicates that refueling is complete and includes the amount and type of fuel added. As an example, the fourth notification message may also include vehicle identification information.

[0129] To illustrate with a scenario example, after the first refueling robot completes refueling, it generates a fourth notification message based on the amount and type of fuel added to the vehicle, and sends the fourth notification message to the control device 101.

[0130] The control device 101 generates the vehicle's payment information based on the fourth notification message and sends the vehicle's payment information to the departure device 106.

[0131] As an example, after receiving the fourth notification message, the control device 101 parses the message to obtain the amount and type of fuel added to the vehicle. The control device 101 has a built-in unit price for different fuel types, calculates the payment information for this refueling based on the amount of fuel added, and sends the payment information to the departure device 106.

[0132] When the vehicle arrives at the exit, the departure device 106 issues a prompt message to remind the user to make payment. After the user successfully makes payment, the departure device 106 controls the gate at the exit to lift.

[0133] As an example, the departure device 106 may include an image input device that identifies the vehicle's license plate number when the vehicle arrives at the exit and compares it with the vehicle identification information in the fourth notification message. If the license plate number identified by the departure device matches the vehicle identification information in the fourth notification message, a prompt message is issued. Further, the departure device 106 may also include a payment display screen. Specifically, the departure device 106 issues a prompt message through the payment display screen, prompting the user to make payment. The prompt message may be a payment QR code, which the user scans to make payment. After the user completes the payment, the departure device 106 controls the gate at the exit to lift, allowing the vehicle to exit.

[0134] Optionally, if the vehicle license plate number identified by the departure device 106 is inconsistent with the vehicle identification information in the fourth notification message, an error message is generated and sent to the control device 101. The control device 101 forwards the error message to the information prompting device 103, which then prompts staff to go to the exit for processing.

[0135] The refueling system provided in this application embodiment is also equipped with a departure device. Payment is completed on the first refueling robot, and the vehicle leaves the gas station via the departure device; or payment is made on the departure device, and the vehicle leaves the gas station via the departure device after payment is completed. This avoids the need for the vehicle to leave the refueling area only after the user has completed payment, as is common in traditional gas stations, achieving intelligent transformation of refueling payment and further improving refueling efficiency.

[0136] The above embodiment illustrates the entire process of refueling a vehicle in a refueling system. During refueling, the refueling robot calculates the pose data of the fuel tank relative to itself by acquiring images of the fuel tank. Based on this pose data, the refueling robot controls its robotic arm to refuel the vehicle. The process by which the refueling robot calculates the pose data of the fuel tank relative to itself from the acquired images will be explained below.

[0137] Figure 8 A flowchart illustrating the refueling control method provided in this application is shown below. Figure 8 As shown, the refueling control method provided in this embodiment is applied to the refueling robot 104 of the refueling system 10 provided in the aforementioned embodiment. The refueling robot 104 is equipped with a binocular camera, and the refueling control method includes:

[0138] Step 301. Acquire the first image of the vehicle from two shooting angles captured by the binocular camera. The first image includes the vehicle's fuel tank.

[0139] Specifically, the binocular camera includes a left camera and a right camera. As an example, the left and right cameras respectively capture images of the vehicle's fuel tank, resulting in the left and right images of the first image. To enable subsequent feature extraction, as an example, the first images of the vehicle from both shooting perspectives are preprocessed, including at least one of the following: image filtering and image contrast correction.

[0140] Specifically, preprocessing is performed on the left and right images of the first image separately. As an example, taking the left image of the first image, image filtering is performed on the left image. Specifically, median filtering is used to filter the left image. The first pixel is selected in the left image, and a fixed-size rectangular window is constructed with the coordinates of the first pixel as the center point. For example, the fixed-size rectangular window can be 5×5, with units in pixels. It can be understood that the 5×5 rectangular window includes 25 pixels, including the first pixel. These 25 pixels are sorted according to their grayscale values, and the median grayscale value is selected to update the grayscale value of the first pixel. Here, the grayscale value of a pixel refers to the brightness level of the pixel, usually represented by an integer value, with a value range between 0 and 255. 0 represents black, 255 represents white, and values ​​in between represent different levels of gray.

[0141] It should be noted that if the rectangular window centered on the coordinates of the first pixel extends beyond the boundary of the left image, the portion extending beyond the boundary is not sorted; only the pixels within the rectangular window and within the boundary of the left image are sorted according to their grayscale values. The steps for image filtering the right image are described above for image filtering the left image, and will not be repeated here.

[0142] Another example, taking the left image of the first image as an example, involves performing image contrast correction on the left image. Specifically, the Gamma correction method is used to correct the image contrast of the left image. Gamma correction is a common non-linear image correction method used to adjust the contrast and brightness of an image. The Gamma correction function can improve the proportion of high and low grayscale pixels in the image, thereby enhancing the image contrast. As an example, the expression for Gamma correction can be:

[0143]

[0144] In formula (1), I out I represents the grayscale value of the output pixel. in This represents the grayscale value of the input pixel, and γ represents the Gamma coefficient, used to control contrast adjustment. Specifically, when γ > 1, it enhances the detail in dark areas of the image and reduces the contrast in bright areas; when γ < 1, it reduces the detail in dark areas of the image and enhances the contrast in bright areas.

[0145] The steps for image contrast correction of the right image are the same as those for image contrast correction of the left image described above, and will not be repeated here.

[0146] It should be noted that when preprocessing the first image, either image filtering or image contrast correction can be performed individually, or they can be performed sequentially. The order of execution is not limited.

[0147] Step 302. Extract the fuel tank feature points from the first image of the vehicle from the two shooting perspectives.

[0148] Specifically, feature points of the fuel tank in the left and right images of the first image are extracted separately. As an example, the extraction of feature points of the fuel tank in the left image is taken as an example. First, image feature points are extracted from the left image; the specific method of extraction is not limited. For example, image feature points in the left image can be extracted using a non-extreme value suppression method. The non-extreme value suppression method involves comparing the grayscale value of the first pixel in the image with other pixels within a preset neighborhood. If the grayscale value of the first pixel is the maximum or minimum value of other pixels in the neighborhood, then the first pixel is retained as an image feature point.

[0149] After extracting the image feature points from the left image, they are matched with the image feature points of the reference image. Specifically, a preliminary judgment on the matching of image feature points in the left image and the reference image is made by constructing a Hessian matrix. The Hessian matrix is ​​a square matrix describing the local curvature of a function and is used to detect reliable edge points in an image. Hessian matrices are constructed for both the left and reference images, and the traces of the Hessian matrices of the image feature points in both images are calculated. If the traces of the Hessian matrices of the image feature points in both the left and reference images are positive, it indicates that the brightness values ​​of the image feature points in both images are greater than the brightness values ​​of the background region, and a preliminary judgment is made that the image feature points in the left image match those in the reference image. Alternatively, if the traces of the Hessian matrices of the image feature points in both images are negative, it indicates that the brightness values ​​of the image feature points in both images are less than the brightness values ​​of the background region, and a preliminary judgment is made that the image feature points in the left image match those in the reference image. If the traces of the Hessian matrices of the image feature points in the left image and the reference image are one positive and one negative, it indicates that the brightness values ​​of the image feature points in the left image and the reference image are different from the brightness values ​​of the background region, and the image feature points of the left image are discarded.

[0150] Generate feature symbols for the feature points of the left image and reference feature symbols for the feature points of the reference image. The feature symbols for the feature points of the left image and the reference feature symbols for the feature points of the reference image can be in vector form. Calculate the Euclidean distance between the feature symbols for the feature points of the left image and the reference feature symbols for the feature points of the reference image. As an example, the expression for the Euclidean distance can be:

[0151]

[0152] In formula (2), d represents the Euclidean distance, (x1,y1) represents the feature symbol of the image feature point of the left image, and (x2,y2) represents the reference feature symbol of the image feature point of the reference image.

[0153] Taking the first feature point corresponding to the first feature symbol in the left image as an example, calculate the Euclidean distance with the reference feature symbols of all image feature points in the reference image. The shortest Euclidean distance is recorded as the nearest neighbor distance, and the distance whose Euclidean distance is only longer than the nearest neighbor distance is recorded as the second nearest neighbor distance. If the nearest neighbor distance is less than a preset proportion of the second nearest neighbor distance, then match the image feature point in the reference image corresponding to that nearest neighbor distance with the first image feature point in the left image, thus using the first image feature point in the left image as the fuel tank feature point. The preset proportion can be 50%. This process is repeated for all image feature points in the left image, matching them with the image feature points in the reference image to determine all fuel tank feature points in the left image.

[0154] The steps for determining all fuel tank feature points in the right image are the same as in the previous example, and will not be repeated here.

[0155] Step 303. Based on the camera parameters of the binocular camera, transform the fuel tank feature points in the first image of the vehicle from the two shooting perspectives to the world coordinate system to obtain the point cloud data of the fuel tank.

[0156] As an example, the camera parameters of a stereo camera can include intrinsic parameters, distortion coefficients, rotation matrix, and translation vector. The intrinsic parameters include the camera's focal length, principal point coordinates, and pixel size, used to transform the coordinates from the image coordinate system to the camera coordinate system. Distortion coefficients, including radial and tangential distortion parameters, are used to correct lens distortion. The rotation matrix describes the rotation relationship between the camera coordinate system and the world coordinate system. The translation vector describes the straight-line distance from the origin of the camera coordinate system to the origin of the world coordinate system. Using the parameters of the stereo camera, the fuel tank feature points in the first image can be transformed from the image coordinate system to the world coordinate system.

[0157] Stereo matching is performed on the fuel tank feature points in the left and right images of the first image to obtain point cloud data. Specifically, the fuel tank feature points in the left and right images are first matched, that is, the fuel tank feature points located at the same position in the fuel tank in the left and right images are matched. The specific matching process is similar to the process of matching image feature points between the left image and the reference image in the previous example. Hessian matrices of the fuel tank feature points in the left and right images are constructed respectively to make a preliminary judgment on whether they correspond. Feature symbols of the fuel tank feature points in the left image are generated, feature symbols of the fuel tank feature points in the right image are generated, Euclidean distance is calculated, and it is determined whether the nearest neighbor distance and the second nearest neighbor distance meet the above-mentioned preset ratio. If the above-mentioned preset ratio is met, it is determined that the fuel tank feature points in the left image and the fuel tank feature points in the right image correspond. It can be understood that at this time, the fuel tank feature points in the left image and the corresponding fuel tank feature points in the right image jointly describe a point of the actual vehicle fuel tank in the world coordinate system. However, due to the different shooting angles, the point describing the actual vehicle fuel tank appears in different positions in the left and right images.

[0158] Therefore, disparity information is calculated based on the fuel tank feature points in the corresponding left and right images. Specifically, disparity information is the difference in horizontal pixel position offset between the images captured by the left and right cameras at different viewpoints, also known as the disparity value. A larger disparity value indicates that the object is closer to the binocular cameras; a smaller disparity value indicates that the object is farther away. The depth information of the fuel tank feature points can be obtained through the disparity value. Thus, the three-dimensional spatial coordinates of the actual vehicle fuel tank described by the fuel tank feature points in the corresponding left and right images are obtained.

[0159] Repeat the above steps until all corresponding fuel tank feature points in the left and right images are traversed, to obtain a set of three-dimensional spatial coordinates of the actual vehicle fuel tank, thus obtaining the point cloud data of the fuel tank.

[0160] Step 304. Perform 3D reconstruction based on the point cloud data of the fuel tank to determine the pose data of the fuel tank relative to the refueling robot.

[0161] As an example, the point cloud data of the fuel tank is first preprocessed to remove noise and outliers. Surface reconstruction is then performed on the preprocessed point cloud data. The specific surface reconstruction method is not limited; for example, a meshing algorithm can be used. This meshing algorithm could be the Delaunay triangulation algorithm, which is an algorithm for generating polymorphic surface meshes from point cloud data.

[0162] The specific steps of the Delaunay triangulation algorithm include:

[0163] Step 3041. Perform mesh initialization and construct a super triangle that encloses all points. The three vertices of this super triangle are usually located outside the minimum convex hull of the point set.

[0164] Step 3042. Insert the points one by one into the existing triangles. For each point, find the triangle containing the point. Starting from the triangle containing the point, search and detect the circumcircles of the triangles adjacent to it. Delete the common edges contained in the circumcircles to form a polygonal cavity. Connect the point to all vertices of the polygonal cavity to form a new triangle.

[0165] Step 3043. Repeat step 3042 to complete the insertion of all points, delete the super triangles constructed during initialization, and obtain the surface mesh of the fuel tank.

[0166] Optionally, texture mapping can be performed based on the constructed fuel tank surface mesh. Mapping the image texture corresponding to the triangular mesh to the constructed fuel tank surface mesh can more realistically reflect the texture of the actual fuel tank in the fuel tank surface mesh.

[0167] The pose data of the fuel tank is calculated based on the surface mesh of the fuel tank. Specifically, as an example, the normal vector of the fuel tank surface can be fitted by the surface mesh of the fuel tank, and the pose matrix of the fuel tank can be obtained from the normal vector. The pose data of the fuel tank relative to the refueling robot 104 can be obtained from the initial pose matrix of the robotic arm of the refueling robot 104.

[0168] Step 305. Based on the pose data, control the robotic arm of the refueling robot to open the fuel tank cap of the vehicle.

[0169] As an example, the first actuator of a refueling robot's robotic arm is controlled based on pose data to open the vehicle's fuel tank cap. Specifically, based on robot inverse kinematics calculations, the rotation angle of each robotic arm can be calculated from the pose of the end effector. As an example, if the refueling robot's robotic arm is a six-degree-of-freedom (DOF) arm, the target rotation angle for each DDF of the robotic arm to reach that target pose can be calculated based on the target pose of the first actuator, i.e., the pose matrix of the fuel tank. Similarly, the initial rotation angle for each DDF of the robotic arm is calculated based on the initial pose matrix of the first actuator. The difference between the target rotation angle and the initial rotation angle for each DDF yields the relative rotation angle for each DDF. By controlling the motor of each DDF to execute this relative rotation angle through the refueling robot's controller, the first actuator of the refueling robot's robotic arm can be controlled based on pose data to open the vehicle's fuel tank cap.

[0170] Step 306. After the robotic arm grabs the fuel nozzle, control the robotic arm to insert the fuel nozzle into the fuel tank to refuel the vehicle.

[0171] As an example, the second actuator of the refueling robot's robotic arm, controlled based on pose data, inserts the fuel nozzle into the fuel tank. The specific calculation process can be found in the description of step 305, and will not be repeated here.

[0172] The refueling control method provided in this application uses a binocular camera to acquire left and right images of the fuel tank, respectively, and performs feature recognition to obtain fuel tank feature points in the left and right images. The fuel tank feature points are transformed to a world coordinate system through camera calibration, and 3D point cloud data of the fuel tank feature points in the world coordinate system is obtained based on stereo matching technology. A 3D model is reconstructed based on the 3D point cloud data, and the pose data of the fuel tank is obtained based on the 3D model. Based on the pose data of the fuel tank, a robotic arm is controlled to open the fuel tank cap and insert the fuel nozzle for refueling. This automates the refueling process with a refueling robot, improves refueling efficiency, and enhances the intelligence level of gas stations.

[0173] Figure 9 A schematic diagram of the refueling control device provided in this application is shown below. Figure 9 As shown, the refueling control device 40 provided in this embodiment includes:

[0174] The acquisition module 401 is used to acquire the first image of the vehicle from two shooting angles captured by the binocular camera. The first image includes the vehicle's fuel tank.

[0175] Processing module 402 is used to extract fuel tank feature points from the first image of the vehicle from two shooting perspectives respectively;

[0176] The processing module 402 is also used to convert the fuel tank feature points in the first image of the vehicle from the two shooting perspectives to the world coordinate system according to the camera parameters of the binocular camera, and obtain the point cloud data of the fuel tank.

[0177] The processing module 402 is also used to perform three-dimensional reconstruction based on the point cloud data of the fuel tank to determine the pose data of the fuel tank relative to the refueling robot.

[0178] The processing module 402 is also used to control the robotic arm of the refueling robot to open the fuel tank cap of the vehicle based on the pose data.

[0179] The processing module 402 is also used to control the robotic arm to insert the fuel nozzle into the fuel tank after the robotic arm grabs the fuel nozzle, so as to refuel the vehicle.

[0180] In one possible implementation, before extracting the fuel tank feature points from the first image of the vehicle from the two shooting perspectives, the method further includes:

[0181] The first images of the vehicle from two shooting perspectives are preprocessed, including at least one of the following: image filtering and image contrast correction.

[0182] The refueling control device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0183] Figure 10 This is a structural diagram of the refueling robot provided in this application. Figure 10 As shown, the refueling robot 104 provided in this embodiment includes: a binocular camera 1041, a robotic arm 1043, and a processor 1042;

[0184] The binocular camera 1041 is used to capture a first image of the vehicle from two shooting angles after the vehicle moves to a preset area of ​​the refueling station. The first image includes the vehicle's fuel tank.

[0185] The processor 1042 is used to execute the refueling control method as described in the above method embodiment.

[0186] The specific implementation process of processor 1042 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0187] In one possible implementation, the refueling robot 104 further includes a memory. The memory stores computer-executable instructions that cause the processor 1042 to execute the refueling control method as described in the above method embodiment.

[0188] In one possible implementation, the refueling robot 104 also includes a communication component. The processor 1041, memory, and communication component are connected via a bus.

[0189] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0190] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0191] A bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. There is no limitation to having only one bus or only one type of bus.

[0192] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0193] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0194] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0195] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0196] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0197] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0198] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0199] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0200] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0201] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A refueling system characterized by, The system comprises: a control device, an approach device, an information prompting device, a plurality of refueling robots, and a plurality of refueling piles; the control device is connected with the approach device, the information prompting device, and the plurality of refueling robots respectively; the approach device is configured to send approach information to the control device when a vehicle approaches, the approach information comprising identification information of the vehicle; the control device is configured to, in response to receiving the approach information, send first indication information to the information prompting device and second indication information to a first refueling robot corresponding to a first refueling pile, if there is a free first refueling pile at present, the first indication information comprising the identification information of the vehicle and the identification information of the first refueling pile, the second indication information being used for instructing the first refueling robot to move to a preset position; the information prompting device is configured to prompt the vehicle to move to the first refueling pile according to the first indication information; the first refueling robot is configured to, after the vehicle moves to a preset area of the first refueling pile, use a fuel gun installed on the first refueling pile to refuel the vehicle.

2. The refueling system of claim 1, wherein, The first refueling robot comprises a mechanical arm, and the first refueling robot is configured to: collect a first image of the vehicle after the vehicle moves to the preset area of the first refueling pile, the first image comprising a fuel tank of the vehicle; determine pose data of the fuel tank relative to the first refueling robot according to the first image; control the mechanical arm to open a fuel tank cover of the vehicle according to the pose data, and control the mechanical arm to grab a fuel gun corresponding to a fuel category on the first refueling pile according to the fuel category to refuel the vehicle.

3. A refuelling system according to claim 1 or 2, characterised in that, The second indication information is also used for indicating a first license plate number of the vehicle, and the first refueling robot is further configured to: collect a second image of the vehicle during the process that the vehicle moves to the preset area of the first refueling pile, and obtain a second license plate number of the vehicle in the second image; if the second license plate number is consistent with the first license plate number, use the fuel gun installed on the first refueling pile to refuel the vehicle.

4. The refueling system of claim 3, wherein the first refueling robot is configured to, if the second license plate number is consistent with the first license plate number, send an inquiry request, the inquiry request being used for requesting to obtain a fuel category and a fuel amount to be added by the vehicle; receive an inquiry response input by a user, and send a refueling instruction to the first refueling pile, the refueling instruction being used for instructing the fuel amount and the fuel category.

5. The refueling system of claim 4, wherein the first refueling robot further comprises a display screen, and the display screen displays a first interface, the first interface displaying the inquiry request; the first refueling robot is configured to, in response to an input operation acting on the first interface, send the refueling instruction to the first refueling pile.

6. The refueling system of any one of claims 2 to 5, wherein The first refueling robot is also configured to send a first notification message to the control device if the second license plate number is inconsistent with the first license plate number, and the first notification message is used to indicate that the current vehicle is not the vehicle indicated by the control device. The control device is also configured to guide the current vehicle to other refueling piles through the information prompting device in response to the first notification message.

7. The refueling system of claim 6, wherein, The refueling system also comprises a departure device connected with the control device. The first refueling robot is also configured to generate payment information according to the amount of fuel added by the vehicle after the refueling is completed. The payment information is displayed on the display screen. After confirming the successful payment, a second notification message is sent to the control device, and the second notification message is used to indicate that the vehicle has completed refueling and the payment is successful. The control device is also configured to send a third notification message to the departure device. The departure device is configured to control the lift bar at the exit to be lifted when the vehicle is identified to reach the exit.

8. The refueling system of claim 6, wherein, The refueling system also comprises a departure device connected with the control device. The first refueling robot is also configured to send a fourth notification message to the control device after the refueling is completed, and the fourth notification message is used to indicate that the vehicle has completed refueling, the amount of fuel added by the vehicle and the fuel category. The control device is also configured to generate payment information of the vehicle according to the fourth notification message, and send the payment information of the vehicle to the departure device. The departure device is configured to issue a prompt information when the vehicle reaches the exit, and the prompt information is used to prompt the user to pay, and the lift bar at the exit is controlled to be lifted after the user successfully pays.

9. A refueling control method characterized by, The refueling robot applied to the refueling system of any one of claims 1-8 is provided with a binocular camera, and the method comprises: obtaining a first image of the vehicle under two shooting angles collected by the binocular camera, the first image comprising a fuel tank of the vehicle; extracting fuel tank feature points in the first image of the vehicle under the two shooting angles respectively; converting the fuel tank feature points in the first image of the vehicle under the two shooting angles to a world coordinate system according to camera parameters of the binocular camera to obtain point cloud data of the fuel tank; performing three-dimensional reconstruction according to the point cloud data of the fuel tank to determine pose data of the fuel tank relative to the refueling robot; controlling a mechanical arm of the refueling robot to open a fuel tank cover of the vehicle based on the pose data; controlling the mechanical arm to insert the oil gun into the fuel tank after the mechanical arm grasps the oil gun to refuel the vehicle.

10. The method of claim 9, wherein, Before the fuel tank feature points in the first image of the vehicle under the two shooting angles are extracted respectively, the method further comprises: preprocessing the first image of the vehicle under the two shooting angles, and the preprocessing comprises at least one of image filtering and image contrast correction.

11. A refueling robot, characterized in that The refueling robot comprises: a binocular camera, a mechanical arm and a processor; the binocular camera is configured to collect a first image of a vehicle under two shooting angles after the vehicle moves to a preset area of a refueling pile, and the first image comprises a fuel tank of the vehicle; The processor is configured to perform the method of claim 9 or 10.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer-executable instructions which, when executed by the processor, implement the method of claim 9 or 10.

13. A computer program product comprising a computer program which, when executed by the processor, implements the method of claim 9 or 10.

Images