Intelligent refueling robot and control method therefor

Abstract

The present application relates to the technical field of computers. Provided are an intelligent refueling robot and a control method therefor. The method comprises: controlling an image collection module to obtain processed first image data; controlling a path planning module to obtain a first planned path on the basis of the processed first image data and first real-time position data of an execution module; on the basis of the first planned path, controlling the execution module to sequentially open a fuel filler door and a fuel cap of a vehicle; controlling the image collection module to obtain processed second image data; controlling the path planning module to obtain a second planned path on the basis of the processed second image data and second real-time position data of the execution module; on the basis of the second planned path, controlling the execution module to execute a refueling operation; and upon detecting that refueling is completed, controlling the execution module to reversely operate along the first planned path, so as to close the fuel filler door and the fuel cap of the vehicle. Therefore, the operation complexity of a self-service gas station is reduced, thereby improving the applicability, and also improving the user experience.

Description

Intelligent refueling robot and its control method

[0001] This application claims priority to Chinese Patent Application No. 202411572722.4, filed on November 5, 2024, entitled “Intelligent Refueling Robot and Control Method Thereof”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of computer technology, and in particular to an intelligent refueling robot and its control method. Background Technology

[0003] With the continuous growth of car ownership, the demand for gas station services is also increasing. In order to improve the operational efficiency of gas stations and reduce labor costs, self-service refueling has been widely introduced.

[0004] Currently, self-service gas stations are usually equipped with self-service refueling machines, where users need to complete a series of operations themselves, including selecting fuel, paying, picking up the nozzle to refuel, monitoring the refueling progress, and returning the nozzle.

[0005] However, existing self-service gas stations are complex to operate and have poor applicability, resulting in a poor user experience. Summary of the Invention

[0006] This application provides an intelligent refueling robot and its control method to solve the technical problems of high operational complexity, poor applicability, and unsatisfactory user experience in existing self-service gas stations. It can reduce the operational complexity of self-service gas stations, improve applicability, and enhance the user experience.

[0007] In a first aspect, embodiments of this application provide an intelligent refueling robot, comprising:

[0008] The system includes an image acquisition module, a path planning module, an execution module, and a central controller; the image acquisition module, path planning module, and execution module are electrically connected to the central controller; the execution module is located on one side of the fuel dispenser.

[0009] The execution module includes a lateral movement module, a longitudinal movement module, a rotation module, a fixed plate, a robotic arm, a refueling assembly, a fuel tank cap, and a refueling cap opening and closing assembly. The lateral movement module is mounted on one side of the fuel dispenser via a support rod. The longitudinal movement module is located at the output end of the lateral movement module. The rotation module is located at the output end of the longitudinal movement module. The fixed plate is located at the output end of the rotation module. The robotic arm, fuel tank cap, and refueling cap opening and closing assembly are all mounted on the fixed plate. The refueling assembly is mounted on the robotic arm. The lateral movement module, longitudinal movement module, rotation module, robotic arm, refueling assembly, fuel tank cap, and refueling cap opening and closing assembly are electrically connected to the central controller.

[0010] The central controller is used for:

[0011] The image acquisition module is controlled to acquire first image data of any vehicle within a preset range of the fuel dispenser, the first image data is preprocessed to obtain processed first image data, and the processed first image data is sent to the path planning module, wherein the first image data is image data of the vehicle with the fuel tank cap and fuel filler cap not opened.

[0012] The path planning module is controlled to obtain a first planned path based on the processed first image data and the first real-time position data of the execution module;

[0013] According to the first planned path, the execution module controls the lateral movement module, the longitudinal movement module, the rotation module, the fuel tank cap opening and closing assembly, and the refueling cap opening and closing assembly to sequentially open the vehicle's fuel tank cap and refueling cap.

[0014] The system controls the image acquisition module to acquire second image data of the vehicle, preprocesses the second image data to obtain processed second image data, and sends the processed second image data to the path planning module, wherein the second image data is image data of the vehicle with the fuel tank cap and filler cap open;

[0015] The path planning module is controlled to obtain a second planned path based on the processed second image data and the second real-time position data of the execution module;

[0016] The robotic arm and refueling assembly of the execution module are controlled to perform refueling operations according to the second planned path;

[0017] Upon detection that refueling is complete, the lateral movement module, longitudinal movement module, rotation module, fuel tank cap opening and closing assembly, and refueling cap opening and closing assembly of the execution module are controlled to reverse the first planned path to close the vehicle's fuel tank cap and refueling cap.

[0018] Based on the above technical content, the central controller controls the image acquisition module to obtain processed first image data; the control path planning module obtains a first planned path based on the processed first image data and the first real-time position data of the execution module; according to the first planned path, the control execution module sequentially opens the vehicle's fuel tank cap and refueling cap; the central controller controls the image acquisition module to obtain processed second image data; the control path planning module obtains a second planned path based on the processed second image data and the second real-time position data of the execution module; according to the second planned path, the control execution module executes the refueling operation; upon detecting that refueling is complete, the control execution module closes the vehicle's fuel tank cap and refueling cap. This reduces the operational complexity of self-service gas stations, improves applicability, and enhances the user experience.

[0019] Optionally, in the intelligent refueling robot described above, the lateral movement module includes a lateral movement shell, a first rotary motor, a first threaded rod, a first sliding rod, and a first mounting block; the support rod is provided on one side of the refueling machine; the lateral movement shell is provided on the top of the support rod, and the first rotary motor is provided on one side of the lateral movement shell; a rotating component is provided on the other side of the lateral movement shell; the output end of the first rotary motor extends into the interior of the lateral movement shell; the first threaded rod is provided between the first rotary motor and the rotating component; the first sliding rod is fixedly provided on the upper and lower sides of the first threaded rod, the back of the first mounting block has a threaded hole corresponding to the first threaded rod, and a sliding hole corresponding to the first sliding rod, and the first mounting block is threadedly connected to the first threaded rod through the threaded hole and slidably connected to the first sliding rod through the sliding hole; the longitudinal movement module is provided on the front of the first mounting block, and a ranging module is provided between the inner side of the lateral movement shell and the first mounting block; the first rotary motor and the ranging module are electrically connected to the central controller.

[0020] Furthermore, the design of the lateral movement module enables the entire refueling process to be automated, reducing the operational complexity of self-service gas stations and minimizing human intervention.

[0021] Optionally, in the intelligent refueling robot described above, the longitudinal movement module includes a longitudinal movement shell, a second rotary motor, a second threaded rod, a second slide rod, and a second mounting block; the longitudinal movement shell is disposed on the front of the first mounting block; the second rotary motor is disposed on the upper side of the longitudinal movement shell; a rotating component is disposed on the lower side of the longitudinal movement shell; the output end of the second rotary motor extends into the interior of the longitudinal movement shell; the second threaded rod is disposed between the second rotary motor and the rotating component; the second slide rod is fixedly disposed on the left and right sides of the second threaded rod; a threaded hole and a sliding hole are disposed on the back of the second mounting block; the second mounting block is threadedly connected to the second threaded rod through the threaded hole and slidably connected to the second slide rod through the sliding hole; the rotary module is disposed on the front of the second mounting block; a ranging module is disposed between the upper interior of the longitudinal movement shell and the second mounting block; the second rotary motor and the ranging module are electrically connected to the central controller.

[0022] Furthermore, the design of the longitudinal movement module enables the entire refueling process to be automated, reducing the operational complexity of self-service gas stations and minimizing human intervention.

[0023] Optionally, in the intelligent refueling robot described above, the rotating module includes a motor mounting plate and a third rotating motor; wherein the motor mounting plate is located at the center of the front of the second mounting block; the third rotating motor is located at the center of the motor mounting plate; the fixing plate is located at the output end of the third rotating motor; and the third rotating motor is electrically connected to the central controller.

[0024] Furthermore, the rotating module design enables the intelligent refueling robot to adapt to a wider range of vehicle types and refueling scenarios, improving its applicability and enhancing the user experience.

[0025] Secondly, embodiments of this application provide a control method for an intelligent refueling robot, applied to the central controller in the intelligent refueling robot described in the first aspect, comprising:

[0026] The image acquisition module acquires first image data of any vehicle within a preset range of the fuel dispenser, preprocesses the first image data to obtain processed first image data, and sends the processed first image data to the path planning module. The first image data is the image data of the vehicle with the fuel tank cap and fuel filler cap not open.

[0027] The path planning module is controlled to obtain a first planned path based on the processed first image data and the first real-time position data of the execution module;

[0028] According to the first planned path, the execution module controls the lateral movement module, longitudinal movement module, rotation module, fuel tank cap and refueling cap opening and closing assembly to sequentially open the vehicle's fuel tank cap and refueling cap.

[0029] The system controls the image acquisition module to acquire second image data of the vehicle, preprocesses the second image data to obtain processed second image data, and sends the processed second image data to the path planning module, wherein the second image data is image data of the vehicle with the fuel tank cap and filler cap open;

[0030] The path planning module is controlled to obtain a second planned path based on the processed second image data and the second real-time position data of the execution module;

[0031] The robotic arm and refueling components of the execution module are controlled to perform refueling operations according to the second planned path;

[0032] Upon detection that refueling is complete, the lateral movement module, longitudinal movement module, rotation module, fuel tank cap opening and closing assembly, and refueling cap opening and closing assembly of the execution module are controlled to reverse the first planned path to close the vehicle's fuel tank cap and refueling cap.

[0033] Based on the above technical content, the central controller controls the image acquisition module to obtain processed first image data; the control path planning module obtains a first planned path based on the processed first image data and the first real-time position data of the execution module; according to the first planned path, the control execution module sequentially opens the vehicle's fuel tank cap and refueling cap; the central controller controls the image acquisition module to obtain processed second image data; the control path planning module obtains a second planned path based on the processed second image data and the second real-time position data of the execution module; according to the second planned path, the control execution module executes the refueling operation; upon detecting that refueling is complete, the control execution module closes the vehicle's fuel tank cap and refueling cap. This reduces the operational complexity of self-service gas stations, improves applicability, and enhances the user experience.

[0034] Optionally, in the method described above, controlling the path planning module to obtain a first planned path based on the processed first image data and the first real-time position data of the execution module includes: controlling the ranging module of the execution module to acquire the first real-time position data of the execution module; controlling the path planning module to use a trained first target recognition model to calculate the position data of the fuel tank cap and fuel filler cap of the vehicle in the processed first image data; and controlling the path planning module to use a preset path planning algorithm to calculate the first planned path from the first real-time position data to the position data of the fuel tank cap and fuel filler cap.

[0035] Furthermore, the central controller calculates the first planned path through the control path planning module, reducing the need for manual intervention, improving the automation level of the intelligent refueling robot, and reducing the operational complexity of self-service gas stations.

[0036] Optionally, in the method described above, controlling the path planning module to obtain a second planned path based on the processed second image data and the second real-time position data of the execution module includes: controlling the ranging module of the execution module to acquire the second real-time position data of the execution module; controlling the path planning module to use a trained second target recognition model to calculate the position data of the vehicle's fuel filler in the processed second image data; controlling the path planning module to use a preset path planning algorithm to calculate the initial path from the second real-time position data to the position data of the fuel filler; controlling the path planning module to adjust the initial path according to the degree of freedom adjustment component of the robotic arm of the execution module to obtain an adjusted path; and controlling the path planning module to perform interpolation processing on the adjusted path to obtain the second planned path.

[0037] Furthermore, the central controller, through the control path planning module, uses a preset path planning algorithm to calculate the initial path from the second real-time location data to the refueling nozzle location data. The control path planning module adjusts the initial path based on the degree of freedom adjustment components of the robotic arm in the execution module to obtain an adjusted path. The control path planning module then performs interpolation processing on the adjusted path to obtain the second planned path. This improves the feasibility of the planned path and further enhances the automation level of the intelligent refueling robot.

[0038] Optionally, according to the method described above, the control image acquisition module acquires first image data of any vehicle within a preset range of the fuel dispenser, and preprocesses the first image data to obtain processed first image data, including: controlling the image acquisition module to calibrate the parameters of the acquisition module using a preset calibration template; controlling the image acquisition module to adjust the direction and angle of the acquisition module according to the vehicle; controlling the image acquisition module to acquire first image data of the vehicle with the fuel tank cap and fuel filler cap not open; controlling the image acquisition module to perform grayscale processing on the first image data to obtain a first grayscale image; controlling the image acquisition module to perform filtering processing on the first grayscale image to obtain a first filtered image; and controlling the image acquisition module to perform thresholding processing on the first filtered image to obtain processed first image data.

[0039] Furthermore, by controlling the parameter calibration and orientation adjustment of the image acquisition module, first image data is acquired. The image acquisition module then performs grayscale processing, filtering, and thresholding on the first image data, improving the quality and usability of the image data and enhancing the accuracy and efficiency of identifying the location of the vehicle's fuel tank cap and filler cap. This increases the automation level of the intelligent refueling robot and also enhances its adaptability and robustness in complex environments.

[0040] Optionally, in the method described above, the calculation formula for adjusting the initial path based on the degree-of-freedom adjustment components of the robotic arm in the execution module is as follows:

[0041] In the formula, σ x σ y and σ z These are the adjustment components in the three-dimensional direction, (x) p ,y p ,z p (x′) represents a node on the initial path; p ,y′ p ,z′ p () represents the adjusted node;

[0042] The calculation formula for interpolating the adjusted path is as follows:

[0043] In the formula, f arm The operating frequency of the robotic arm, (x p-1 ,y p-1 ,z p-1 ) is the node to be adjusted (x) p ,y p ,z p The node preceding ) d c For the interpolation distance, N c The number of steps.

[0044] Optionally, in the method described above, the calculation formula for grayscale processing of the first image data is: f grey =0.299R + 0.587G + 0.114B

[0045] In the formula, f grey R represents the grayscale value of the image, and R, G, and B are the red, green, and blue components in the first image data, respectively.

[0046] The formula for calculating the filtering process of the first grayscale image is as follows:

[0047] In the formula, f wave(i,j) are the weight coefficients of each pixel in the first grayscale image, ω(k,l) are the filtering weighting coefficients, and k and l are the filtering components of the first grayscale image in the horizontal and vertical directions.

[0048] The calculation formula for thresholding the first filtered image is as follows:

[0049] In the formula, f threshold (i,j) represents the threshold value for each pixel in the first filtered image, and maxf grey (i,j) represents the maximum gray value of the pixel, and δ is the segmentation threshold.

[0050] The intelligent refueling robot and its control method provided in this application involve a central controller controlling an image acquisition module to obtain processed first image data; a control path planning module obtaining a first planned path based on the processed first image data and the first real-time position data of the execution module; controlling the execution module to sequentially open the vehicle's fuel tank cap and refueling cap according to the first planned path; controlling the image acquisition module to obtain processed second image data; the control path planning module obtaining a second planned path based on the processed second image data and the second real-time position data of the execution module; controlling the execution module to perform the refueling operation according to the second planned path; and controlling the execution module to close the vehicle's fuel tank cap and refueling cap upon detection of refueling completion. This reduces the operational complexity of self-service gas stations, improves applicability, and enhances the user experience. Attached Figure Description

[0051] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0052] Figure 1 is a structural schematic diagram of the intelligent refueling robot provided in an embodiment of this application;

[0053] Figure 2 is a structural front view of the execution module provided in an embodiment of this application;

[0054] Figure 3 is a structural side view of the execution module provided in an embodiment of this application;

[0055] Figure 4 is a flowchart illustrating the control method for an intelligent refueling robot provided in one embodiment of this application.

[0056] Explanation of reference numerals in the attached drawings: 10-Image acquisition module; 20-Path planning module; 30-Execution module; 40-Central controller; 301-Horizontal moving housing; 302-First rotary motor; 303-First threaded rod; 304-First slide bar; 305-Support rod; 306-Longitudinal moving housing; 307-Second rotary motor; 308-Second threaded rod; 309-Second slide bar; 310-Motor mounting plate; 311-Second mounting block; 312-Third rotary motor; 313-Fixing plate; 314-Robotic arm; 315-Refueling assembly; 316-Fuel tank cap and refueling cap opening and closing assembly. Detailed Implementation

[0057] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0058] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0059] Figure 1 is a schematic diagram of the structure of the intelligent refueling robot provided in the embodiment of this application.

[0060] Figure 2 is a structural front view of the execution module provided in an embodiment of this application.

[0061] Figure 3 is a structural side view of the execution module provided in an embodiment of this application.

[0062] Referring to Figures 1 to 3, this application embodiment provides an intelligent refueling robot, including an image acquisition module 10, a path planning module 20, an execution module 30, and a central controller 40. The image acquisition module 10, the path planning module 20, and the execution module 30 are electrically connected to the central controller 40; the execution module 30 is disposed on one side of the refueling machine.

[0063] The execution module 30 includes a lateral movement module, a longitudinal movement module, a rotation module, a fixed plate 313, a robotic arm 314, a refueling component 315, a fuel tank cap, and a refueling cap opening and closing component 316. The lateral movement module is mounted on one side of the fuel dispenser via a support rod 305. The longitudinal movement module is located at the output end of the lateral movement module. The rotation module is located at the output end of the longitudinal movement module. The fixed plate 313 is located at the output end of the rotation module. The robotic arm 314, fuel tank cap, and refueling cap opening and closing component 316 are all mounted on the fixed plate 313. The refueling component 315 is mounted on the robotic arm 314. The lateral movement module, longitudinal movement module, rotation module, robotic arm 314, refueling component 315, fuel tank cap, and refueling cap opening and closing component 316 are electrically connected to the central controller 40.

[0064] Among them, the robotic arm 314 can be a robotic arm that only swings up and down; the refueling component 315 can be an automatic refueling nozzle.

[0065] The fuel tank cap and the fuel filler cap opening and closing assembly 316 can be implemented by a combination of an electric telescopic rod and an electromagnetic chuck; for example, an electric telescopic rod is provided on the fixed plate 313, and an electromagnetic chuck is provided at the output end of the electric telescopic rod, so that the fuel tank cap is opened by attracting the electromagnetic chuck.

[0066] The fuel tank cap and the fuel filler cap opening and closing assembly 316 can be implemented by using an electric telescopic rod, a rotary motor and a gripper. For example, the fuel filler cap is moved to the fuel filler cap position by the electric telescopic rod, the fuel filler cap is clamped by the gripper, and the fuel filler cap is disassembled and installed by rotating the rotary motor clockwise or counterclockwise.

[0067] The central controller 40 is used to: control the image acquisition module 10 to acquire first image data of any vehicle within a preset range of the fuel dispenser; preprocess the first image data to obtain processed first image data; and send the processed first image data to the path planning module 20, wherein the first image data is image data of the vehicle with the fuel tank cap and fuel filler cap not open; control the path planning module 20 to obtain a first planned path based on the processed first image data and the first real-time position data of the execution module 30; control the execution module 30 to sequentially open the fuel tank cap and fuel filler cap by controlling the lateral movement module, longitudinal movement module, rotation module, fuel tank cap and fuel filler cap opening and closing assembly 316 according to the first planned path; and control the image acquisition module 10 to acquire vehicle... The second image data of the vehicle is preprocessed to obtain processed second image data, which is then sent to the path planning module 20. The second image data is an image of the vehicle with the fuel tank cap and refueling cap open. The path planning module 20 obtains a second planned path based on the processed second image data and the second real-time position data of the execution module 30. The execution module 30 controls the robotic arm 314 and the refueling component 315 to perform the refueling operation according to the second planned path. After the refueling is detected to be completed, the execution module 30 controls the lateral movement module, longitudinal movement module, rotation module, fuel tank cap and refueling cap opening and closing component 316 to reverse the first planned path to close the vehicle's fuel tank cap and refueling cap.

[0068] The lateral movement module includes a lateral movement housing 301, a first rotary motor 302, a first threaded rod 303, a first sliding rod 304, and a first mounting block; a support rod 305 is provided on one side of the refueling machine; the lateral movement housing 301 is provided on the top of the support rod 305, and the first rotary motor 302 is provided on one side of the lateral movement housing 301; a rotating component is provided on the other side of the lateral movement housing 301; the output end of the first rotary motor 302 extends into the interior of the lateral movement housing 301; the first threaded rod 303 is provided between the first rotary motor 302 and the rotating component. The first threaded rod 303 is fixedly provided with a first sliding rod 304 on its upper and lower sides respectively. The back of the first mounting block is provided with a threaded hole corresponding to the first threaded rod 303 and a sliding hole corresponding to the first sliding rod 304. The first mounting block is threadedly connected to the first threaded rod 303 through the threaded hole and slidably connected to the first sliding rod 304 through the sliding hole. The front of the first mounting block is provided with a longitudinal moving module. The inner side of the transverse moving housing 301 is provided with a ranging module between the first mounting block and the first mounting block. The first rotary motor 302 and the ranging module are electrically connected to the central controller 40.

[0069] The transversely movable housing 301 and the support rod 305 can be connected by welding or screws.

[0070] The ranging module can be an infrared ranging module.

[0071] The longitudinal movement module includes a longitudinal movement housing 306, a second rotary motor 307, a second threaded rod 308, a second slide rod 309, and a second mounting block 311. The longitudinal movement housing 306 is disposed on the front of the first mounting block. The second rotary motor 307 is disposed on the upper side of the longitudinal movement housing 306. A rotating component is disposed on the lower side of the longitudinal movement housing 306. The output end of the second rotary motor 307 extends into the interior of the longitudinal movement housing 306. The second threaded rod 308 is disposed between the second rotary motor 307 and the rotating component. The second slide rod 309 is fixedly disposed on the left and right sides of the second threaded rod 308, respectively. The back of the second mounting block 311 is provided with a threaded hole and a sliding hole. The second mounting block 311 is threadedly connected to the second threaded rod 308 through the threaded hole and slidably connected to the second slide rod 309 through the sliding hole. The rotation module is disposed on the front of the second mounting block 311. A ranging module is disposed between the upper interior of the longitudinal movement housing 306 and the second mounting block 311. The second rotary motor 307 and the ranging module are electrically connected to the central controller 40.

[0072] The ranging module can be an infrared ranging module.

[0073] The rotating module includes a motor mounting plate 310 and a third rotating motor 312; wherein, the motor mounting plate 310 is located at the center of the front of the second mounting block 311; the third rotating motor 312 is located at the center of the motor mounting plate 310; the fixing plate 313 is located at the output end of the third rotating motor 312; the third rotating motor 312 is electrically connected to the central controller 40.

[0074] As described above, this application uses a central controller to control the image acquisition module to obtain processed first image data; the control path planning module obtains a first planned path based on the processed first image data and the first real-time position data of the execution module; the execution module is then controlled to sequentially open the vehicle's fuel tank cap and refueling cap according to the first planned path; the image acquisition module obtains processed second image data; the control path planning module obtains a second planned path based on the processed second image data and the second real-time position data of the execution module; the execution module is then controlled to perform the refueling operation according to the second planned path; and upon detection that refueling is complete, the execution module closes the vehicle's fuel tank cap and refueling cap. This reduces the operational complexity of self-service gas stations, improves applicability, and enhances the user experience.

[0075] Figure 4 is a flowchart illustrating a control method for an intelligent refueling robot according to an embodiment of this application. The executing entity in this embodiment can be the central controller 40 shown in Figure 1, or other computer devices; this embodiment does not impose any particular limitation on this. As shown in Figure 4, the method includes:

[0076] S401: Control the image acquisition module to acquire first image data of any vehicle within the preset range of the fuel dispenser, preprocess the first image data to obtain processed first image data, and send the processed first image data to the path planning module. The first image data is the image data of the vehicle with the fuel tank cap and fuel filler cap not open.

[0077] Specifically, S401 includes S4011 to S4016:

[0078] S4011: Control the image acquisition module to use a preset calibration template to calibrate the parameters of the acquisition module.

[0079] S4012: Controls the image acquisition module to adjust the direction and angle of the acquisition module according to the vehicle.

[0080] Specifically, the system acquires the vehicle's current position, orientation, and attitude information; and adjusts the direction and angle of the acquisition module based on the vehicle's current position, orientation, and attitude information.

[0081] S4013: Control the image acquisition module to acquire the first image data of the vehicle with the fuel tank cap and filler cap not open.

[0082] The coordinate transformation formula between pixels in the first image data and world coordinate points is:

[0083] In the formula, ψ represents the pixel in the first image data; ψ represents the imaging focal length of the image acquisition module. This is the world coordinate point.

[0084] S4014: Control the image acquisition module to perform grayscale processing on the first image data to obtain the first grayscale image.

[0085] The formula for calculating the grayscale value of the first image data is: f grey =0.299R + 0.587G + 0.114B

[0086] In the formula, f grey Let R be the grayscale value of the image, and let G and B be the red, green, and blue components in the first image data, respectively.

[0087] S4015: Control the image acquisition module to perform filtering processing on the first grayscale image to obtain the first filtered image.

[0088] The formula for calculating the filtering process of the first grayscale image is as follows:

[0089] In the formula, f wave(i,j) are the weight coefficients of each pixel in the first grayscale image, ω(k,l) are the filtering weighting coefficients, and k and l are the filtering components of the first grayscale image in the horizontal and vertical directions.

[0090] S4016: Control the image acquisition module to perform thresholding processing on the first filtered image to obtain the processed first image data.

[0091] The calculation formula for thresholding the first filtered image is as follows:

[0092] In the formula, f threshold (i,j) represents the threshold value for each pixel in the first filtered image, and maxf grey (i,j) represents the maximum gray value of the pixel, and δ is the segmentation threshold.

[0093] S402: The control path planning module obtains the first planned path based on the processed first image data and the first real-time position data of the execution module.

[0094] Specifically, S402 includes S4021 to S4023:

[0095] S4021: The ranging module of the control execution module obtains the first real-time position data of the execution module.

[0096] The ranging module can be an infrared ranging module.

[0097] It should be noted that the position data on the upper side of the fixed plate is used as the position data of the execution module.

[0098] S4022: The control path planning module uses the trained first target recognition model to calculate the position data of the vehicle's fuel tank cap and fuel filler cap in the processed first image data.

[0099] The first target recognition model can be constructed based on a convolutional neural network algorithm.

[0100] The structure of the first target recognition model includes a visual layer and a feature extraction layer. The feature extraction layer includes a convolutional layer, a pooling layer, a fully connected layer, and an activation layer.

[0101] Convolutional layers are used to extract feature information; pooling layers are used to simplify and extract feature information; and fully connected layers are used to integrate the difference data in convolutional or pooling layers.

[0102] The outputs of the convolutional and pooling layers are as follows:

[0103] In the formula, H(m,n) is the convolution kernel, and m and n represent the size components of the convolution kernel, respectively. denoted as the number of pixels in the input convolutional layer, and f(i,j) is the value of the input image at position (i,j).

[0104] The activation function of the activation layer is:

[0105] It should be noted that the training process of the first target recognition model includes: using a preset algorithm to configure the weights and biases of the initial recognition model; inputting images of vehicles with open fuel tank caps and fuel filler caps from the first training dataset into the initial recognition model; processing the images of vehicles with open fuel tank caps and fuel filler caps to obtain quantized contour feature extraction results; locating the target based on the quantized contour feature extraction results to obtain predicted coordinate values; determining the error between the predicted coordinate values ​​and the true coordinate values ​​in the training set as the objective function; updating the model parameters according to the objective function and the preset optimization algorithm to obtain the trained model.

[0106] The first training dataset includes images of vehicles with the fuel tank cap and fuel filler cap undone, as well as the actual coordinates of the fuel tank cap and fuel filler cap.

[0107] The algorithm formula for configuring the weights and biases of the initial recognition model is as follows:

[0108] In the formula, εconvolution represents the weights of the initial recognition model, and ω k As the bias of the initial recognition model, R k For the output of layer k, n layer ω0 represents the number of layers in the initial recognition model, μ represents the training margin of the neurons, ω0 represents the initial weights, and f(i,j) represents the values ​​of the images of the vehicle with the fuel tank cap and fuel filler cap undone at position (i,j).

[0109] The formula for the quantized contour feature extraction result is expressed as:

[0110] In the formula, R x R represents the horizontal convolution output of a convolutional neural network. y This refers to the convolution output in the vertical direction of the convolutional neural network. θ(x,y) represents the gradient magnitude of the images of the vehicle with the fuel tank cap and fuel filler cap open, and θ(x,y) represents the orientation angle of the images of the vehicle with the fuel tank cap and fuel filler cap open.

[0111] The formula for localization based on the quantified contour feature extraction results is as follows:

[0112] In the formula, x targetLet y be the horizontal coordinates of the fuel tank cap and the filler cap. target The coordinates of the fuel tank cap and filler cap in the vertical direction are given. θ(x,y) represents the gradient magnitude of the images of the vehicle with the fuel tank cap and fuel filler cap open, and θ(x,y) represents the orientation angle of the images of the vehicle with the fuel tank cap and fuel filler cap open.

[0113] It should be noted that the coordinates of the fuel tank cap and filler cap in the depth direction can be obtained by using the coordinate transformation formula.

[0114] The preset optimization algorithm can be the gradient descent algorithm.

[0115] S4023: The control path planning module uses a preset path planning algorithm to calculate the first planned path from the first real-time location data to the location data of the fuel tank cap and the filler cap.

[0116] S403: According to the lateral movement module, longitudinal movement module, rotation module, fuel tank cap and refueling cap opening and closing assembly of the first planning path control execution module, the fuel tank cap and refueling cap of the vehicle are opened in sequence.

[0117] For example, according to the first planned path control execution module, the third rotary motor of the rotary module rotates, driving the fixed plate to rotate, which in turn drives the fuel tank cap and refueling cap opening and closing assembly or the robotic arm to the upper side of the fixed plate. First, the horizontal movement module and the vertical movement module are controlled to move, so that the vehicle fuel tank cap and refueling cap coincide with the execution module on the x-axis and z-axis. The electric telescopic rod of the fuel tank cap and refueling cap opening and closing assembly is activated, and contacts the fuel tank cap to open it. After the fuel tank cap is opened, the third rotary motor rotates, driving the fuel tank cap and refueling cap opening and closing assembly to return to the upper side of the fixed plate. The electric telescopic rod of the fuel tank cap and refueling cap opening and closing assembly drives the gripper to clamp the refueling cap. The refueling cap is then removed by rotating the rotary motor.

[0118] S404: Control the image acquisition module to acquire the second image data of the vehicle, preprocess the second image data to obtain the processed second image data, and send the processed second image data to the path planning module. The second image data is the image data of the vehicle with the fuel tank cap and filler cap open.

[0119] Specifically, the image acquisition module is controlled to use a preset calibration template to calibrate its parameters; the image acquisition module is controlled to adjust its direction and angle according to the vehicle; the image acquisition module is controlled to acquire second image data of the vehicle with the fuel tank cap and filler cap open; the image acquisition module is controlled to perform grayscale processing on the second image data to obtain a second grayscale image; the image acquisition module is controlled to perform filtering processing on the second grayscale image to obtain a second filtered image; the image acquisition module is controlled to perform thresholding processing on the second filtered image to obtain the processed second image data.

[0120] S405: The control path planning module obtains the second planned path based on the processed second image data and the second real-time position data of the execution module.

[0121] Specifically, the ranging module of the control execution module acquires the second real-time position data of the execution module; the control path planning module uses a trained second target recognition model to calculate the position data of the vehicle's fuel tank cap and fuel filler cap in the processed second image data; the control path planning module uses a preset path planning algorithm to calculate the second planned path from the second real-time position data to the position data of the fuel tank cap and fuel filler cap.

[0122] S406: The robotic arm and refueling components of the control execution module perform the refueling operation according to the second planned path.

[0123] It should be noted that before acquiring the second image data, the robotic arm and refueling components need to be rotated to the upper side of the fixed plate by the third rotary motor.

[0124] It should be noted that the fuel dispenser should be equipped with a QR code. Users can select the corresponding fuel type and amount based on the QR code, and the central controller will dispense fuel according to the user's selection. Users can then pay using the QR code.

[0125] S407: After refueling is detected, the lateral movement module, longitudinal movement module, rotation module, fuel tank cap opening and closing assembly, and refueling cap opening and closing assembly of the control execution module reverse the first planned path to close the vehicle's fuel tank cap and refueling cap.

[0126] As described above, this application uses a central controller to control the image acquisition module to obtain processed first image data; the control path planning module obtains a first planned path based on the processed first image data and the first real-time position data of the execution module; the execution module is then controlled to sequentially open the vehicle's fuel tank cap and refueling cap according to the first planned path; the image acquisition module obtains processed second image data; the control path planning module obtains a second planned path based on the processed second image data and the second real-time position data of the execution module; the execution module is then controlled to perform the refueling operation according to the second planned path; and upon detection that refueling is complete, the execution module closes the vehicle's fuel tank cap and refueling cap. This reduces the operational complexity of self-service gas stations, improves applicability, and enhances the user experience.

[0127] In one embodiment of this application, based on the above embodiments, step S405 is further provided in another way, as detailed below:

[0128] S4051: The ranging module of the control execution module acquires the second real-time position data of the execution module.

[0129] The ranging module can be an infrared ranging module.

[0130] S4052: The control path planning module uses a trained second target recognition model to calculate the location data of the vehicle's refueling port in the processed second image data.

[0131] The second target recognition model has the same structure as the first target recognition model.

[0132] It should be noted that the training process of the second target recognition model includes: using a preset algorithm to configure the weights and biases of the initial recognition model; inputting images of vehicles with open fuel tank caps and fuel filler caps from the second training dataset into the initial recognition model; processing the images of vehicles with open fuel tank caps and fuel filler caps to obtain quantified contour feature extraction results; locating the target based on the quantified contour feature extraction results to obtain predicted coordinate values; determining the error between the predicted coordinate values ​​and the true coordinate values ​​in the training set as the objective function; updating the model parameters according to the objective function and the preset optimization algorithm to obtain the trained model.

[0133] The second training dataset includes images of vehicles with the fuel tank cap and fuel filler cap open, as well as the actual coordinates of the fuel tank cap and fuel filler cap.

[0134] S4053: The control path planning module uses a preset path planning algorithm to calculate the initial path from the second real-time location data to the location data of the refueling port.

[0135] S4054: The control path planning module adjusts the initial path based on the degree of freedom adjustment components of the robotic arm in the execution module to obtain the adjusted path.

[0136] The formula for adjusting the initial path based on the degree-of-freedom adjustment components of the robotic arm in the execution module is as follows:

[0137] In the formula, σ x σ y and σ z These are the adjustment components in the three-dimensional direction, (x) p ,y p ,z p (x′) represents a node on the initial path; p ,y′ p ,z′ p ) represents the adjusted node.

[0138] S4055: The control path planning module performs interpolation processing on the adjusted path to obtain the second planned path.

[0139] The calculation formula for interpolating the adjusted path is as follows:

[0140] In the formula, f arm The operating frequency of the robotic arm, (x p-1 ,y p-1 ,z p-1 ) is the node to be adjusted (x) p ,y p ,z p The node preceding ) d c For the interpolation distance, N c The number of steps.

[0141] As described above, in this application, the central controller uses a preset path planning algorithm through the control path planning module to calculate the initial path from the second real-time location data to the refueling nozzle location data; the control path planning module adjusts the initial path according to the degree of freedom adjustment components of the robotic arm of the execution module to obtain an adjusted path; the control path planning module performs interpolation processing on the adjusted path to obtain a second planned path. This improves the feasibility of the planned path and further enhances the automation level of the intelligent refueling robot.

[0142] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0143] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0144] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0145] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0146] When an integrated unit / module is implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field-Programmable Gate Array), DSP (Digital Signal Processor), and ASIC (Application-Specific Integrated Circuit), etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as RRAM (Resistive Random Access Memory), DRAM (Dynamic Random Access Memory), SRAM (Static Random-Access Memory), EDRAM (Enhanced Dynamic Random Access Memory), HBM (High-Bandwidth Memory), HMC (Hybrid Memory Cube), etc.

[0147] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, ROM (Read-Only Memory), RAM (Random Access Memory), portable hard drives, magnetic disks, or optical disks.

[0148] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0149] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0150] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An intelligent refueling robot, characterized in that, include: The system includes an image acquisition module, a path planning module, an execution module, and a central controller; the image acquisition module, path planning module, and execution module are electrically connected to the central controller; the execution module is located on one side of the fuel dispenser. The execution module includes a lateral movement module, a longitudinal movement module, a rotation module, a fixed plate, a robotic arm, a refueling assembly, a fuel tank cap, and a refueling cap opening and closing assembly. The lateral movement module is mounted on one side of the fuel dispenser via a support rod. The longitudinal movement module is located at the output end of the lateral movement module. The rotation module is located at the output end of the longitudinal movement module. The fixed plate is located at the output end of the rotation module. The robotic arm, fuel tank cap, and refueling cap opening and closing assembly are all mounted on the fixed plate. The refueling assembly is mounted on the robotic arm. The lateral movement module, longitudinal movement module, rotation module, robotic arm, refueling assembly, fuel tank cap, and refueling cap opening and closing assembly are electrically connected to the central controller. The central controller is used for: The image acquisition module is controlled to acquire first image data of any vehicle within a preset range of the fuel dispenser, the first image data is preprocessed to obtain processed first image data, and the processed first image data is sent to the path planning module, wherein the first image data is image data of the vehicle with the fuel tank cap and fuel filler cap not opened. The path planning module is controlled to obtain a first planned path based on the processed first image data and the first real-time position data of the execution module; According to the first planned path, the execution module controls the lateral movement module, the longitudinal movement module, the rotation module, the fuel tank cap opening and closing assembly, and the refueling cap opening and closing assembly to sequentially open the vehicle's fuel tank cap and refueling cap. The system controls the image acquisition module to acquire second image data of the vehicle, preprocesses the second image data to obtain processed second image data, and sends the processed second image data to the path planning module, wherein the second image data is image data of the vehicle with the fuel tank cap and filler cap open; The path planning module is controlled to obtain a second planned path based on the processed second image data and the second real-time position data of the execution module; The robotic arm and refueling assembly of the execution module are controlled to perform refueling operations according to the second planned path; Upon detection that refueling is complete, the lateral movement module, longitudinal movement module, rotation module, fuel tank cap opening and closing assembly, and refueling cap opening and closing assembly of the execution module are controlled to reverse the first planned path to close the vehicle's fuel tank cap and refueling cap.

2. The intelligent refueling robot according to claim 1, characterized in that, The lateral movement module includes a lateral movement housing, a first rotary motor, a first threaded rod, a first slide rod, and a first mounting block; The support rod is provided on one side of the fuel dispenser; the lateral moving housing is provided on the top of the support rod, and the first rotary motor is provided on one side of the lateral moving housing; a rotating component is provided on the other side of the lateral moving housing; the output end of the first rotary motor extends into the interior of the lateral moving housing; a first threaded rod is provided between the first rotary motor and the rotating component; the first sliding rod is fixedly provided on the upper and lower sides of the first threaded rod, and a threaded hole is provided on the back of the first mounting block corresponding to the first threaded rod, and a sliding hole is provided corresponding to the first sliding rod. The first mounting block is threadedly connected to the first threaded rod through the threaded hole and slidably connected to the first sliding rod through the sliding hole. The longitudinal moving module is provided on the front of the first mounting block, and a ranging module is provided between the interior side of the lateral moving housing and the first mounting block; the first rotary motor and the ranging module are electrically connected to the central controller.

3. The intelligent refueling robot according to claim 1 or 2, characterized in that, The longitudinal movement module includes a longitudinal movement housing, a second rotary motor, a second threaded rod, a second slide bar, and a second mounting block; The longitudinally moving housing is disposed on the front side of the first mounting block; the second rotary motor is disposed on the upper side of the longitudinally moving housing; a rotating component is disposed on the lower side of the longitudinally moving housing; the output end of the second rotary motor extends into the interior of the longitudinally moving housing; a second threaded rod is disposed between the second rotary motor and the rotating component; second sliding rods are respectively fixedly disposed on the left and right sides of the second threaded rod; a threaded hole and a sliding hole are disposed on the back of the second mounting block; the second mounting block is threadedly connected to the second threaded rod through the threaded hole and slidably connected to the second sliding rod through the sliding hole; the rotating module is disposed on the front side of the second mounting block; a ranging module is disposed between the upper interior side of the longitudinally moving housing and the second mounting block; the second rotary motor and the ranging module are electrically connected to the central controller.

4. The intelligent refueling robot according to any one of claims 1 to 3, characterized in that, The rotating module includes a motor mounting plate and a third rotating motor; wherein, the motor mounting plate is located at the center of the front of the second mounting block; the third rotating motor is located at the center of the motor mounting plate; the fixing plate is located at the output end of the third rotating motor; the third rotating motor is electrically connected to the central controller.

5. A control method for an intelligent refueling robot, characterized in that, The central controller applied in the intelligent refueling robot according to any one of claims 1 to 4 comprises: The image acquisition module acquires first image data of any vehicle within a preset range of the fuel dispenser, preprocesses the first image data to obtain processed first image data, and sends the processed first image data to the path planning module. The first image data is the image data of the vehicle with the fuel tank cap and fuel filler cap not open. The path planning module is controlled to obtain a first planned path based on the processed first image data and the first real-time position data of the execution module; According to the first planned path, the execution module controls the lateral movement module, longitudinal movement module, rotation module, fuel tank cap and refueling cap opening and closing assembly to sequentially open the vehicle's fuel tank cap and refueling cap. The system controls the image acquisition module to acquire second image data of the vehicle, preprocesses the second image data to obtain processed second image data, and sends the processed second image data to the path planning module, wherein the second image data is image data of the vehicle with the fuel tank cap and filler cap open; The path planning module is controlled to obtain a second planned path based on the processed second image data and the second real-time position data of the execution module; The robotic arm and refueling components of the execution module are controlled to perform refueling operations according to the second planned path; Upon detection that refueling is complete, the lateral movement module, longitudinal movement module, rotation module, fuel tank cap opening and closing assembly, and refueling cap opening and closing assembly of the execution module are controlled to reverse the first planned path to close the vehicle's fuel tank cap and refueling cap.

6. The method according to claim 5, characterized in that, The path planning module, based on the processed first image data and the first real-time position data from the execution module, obtains a first planned path, including: The ranging module of the control execution module acquires the first real-time position data of the execution module; The path planning module is controlled to use a trained first target recognition model to calculate the position data of the vehicle's fuel tank cap and fuel filler cap in the processed first image data; The path planning module is controlled to use a preset path planning algorithm to calculate the first planned path from the first real-time location data to the location data of the fuel tank cap and the filler cap.

7. The method according to claim 5 or 6, characterized in that, The path planning module, based on the processed second image data and the second real-time position data from the execution module, obtains a second planned path, including: The ranging module of the control execution module acquires the second real-time position data of the execution module; The path planning module is controlled to use a trained second target recognition model to calculate the location data of the vehicle's fuel filler in the processed second image data; The path planning module is controlled to use a preset path planning algorithm to calculate the initial path from the second real-time location data to the location data of the refueling nozzle; The path planning module adjusts the initial path based on the degree-of-freedom adjustment components of the robotic arm in the execution module to obtain an adjusted path; The path planning module is controlled to perform interpolation processing on the adjusted path to obtain a second planned path.

8. The method according to any one of claims 5 to 7, characterized in that, The control image acquisition module acquires first image data of any vehicle within a preset range of the fuel dispenser, and preprocesses the first image data to obtain processed first image data, including: The image acquisition module is controlled using a preset calibration template to calibrate its parameters. The image acquisition module is controlled to adjust its direction and angle according to the vehicle. The image acquisition module is controlled to acquire first image data of the vehicle with the fuel tank cap and filler cap not open; The image acquisition module is controlled to perform grayscale processing on the first image data to obtain a first grayscale image; The image acquisition module is controlled to perform filtering processing on the first grayscale image to obtain a first filtered image; The image acquisition module is controlled to perform thresholding processing on the first filtered image to obtain the processed first image data.

9. The method according to claim 7, characterized in that, The calculation formula for adjusting the initial path based on the degree-of-freedom adjustment components of the robotic arm in the execution module is as follows: In the formula, σ x σ y and σ z These are the adjustment components in the three-dimensional direction, (x) p ,y p ,z p (x′) represents a node on the initial path; p ,y′ p ,z′ p () represents the adjusted node; The calculation formula for interpolating the adjusted path is as follows: In the formula, f arm The operating frequency of the robotic arm, (x p-1 ,y p-1 ,z p-1 ) is the node to be adjusted (x) p ,y p ,z p The node preceding ) d c For the interpolation distance, N c The number of steps.

10. The method according to claim 8, characterized in that, The formula for calculating the grayscale processing of the first image data is: f grey =0.299R + 0.587G + 0.114B In the formula, f grey R represents the grayscale value of the image, and R, G, and B are the red, green, and blue components in the first image data, respectively. The formula for calculating the filtering process of the first grayscale image is as follows: In the formula, f wave (i,j) are the weight coefficients of each pixel in the first grayscale image, ω(k,l) are the filtering weighting coefficients, and k and l are the filtering components of the first grayscale image in the horizontal and vertical directions. The calculation formula for thresholding the first filtered image is as follows: In the formula, f threshold (i,j) represents the threshold value for each pixel in the first filtered image, and max f grey (i,j) represents the maximum gray value of the pixel, and δ is the segmentation threshold.

Images