A method, system, device and medium for joint docking

Abstract

The present invention discloses a method, system, device and medium for joint docking, which relates to the field of automatic joint docking; the movement trajectory of the actuator is determined according to the obtained position information of the actuator and the points of the fuel tank joint; the fuel pipe joint is moved above the fuel tank joint according to the movement trajectory; a circular detection algorithm is used to extract a set area in the joint image of the fuel tank joint captured by the camera module and determine it as the region of interest; the execution data is determined according to the depth parameter built in the camera module, as well as the two-dimensional coordinates of the center of the region of interest and the coordinates of the card slot notch obtained by detecting and calculating the region of interest by using computer vision methods; after obtaining the target conversion coordinates of the actuator according to the conversion matrix, the execution route of the actuator is determined, and then the fuel tank joint and the fuel pipe joint are rotationally docked according to the target rotation angle; the present invention can be applied to transport vehicles and improve the joint docking accuracy.

Description

Technical Field

[0001] The present invention relates to the field of automatic docking of connectors, and particularly to a method, system, device and medium for connector docking. Background Art

[0002] Traditional refueling modes expose operators to toxic and flammable gases, and in addition, refueling operations also face a series of problems such as harsh environments. In the field of automatic refueling of transportation equipment, only the United States has conducted certain research and has a technological monopoly globally.

[0003] To improve the efficiency of automatic refueling of transportation vehicles, a special self-sealing refueling connector is usually used at the fuel tank opening of transportation vehicles. The female end of this connector is installed on the fuel tank, and the male end is installed on the end mechanism of the robotic arm. In addition, the whole body of this refueling connector is made of metal, with a circular flat surface of metal material and weak texture information. Due to the special mechanical structure of the refueling connector, the positioning error is required to be no more than 6 mm. However, the special design of the refueling connector makes it difficult to identify the refueling interface, making it difficult to accurately identify the fuel tank connector and difficult to achieve the docking of the fuel tank and the robotic arm, which restricts the development of automatic refueling of transportation equipment.

[0004] Most of the existing technologies use visual recognition for the fuel tanks of general vehicles, and there is no visual recognition algorithm and automatic docking system for the self-sealing refueling connectors of transportation vehicles. Moreover, the existing algorithms generally have low positioning accuracy and it is difficult to achieve accurate positioning of the refueling interface. Summary of the Invention

[0005] The purpose of the present invention is to provide a method, system, device and medium for connector docking, which are applied to transportation vehicles and improve the connector docking accuracy.

[0006] To achieve the above purpose, the present invention provides the following solutions:

[0007] A method for connector docking, which is used for a docking device of a transportation vehicle. The docking device includes: a fuel tank connector, a fuel pipe connector, an actuator and a camera module; the camera module is arranged on the actuator; the fuel tank connector is arranged on the fuel tank of the transportation vehicle through a flange; two slot notches are symmetrically arranged on the flange; the actuator is used to clamp the fuel pipe connector.

[0008] The method includes:

[0009] Obtain the position information of the actuator and the position of the fuel tank connector;

[0010] Determine the movement trajectory of the actuator according to the position information and the position.

[0011] Control the actuator to move along the action trajectory so that the tubing joint is located above the fuel tank joint, and control the imaging module to capture the fuel tank joint to obtain a joint image;

[0012] Use a circle detection algorithm to extract a set area in the joint image and determine the area as the region of interest;

[0013] Use computer vision methods to detect and calculate the region of interest to obtain the two-dimensional coordinates of the center of the region of interest and the coordinates of the card slot notch in the region of interest;

[0014] Determine execution data based on the two-dimensional coordinates, the coordinates of the card slot notch, and the depth parameter built into the imaging module; the execution data includes: the target rotation angle and the three-dimensional spatial coordinates of the center of the region of interest; the target rotation angle is the rotation angle of the actuator relative to the card slot notch;

[0015] Convert the three-dimensional spatial coordinates in the execution data to the coordinate system where the actuator is located according to the transformation matrix to obtain the target transformation coordinates of the actuator, and determine the execution route of the actuator according to the target transformation coordinates; the transformation matrix represents the transformation relationship between the coordinate system where the fuel tank joint is located and the coordinate system where the actuator is located;

[0016] Control the actuator to move according to the execution route so that the tubing joint contacts the fuel tank joint, and control the tubing joint to rotate according to the target rotation angle to achieve the docking of the fuel tank joint and the tubing joint.

[0017] Optionally, using computer vision methods to detect and calculate the region of interest to obtain the two-dimensional coordinates of the center of the region of interest and the coordinates of the card slot notch in the region of interest specifically includes:

[0018] Determine the center coordinates of the region of interest;

[0019] Use the corner detection method to identify the corners in the region of interest to obtain four corners;

[0020] Calculate the Euclidean distance between every two of the four corners, and determine the coordinates of each card slot notch according to the calculated Euclidean distance and the set Euclidean distance; one end of the card slot notch corresponds to one corner; each card slot notch corresponds to two corners;

[0021] Determine two intersecting line segments; the four endpoints of the two intersecting line segments correspond one by one to the four corners;

[0022] Determine the two-dimensional coordinates of the center of the region of interest based on the coordinates of the center of the circle and the coordinates of the intersection point of two intersecting line segments.

[0023] Optionally, a circle detection algorithm is used to extract a set area in the joint image and determine the area as the region of interest, specifically including:

[0024] Perform smoothing filtering on the joint image to obtain a processed joint image;

[0025] Convert the processed joint image to grayscale to obtain a joint grayscale image;

[0026] Use a circle detection algorithm to identify a set area in the joint grayscale image;

[0027] Determine the area as the region of interest.

[0028] Optionally, based on the two-dimensional coordinates, the coordinates of the slot notch, and the depth parameter built into the camera module, execution data is determined, specifically including:

[0029] Determine the three-dimensional spatial coordinates of the center of the region of interest based on the two-dimensional coordinates and the depth parameter built into the camera module;

[0030] Determine the target rotation angle based on the two-dimensional coordinates and the coordinates of the slot notch.

[0031] A system for joint docking, the system includes:

[0032] An acquisition module for acquiring the position information of the actuator and the position of the fuel tank joint;

[0033] An action trajectory determination module for determining the action trajectory of the actuator according to the position information and the position;

[0034] A joint image acquisition module for controlling the actuator to move along the action trajectory so that the tubing joint is above the fuel tank joint, and controlling the camera module to take a picture of the fuel tank joint to obtain a joint image;

[0035] An area of interest determination module for using a circle detection algorithm to extract a set area in the joint image and determining the area as the area of interest;

[0036] A calculation module for detecting and calculating the area of interest using computer vision methods to obtain the two-dimensional coordinates of the center of the area of interest and the coordinates of the slot notch in the area of interest;

[0037] An execution data determination module, configured to determine execution data according to the two-dimensional coordinates, the coordinates of the card slot notch, and the depth parameter built in the camera module; the execution data includes: a target rotation angle and the three-dimensional spatial coordinates of the center of the region of interest; the target rotation angle is the rotation angle of the execution mechanism relative to the card slot notch;

[0038] An execution route determination module, configured to convert the three-dimensional spatial coordinates in the execution data to the coordinate system where the execution mechanism is located according to a conversion matrix, obtain the target conversion coordinates of the execution mechanism, and determine the execution route of the execution mechanism according to the target conversion coordinates; the conversion matrix represents the conversion relationship between the three-dimensional spatial coordinates and the coordinates of the execution mechanism;

[0039] A docking module, configured to control the execution mechanism to move according to the execution route, so that the oil pipe joint contacts the fuel tank joint, and control the oil pipe joint to rotate according to the target rotation angle, so as to realize the docking of the fuel tank joint and the oil pipe joint.

[0040] Optionally, the calculation module specifically includes:

[0041] A center coordinate determination sub-module, configured to determine the center coordinates of the region of interest;

[0042] A corner point determination sub-module, configured to identify the corner points in the region of interest by using the corner point detection method to obtain four corner points;

[0043] A card slot notch coordinate determination sub-module, configured to calculate the Euclidean distance between every two of the four corner points, and determine the coordinates of each card slot notch according to the calculated Euclidean distance and a set Euclidean distance; one end of the card slot notch corresponds to one corner point; each card slot notch corresponds to two corner points;

[0044] A line segment determination sub-module, configured to determine two intersecting line segments; the four endpoints of the two intersecting line segments correspond one by one to the four corner points;

[0045] A two-dimensional coordinate determination sub-module, configured to determine the two-dimensional coordinates of the center of the region of interest according to the center coordinates and the intersection coordinates of the two intersecting line segments.

[0046] Optionally, the region of interest determination module specifically includes:

[0047] A processing sub-module, configured to perform smoothing filtering processing on the joint image to obtain a processed joint image;

[0048] A conversion sub-module, configured to perform gray conversion on the processed joint image to obtain a joint gray image;

[0049] A setting area determination sub-module, configured to identify a setting area in the joint grayscale image by using a circle detection algorithm;

[0050] A region of interest determination sub-module, configured to determine the region as the region of interest.

[0051] Optionally, the execution data determination module specifically includes:

[0052] A three-dimensional space coordinate determination sub-module, configured to determine the three-dimensional space coordinates of the center of the region of interest according to the two-dimensional coordinates and the depth parameter built in the camera module;

[0053] A target rotation angle determination sub-module, configured to determine the target rotation angle according to the two-dimensional coordinates and the coordinates of the card slot notch.

[0054] An electronic device includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned joint docking method.

[0055] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned joint docking method is implemented.

[0056] According to the specific embodiments provided by the present invention, the following technical effects are disclosed by the present invention:

[0057] The present invention provides a method, a system, a device and a medium for joint docking. The method is applied to a transfer vehicle. By obtaining the position information of the actuator and the point position of the fuel tank joint, the approximate position of the fuel tank joint is obtained. Then, through the captured joint image, a circle detection algorithm is used to determine the region of interest, and a computer vision method is used to determine the execution data to perform precise coordinate determination on the fuel tank joint; according to the transformation matrix and the execution data, the execution route of the actuator is determined, and then according to the target rotation angle, the docking of the fuel tank joint and the fuel pipe joint is realized, improving the joint docking accuracy. Description of the Drawings

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings.

[0059] Figure 1 It is a flowchart of the joint docking method provided by the embodiment of the present invention;

[0060] Figure 2Structural diagram of the joint docking system provided by the embodiment of the present invention;

[0061] Figure 3 Front view of the fuel tank joint provided by the embodiment of the present invention;

[0062] Figure 4 Top view of the fuel tank joint provided by the embodiment of the present invention;

[0063] Figure 5 Isometric view of the fuel tank joint provided by the embodiment of the present invention;

[0064] Figure 6 Flowchart of the method for joint docking provided by the embodiment of the present invention in practical application;

[0065] Figure 7 Schematic diagram of the boundary point provided by the embodiment of the present invention;

[0066] Figure 8 Schematic diagram of the corner point provided by the embodiment of the present invention;

[0067] Figure 9 Schematic diagram of the three-dimensional space coordinates of the center of the region of interest provided by the embodiment of the present invention.

[0068] Symbol description:

[0069] Acquisition module - 1, Action trajectory determination module - 2, Joint image acquisition module - 3, Region of interest determination module - 4, Calculation module - 5, Execution data determination module - 6, Execution route determination module - 7, Docking module - 8. Detailed implementation manners

[0070] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0071] The purpose of the present invention is to provide a method, system, device and medium for joint docking, which can improve the joint docking accuracy.

[0072] To make the above objects, features and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0073] Embodiment 1

[0074] An embodiment of the present invention provides a method for butt - jointing connectors, which is used for the docking device of a transport vehicle. The docking device includes: a fuel tank connector, a fuel pipe connector, an actuator, and a camera module; the camera module is arranged on the actuator; the fuel tank connector is arranged on the fuel tank of the target transport vehicle through a flange; two slot notches are symmetrically arranged on the flange; the actuator is used to clamp the fuel pipe connector. The fuel tank connector in the embodiment of the present invention can be directly installed at the fuel tank opening of the transport vehicle. There is an external thread around the tail of the fuel tank connector, which can be directly screwed onto the fuel tank opening for fixation. In addition, the shape of the fuel tank connector is circular, the whole body is made of metal, there are two slots passing through the center of the circle, the width of the slots is 6 mm, the slots rotate 135°, a circular groove is arranged at the bottom of the slots to prevent the connector from falling off during docking, and the two slots are centrosymmetrically distributed. The schematic diagram of the slot structure is as Figures 3 - 5 shown.

[0075] As Figure 1 shown, the method for butt - jointing connectors provided by the embodiment of the present invention includes:

[0076] Step 100: Obtain the position information of the actuator and the position of the fuel tank connector.

[0077] Step 200: Determine the movement trajectory of the actuator according to the position information and the position.

[0078] Step 300: Control the actuator to move along the movement trajectory so that the fuel pipe connector is located above the fuel tank connector, and control the camera module to take a picture of the fuel tank connector to obtain a connector image.

[0079] The specific operations of the above steps in practical applications can be as follows:

[0080] First, use the Zhang Zhengyou calibration method to obtain the internal parameters of the camera. Place the fuel tank within the field of view of the camera module such as a camera or a webcam, take at least one fuel tank photo for the first time; perform gray - scale processing on the taken fuel tank photos to obtain a gray - scale image; extract the contour information of the picture, and based on the geometric contours and geometric features of the fuel tank and the fueling connector, preliminarily identify the fuel tank and the fueling connector to determine the approximate position of the fueling connector.

[0081] According to the results of the preliminary identification, plan the movement position of the actuator, that is, the robotic arm. The robotic arm moves towards the fuel tank opening along the planned trajectory and stops moving when it reaches directly above the fuel tank opening. At this time, the camera plane is approximately parallel to the fueling interface plane, and the fuel tank connector fueling opening plane is approximately at the center position of the camera.

[0082] Step 400: Use a circle detection algorithm to extract a set area in the joint image and determine this area as the region of interest (ROI).

[0083] In practical applications, the fuel tank and the fuel filling joint are connected by a metal flange. The end of the robotic arm for fuel filling is fixed to the male end of the quick connector through a special clamping device. There is a camera and a ring light source above the gripper. The light source is turned on before taking pictures. After the light source is turned on, the camera makes an accurate identification. The ROI is determined by identifying the circular feature of the metal flange in the picture for auxiliary identification: First, preprocess the collected picture. Use the bilateral filtering method to perform edge smoothing filtering on the captured picture, neutralize the colors with similar color distributions, classify the parts with obvious color features, and retain the general features in the picture. Then convert the obtained picture into a grayscale picture as the input picture for circle detection. Use the Hough circle detection algorithm to identify the circular features in the picture, set the maximum and minimum radii for detecting the existence of circles, detect the center coordinates of the flange in the picture and the radius of the flange, and set the area where the flange is located as the ROI, that is, the region of interest, realizing the preliminary positioning of the ROI.

[0084] Among them, step 400 specifically includes:[

[0085] Perform smoothing filtering on the joint image to obtain a processed joint image.

[0086] Convert the processed joint image to grayscale to obtain a joint grayscale image.

[0087] Use a circle detection algorithm to identify the set area in the joint grayscale image.

[0088] Determine this area as the region of interest.

[0089] Step 500: Use computer vision methods to detect and calculate the region of interest to obtain the two-dimensional coordinates of the center of the region of interest and the coordinates of the card slot notch in the region of interest.

[0090] Specifically, step 500 specifically includes:[

[0091] Determine the center coordinates of the region of interest.

[0092] Use the corner detection method to identify the corners in the region of interest to obtain four corners.

[0093] Calculate the Euclidean distance between every two of the four corners. According to the calculated Euclidean distance and the set Euclidean distance, determine the coordinates of each card slot notch; one end of the card slot notch corresponds to one corner; each card slot notch corresponds to two corners.

[0094] Determine two intersecting line segments; the four endpoints of the two intersecting line segments correspond one-to-one with the four corner points.

[0095] Determine the two-dimensional coordinates of the center of the region of interest according to the center coordinates of the circle and the intersection coordinates of the two intersecting line segments.

[0096] Step 600: Determine execution data according to the two-dimensional coordinates, the coordinates of the card slot notch, and the depth parameter built in the imaging module; the execution data includes: the target rotation angle and the three-dimensional spatial coordinates of the center of the region of interest; the target rotation angle is the rotation angle of the actuator relative to the card slot notch.

[0097] Specifically, use the picture of the ROI area that has been obtained as the input picture for corner detection, convert the image to a grayscale image, and perform circle detection on the obtained image data. The circle detection method used is a fast circle detection algorithm based on an ellipse to improve the image processing speed during circle detection. First, perform a perfect circle detection, set the maximum and minimum radii for detecting the existence of a circle. If the detection is successful, directly measure the center coordinates of the circle; if the detection fails, perform an ellipse detection. The situation where an ellipse appears means that when the target area of the circle is projected onto the camera under non-frontal conditions, an ellipse will appear. The region of interest is a circle when the imaging module, i.e., the camera, is facing directly. However, if there is a deviation and the camera is not facing directly, the photographed result will be an ellipse. Therefore, in practical applications, it is very likely to first capture an ellipse.

[0098] When an ellipse appears, in the obtained image data, the boundary points of the longest diameter and the shortest diameter of the ellipse, namely the left boundary point (leftEdgePoint), the right boundary point (rightEdgePoint), the upper boundary point (upEdgePoint), and the lower boundary point (downEdgePoint), can be directly obtained. The four points are as Figure 7 shown. In the image coordinate system, Xo = (leftEdgePoint + rightEdgePoint) / 2, Yo = (upEdgePoint + downEdgePoint) / 2, and the center coordinates are obtained by calculating the longest diameter and the shortest diameter of the ellipse; use the Shi-Tomasi corner detection method to detect the corner points of the connector card slot. There are two card slots for the connector. Set the maximum number that can be detected to 4, then set the minimum quality factor for accepting image corner points and the minimum Euclidean distance between two corner points, and set the number of iterations to obtain the coordinate information of the card slot points and complete the detection of the card slot corner points.

[0099] The maximum number of detected corner points is set to 4 to obtain the corner coordinate values of the card slot notch. The detected 4 corner coordinates are divided into two groups according to the distance from their respective coordinate points. The two corner points with the closest distance are the two corner points of the same card slot, denoted as A1, A2, B1, B2, as Figure 8 shown. Connect the two corner points with the farthest distance between different card slots as Figure 9 shown, find the intersection coordinate of the two straight lines. The obtained intersection coordinate is the center coordinate of the joint plane. The average value of the obtained center coordinate and the previously obtained center coordinate is calculated to obtain the accurate center coordinate of the circle. Using the SDK built in the camera, the RGB image and the depth image are aligned to obtain the depth value of the fuel filling joint center in the depth map. Combining the internal parameters calibrated by the camera before, the spatial point coordinate of the center of the circle is obtained, that is, the three-dimensional spatial coordinate of the center of the region of interest.

[0100] Specifically, step 600 specifically includes:

[0101] Determine the three-dimensional spatial coordinate of the center of the region of interest according to the two-dimensional coordinate and the depth parameter built in the imaging module.

[0102] Determine the target rotation angle according to the two-dimensional coordinate and the coordinate of the card slot notch.

[0103] Step 700: Convert the three-dimensional spatial coordinate in the execution data to the coordinate system where the actuator is located according to the conversion matrix to obtain the target conversion coordinate of the actuator, and determine the execution route of the actuator according to the target conversion coordinate; the conversion matrix represents the conversion relationship between the coordinate system where the fuel tank joint is located and the coordinate system where the actuator is located.

[0104] Set this position as the calibration position of the robotic arm. Use the Aruco code to calibrate the joint. Fix the magnet universal bracket on the flange, and pass the center of the Aruco code through the universal magnet bracket. First, identify the Aruco code and record the three-dimensional coordinates in the camera coordinate system. Then, move the center of the end of the robotic arm to the fixed point of the bracket and record the three-dimensional coordinates of the robotic arm. Return to the calibration point. Change the position of the Aruco code, record the new three-dimensional coordinate position, move the center of the end of the robotic arm to the fixed point of the bracket again, and record the new three-dimensional coordinates of the robotic arm. Repeat the above operation ten times. Denote the obtained camera coordinates as a 10×3 matrix, called matrix A; denote the three-dimensional coordinates of the robotic arm as a 10×3 matrix, called matrix B. Multiply the transpose of matrix A by matrix B to obtain matrix C. Perform SVD decomposition on matrix C, decomposing it into three matrices U, S, and Vt. Transpose and multiply matrices U and Vt respectively to obtain the rotation matrix R. Then, add up all the values of matrix A row by row and take the average to obtain a 1×3 row matrix, denoted as matrix D. Also, add up all the values of matrix B and take the average, denoted as matrix E; subtract the product of matrix R and matrix D from matrix E to obtain the translation matrix T. Combine the transpose of matrix R and matrix T to obtain a 3×4 matrix, which is the transformation matrix between the robotic arm and the spatial coordinate point, i.e., the [R|t] matrix, as follows:

[0105]

[0106] Step 800: Control the actuator to move according to the execution route so that the oil pipe joint contacts the fuel tank joint, and control the oil pipe joint to rotate according to the target rotation angle to achieve the docking of the fuel tank joint and the oil pipe joint.

[0107] Specifically, connect the two corner points of the same group to the center coordinates of the circle respectively to obtain two vectors, namely A1O and A2O. Apply the cosine formula to calculate the angles θ1 and θ2 between the two vectors and the vertical direction of the y-axis respectively. Add the two calculated angles and take the average to obtain the angle between the center position of the card slot and the vertical position, that is, the angle that the end actuator of the robotic arm needs to turn. Transmit the angle information to the robotic arm to complete the alignment operation of the buckle and the card slot.

[0108] Convert the previously obtained spatial point coordinate values through the previously calculated [R|t] matrix to obtain the spatial position coordinate values that the robotic arm needs to move to. Transmit the coordinate values to the robotic arm. The robotic arm moves to the specified coordinate point according to the planned path, and the card slot and the buckle are aligned. Then, the robotic arm inserts 20 mm along the card slot and then rotates 135° along the card slot until the buckle slides into the semi-circular structure of the anti-disconnection slot, and the robotic arm stops moving to complete the precise docking of both ends of the fueling joint. Then, reverse the operation of the robotic arm to achieve the automatic separation of both ends of the fueling interface.

[0109] In practical applications, for the specific operation flowchart of the method provided by the present invention, see Figure 6 .

[0110] First, take a photo through the camera, i.e., the imaging module, to obtain the approximate positions of the fuel tank and the fuel filling connector, i.e., the fuel tank connector. Then, plan a trajectory based on the position information of the fuel tank and the fuel tank connector. The actuating mechanism can use a robotic arm to clamp and move the tubing connector directly above the fuel filling port of the fuel tank. Take a second photo of the fuel tank connector to identify the flange of the fuel filling port, determine the ROI area, then detect the slot angle points to obtain position information, and calculate the spatial coordinates of the center of the circle and the angle that the actuating mechanism needs to rotate to clamp the tubing connector, which can also be referred to as the angle that the end of the actuating mechanism needs to rotate.

[0111] Embodiment 2

[0112] As Figure 2 shown, the embodiment of the present invention provides a system for joint docking, which includes: an acquisition module 1, a movement trajectory determination module 2, a joint image acquisition module 3, a region of interest determination module 4, a calculation module 5, an execution data determination module 6, an execution route determination module 7, and a docking module 8.

[0113] The acquisition module 1 is used to acquire the position information of the actuating mechanism and the position of the fuel tank connector.

[0114] The movement trajectory determination module 2 is used to determine the movement trajectory of the actuating mechanism according to the position information and the position.

[0115] The joint image acquisition module 3 is used to control the actuating mechanism to move according to the movement trajectory, so that the tubing connector is located above the fuel tank connector, and control the imaging module to take a photo of the fuel tank connector to obtain a joint image.

[0116] The region of interest determination module 4 is used to extract a set area in the joint image by using a circle detection algorithm and determine the area as the region of interest.

[0117] Among them, the region of interest determination module 4 specifically includes: a processing sub-module, a conversion sub-module, a set area determination sub-module, and a region of interest determination sub-module.

[0118] The processing sub-module is used to perform smoothing filtering processing on the joint image to obtain a processed joint image.

[0119] The conversion sub-module is used to perform gray conversion on the processed joint image to obtain a joint gray image.

[0120] The set area determination sub-module is used to identify the set area in the joint gray image by using a circle detection algorithm.

[0121] The region of interest determination sub-module is used to determine the region as the region of interest.

[0122] The calculation module 5 is used to detect and calculate the region of interest by using computer vision methods, and obtain the two-dimensional coordinates of the center of the region of interest and the coordinates of the card slot notch in the region of interest.

[0123] Among them, the calculation module 5 specifically includes: a center coordinate determination sub-module, a corner point determination sub-module, a card slot notch coordinate determination sub-module, a line segment determination sub-module, and a two-dimensional coordinate determination sub-module.

[0124] The center coordinate determination sub-module is used to determine the center coordinates of the region of interest.

[0125] The corner point determination sub-module is used to identify the corner points in the region of interest by using the corner point detection method, and obtain four corner points.

[0126] The card slot notch coordinate determination sub-module is used to calculate the Euclidean distance between every two of the four corner points, and determine the coordinates of each card slot notch according to the calculated Euclidean distance and the set Euclidean distance; one end of the card slot notch corresponds to one corner point; each card slot notch corresponds to two corner points.

[0127] The line segment determination sub-module is used to determine two intersecting line segments; the four endpoints of the two intersecting line segments correspond one-to-one to the four corner points.

[0128] The two-dimensional coordinate determination sub-module is used to determine the two-dimensional coordinates of the center of the region of interest according to the center coordinates and the intersection coordinates of the two intersecting line segments.

[0129] The execution data determination module 6 is used to determine the execution data according to the two-dimensional coordinates, the coordinates of the card slot notch, and the depth parameter built in the camera module; the execution data includes: the target rotation angle and the three-dimensional space coordinates of the center of the region of interest; the target rotation angle is the rotation angle of the actuator relative to the card slot notch.

[0130] Among them, the execution data determination module 6 specifically includes: a three-dimensional space coordinate determination sub-module and a target rotation angle determination sub-module.

[0131] The three-dimensional space coordinate determination sub-module is used to determine the three-dimensional space coordinates of the center of the region of interest according to the two-dimensional coordinates and the depth parameter built in the camera module.

[0132] The target rotation angle determination sub-module is used to determine the target rotation angle according to the two-dimensional coordinates and the coordinates of the card slot notch.

[0133] An execution route determination module 7 is configured to convert the three-dimensional space coordinates in the execution data into the coordinate system where the actuator is located according to a conversion matrix, obtain the target conversion coordinates of the actuator, and determine the execution route of the actuator according to the target conversion coordinates; the conversion matrix represents the conversion relationship between the three-dimensional space coordinates and the coordinates of the actuator.

[0134] A docking module 8 is configured to control the actuator to move according to the execution route, so that the oil pipe joint contacts the fuel tank joint, and control the oil pipe joint to rotate according to the target rotation angle to achieve the docking of the fuel tank joint and the oil pipe joint.

[0135] Embodiment 3

[0136] An embodiment of the present invention provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the method for joint docking described in Embodiment 1.

[0137] As an optional implementation manner, a computer-readable storage medium is further provided, which stores a computer program. When the computer program is executed by a processor, the method for joint docking described in Embodiment 1 is implemented.

[0138] The method for joint docking provided by the present invention solves the problem of automatic docking of the fueling joint of a transport vehicle using a special fuel tank joint, i.e., a fueling joint. The fuel tank information and the approximate position of the fuel tank joint are obtained through a camera to complete preliminary identification; the robotic arm, i.e., the actuator, moves towards the joint position; after reaching the planned position, the camera takes a second photo, and the ROI (Region of Interest) of the fuel tank joint is located by identifying the flange of the fuel filling port; the recognition accuracy is improved by installing a light source at the end of the actuator, and the position information of each point on the notch of the joint slot is detected by the camera, i.e., the imaging module, to calculate the center coordinates of the female end of the fuel tank joint and the end of the actuator, i.e., the degree of the rotation angle of the oil pipe joint; the end of the robotic arm rotates and completes the automatic docking operation. It is not only efficient but also can ensure the precise docking of the end of the actuator with the fueling joint to achieve automatic fueling, ensuring that the docking accuracy requirements of the fueling joint are met.

[0139] The various embodiments in this specification are described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0140] In this article, specific examples are used to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for butt-jointing connectors, characterized in that, The method is used for a docking device of a transport vehicle. The docking device includes: a fuel tank connector, a fuel pipe connector, an actuator, and a camera module; the camera module is arranged on the actuator; the fuel tank connector is arranged on the fuel tank of the transport vehicle through a flange; two slot notches are symmetrically arranged on the flange; the actuator is used for clamping the fuel pipe connector; The method includes: Obtaining the position information of the actuator and the position of the fuel tank connector; Determining the movement trajectory of the actuator according to the position information and the position; Controlling the actuator to move along the movement trajectory so that the fuel pipe connector is located above the fuel tank connector, and controlling the camera module to take a picture of the fuel tank connector to obtain a connector image; Extracting a set area in the connector image by using a circle detection algorithm and determining the area as the region of interest; Detecting and calculating the region of interest by using a computer vision method to obtain the two-dimensional coordinates of the center of the region of interest and the coordinates of the slot notches in the region of interest; Determining execution data according to the two-dimensional coordinates, the coordinates of the slot notches, and the depth parameter built in the camera module; the execution data includes: the target rotation angle and the three-dimensional space coordinates of the center of the region of interest; the target rotation angle is the rotation angle of the actuator relative to the slot notch; Converting the three-dimensional space coordinates in the execution data to the coordinate system where the actuator is located according to a transformation matrix to obtain the target transformation coordinates of the actuator, and determining the execution route of the actuator according to the target transformation coordinates; the transformation matrix represents the transformation relationship between the coordinate system where the fuel tank connector is located and the coordinate system where the actuator is located; Controlling the actuator to move according to the execution route so that the fuel pipe connector contacts the fuel tank connector, and controlling the fuel pipe connector to rotate according to the target rotation angle to realize the docking of the fuel tank connector and the fuel pipe connector.

2. The method for butt-joint of joints according to claim 1, characterized in that, Detecting and calculating the region of interest by using a computer vision method to obtain the two-dimensional coordinates of the center of the region of interest and the coordinates of the slot notches in the region of interest, specifically including: Determining the center coordinates of the region of interest; Identifying the corner points in the region of interest by using a corner point detection method to obtain four corner points; Calculating the Euclidean distance between every two of the four corner points, and determining the coordinates of each slot notch according to the calculated Euclidean distance and a set Euclidean distance; one end of the slot notch corresponds to one corner point; each slot notch corresponds to two corner points; Determining two intersecting line segments; the four endpoints of the two intersecting line segments correspond one by one to the four corner points; Determining the two-dimensional coordinates of the center of the region of interest according to the center coordinates and the intersection coordinates of the two intersecting line segments.

3. The method for butt-jointing joints according to claim 1, characterized in that, Extracting a set area in the connector image by using a circle detection algorithm and determining the area as the region of interest, specifically including: Performing smoothing filtering processing on the connector image to obtain a processed connector image; Performing gray conversion on the processed connector image to obtain a connector gray image; Identify a set area in the grayscale image of the joint using a circle detection algorithm; Determine the area as the region of interest.

4. The method for butt-joint of connectors according to claim 1, characterized in that, Determine execution data according to the two-dimensional coordinates, the coordinates of the card slot notch, and the depth parameter built in the camera module, specifically including: Determine the three-dimensional space coordinates of the center of the region of interest according to the two-dimensional coordinates and the depth parameter built in the camera module; Determine the target rotation angle according to the two-dimensional coordinates and the coordinates of the card slot notch.

5. A system for butt-jointing connectors, characterized in that, The system includes: A docking device for a transport vehicle, the docking device including: a fuel tank joint, a fuel pipe joint, an actuator, and a camera module; the camera module is arranged on the actuator; the fuel tank joint is arranged on the fuel tank of the transport vehicle through a flange; two card slot notches are symmetrically arranged on the flange; the actuator is used for clamping the fuel pipe joint; An acquisition module for acquiring the position information of the actuator and the position of the fuel tank joint; A movement trajectory determination module for determining the movement trajectory of the actuator according to the position information and the position; A joint image acquisition module for controlling the actuator to move according to the movement trajectory so that the fuel pipe joint is located above the fuel tank joint, and controlling the camera module to take a picture of the fuel tank joint to obtain a joint image; A region of interest determination module for extracting a set area in the joint image using a circle detection algorithm and determining the area as the region of interest; A calculation module for detecting and calculating the region of interest using computer vision methods to obtain the two-dimensional coordinates of the center of the region of interest and the coordinates of the card slot notch in the region of interest; An execution data determination module for determining execution data according to the two-dimensional coordinates, the coordinates of the card slot notch, and the depth parameter built in the camera module; the execution data includes: the target rotation angle and the three-dimensional space coordinates of the center of the region of interest; the target rotation angle is the rotation angle of the actuator relative to the card slot notch; An execution route determination module for converting the three-dimensional space coordinates in the execution data to the coordinate system where the actuator is located according to a transformation matrix to obtain the target transformation coordinates of the actuator, and determining the execution route of the actuator according to the target transformation coordinates; the transformation matrix represents the transformation relationship between the three-dimensional space coordinates and the coordinates of the actuator; A docking module for controlling the actuator to move according to the execution route so that the fuel pipe joint contacts the fuel tank joint, and controlling the fuel pipe joint to rotate according to the target rotation angle to achieve the docking of the fuel tank joint and the fuel pipe joint.

6. The system for butt-jointing joints according to claim 5, characterized in that, The calculation module specifically includes: A center coordinate determination sub-module for determining the center coordinates of the region of interest; A corner point determination sub-module for identifying corner points in the region of interest using a corner point detection method to obtain four corner points; The card slot notch coordinate determination sub-module is used to calculate the Euclidean distance between every two of the four corner points, and determine the coordinates of each card slot notch according to the calculated Euclidean distance and the set Euclidean distance; one end of each card slot notch corresponds to one corner point; each card slot notch corresponds to two corner points; The line segment determination sub-module is used to determine two intersecting line segments; the four endpoints of the two intersecting line segments correspond one-to-one to the four corner points; The two-dimensional coordinate determination sub-module is used to determine the two-dimensional coordinates of the center of the region of interest according to the center coordinates of the circle and the intersection coordinates of the two intersecting line segments.

7. The system for butt-joint of connectors according to claim 5, characterized in that, The region of interest determination module specifically includes: The processing sub-module is used to perform smoothing filtering on the joint image to obtain the processed joint image; The conversion sub-module is used to perform gray conversion on the processed joint image to obtain the joint gray image; The set region determination sub-module is used to identify the set region in the joint gray image by using a circle detection algorithm; The region of interest determination sub-module is used to determine the region as the region of interest.

8. The system for butt-jointing joints according to claim 5, characterized in that, The execution data determination module specifically includes: The three-dimensional space coordinate determination sub-module is used to determine the three-dimensional space coordinates of the center of the region of interest according to the two-dimensional coordinates and the depth parameter built in the camera module; The target rotation angle determination sub-module is used to determine the target rotation angle according to the two-dimensional coordinates and the coordinates of the card slot notch.

9. An electronic device, characterized in that, It includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the method for joint docking described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, It stores a computer program, and when the computer program is executed by the processor, it implements the method for joint docking described in any one of claims 1 to 4.

Images