Automatic docking method of dynamic flanges for fluid loading and unloading arms based on image visual servoing

Abstract

The present invention discloses a method for automatic docking of dynamic flanges of a fluid loading and unloading arm based on image visual servoing, comprising: initializing the end position of the loading and unloading arm, controlling the loading and unloading arm to move until the complete image information of the flange to be docked can be collected; collecting the image of the flange to be docked, and using the Hough circle detection algorithm to extract the initial candidate circle with circular features in the image; performing fine fitting through the RANSAC algorithm to obtain a fine candidate circle; extracting the coordinates of the center of the positioning hole from the fine candidate circle; calculating the deviation at the current moment based on the coordinates of the center of the positioning hole and its docking state coordinate position; solving the angular velocity instructions of the angle movement of each joint of the loading and unloading arm with the goal of minimizing the deviation, driving each joint of the loading and unloading arm to adjust the posture until the deviation converges, and completing the docking of the end flange of the loading and unloading arm with the flange to be docked. The present invention is suitable for millimeter-level precise docking and control of loading and unloading arm flanges in dynamic marine environments, significantly improving the operating efficiency and safety in complex environments.

Description

Technical Field

[0001] The present invention relates to the technical field of marine engineering equipment automation, and in particular to an automatic docking method for dynamic flanges of a fluid loading and unloading arm based on image visual servoing. Background Art

[0002] With the development of automation technology for marine engineering equipment, the demand for automatic docking of fluid loading and unloading arms and ship flanges is becoming increasingly urgent in scenarios such as oil and gas transportation and LNG loading and unloading. Traditional docking methods rely on manual operation or mechanical positioning devices in static environments, which achieve flange alignment through fixed guide structures or operator visual calibration. However, in a dynamic marine environment, ships are affected by waves, wind, and tides, resulting in high-frequency shaking, which leads to the following limitations of traditional methods: manual visual calibration relies on operator experience, and repeated manual adjustments cause a single docking to take a lot of time, which cannot meet the needs of modern efficient operations and is prone to the risk of misoperation in severe sea conditions; existing mechanical positioning systems are mostly based on preset paths or fixed coordinate systems, and cannot adapt to the dynamic changes in the ship's position in real time, making it difficult to guarantee docking accuracy and sealing requirements.

[0003] In recent years, some studies have attempted to apply visual servoing technology to flange docking in dynamic environments, but existing methods still have significant drawbacks: Vision-guided methods require the creation and placement of pose encoding points for the flanges to be docked, resulting in a complex workflow and unsuitable for the production cycle of port operations. Visual servoing methods based on static feature matching (such as template matching and SIFT feature points) have poor robustness in dynamic environments, with features easily lost or mismatched. Open-loop control methods based on predefined trajectories lack the ability to dynamically compensate for real-time deviations and are difficult to adapt to real-time pose changes. Most image-based visual servoing (IBVS) algorithms are highly complex, with the process from data processing to control command generation significantly lagging behind the dynamic transition from the current state of the flanges to the next. Therefore, an automated method for rapid and precise flange docking in dynamic marine environments is urgently needed. Summary of the Invention

[0004] To address the existing challenges of automated docking of loading arm flanges in dynamic marine environments, this paper proposes a method for automated docking of dynamic fluid loading arm flanges based on image visual servoing. Through real-time image data acquisition and feature detection, pose deviation calculation, dynamic motion planning, and multi-joint coordinated control, the method aims to achieve millimeter-level precision docking of loading arm flanges, significantly improving operational efficiency and safety in complex environments.

[0005] In order to achieve the above technical objectives, the present invention provides the following technical solutions:

[0006] The method for automatic docking of dynamic flanges of fluid loading and unloading arms based on image visual servoing specifically includes:

[0007] S1. Manually initialize the end position of the loading and unloading arm and control the loading and unloading arm to move until the industrial camera at its end can collect complete image information of the flange to be connected;

[0008] S2. At any time t, an image of the flange to be connected is collected, and the Hough circle detection algorithm is used to extract initial candidate circles with circular features in the image;

[0009] S3. Using the RANSAC algorithm that combines geometric constraints with statistical verification, the initial candidate circle is refined to obtain a refined candidate circle.

[0010] S4. Extracting the center coordinates of four positioning holes from the refined candidate circles according to the geometric characteristics of the positioning holes;

[0011] S5. Calculate the current moment deviation based on the extracted center coordinates of the four positioning holes and their docking state coordinate positions;

[0012] S6, to minimize The deviation at the moment is taken as the target, and the angular velocity instructions of the joints of the loading and unloading arm are solved. The joints of the loading and unloading arm are driven to adjust their posture until the deviation converges, and the end flange of the loading and unloading arm is docked with the flange to be docked.

[0013] Furthermore, step S2 specifically includes:

[0014] S21. At any time t, collect the image of the flange to be connected and obtain RGB image data , and then Convert to grayscale ;

[0015] S22, use Canny edge detector to detect grayscale images Perform edge detection to obtain a set of edge points consisting of non-zero pixels and its corresponding gradient direction set ;

[0016] S23, according to the Hough circle detection method, the edge point set Each edge point in , according to its gradient direction , Calculate possible circle center coordinates , the calculation formula is as follows:

[0017] ;

[0018] in, According to the prior size of the flange positioning hole, the range is set The inner circle is sampled according to the set step size, which is also the radius of the candidate circle; Indicates the edge points, Indicates the Sampling step length;

[0019] S24. Construct a two-dimensional accumulator matrix , for every possible coordinate of the circle center Count the number of votes, based on the set voting threshold , retain the candidate circle center coordinates whose votes exceed the threshold and the corresponding candidate circle radius ;

[0020] S26, the radius of the candidate circle With standard radius The relative error exceeds the preset radius threshold The candidate circles with the highest number of votes are eliminated, and the remaining candidate circles are sorted in descending order according to the number of votes, and the top candidate circles are retained. candidate circles.

[0021] Furthermore, step S3 specifically includes:

[0022] S31, define each candidate circle as is the center of the circle, An annular area with a radius of ;

[0023] S32, through the Zernike moment method Sub-pixel positioning of edge points within the region is performed to extract edge point sets that meet sub-pixel accuracy ;

[0024] S33, from the edge point set Randomly select 3 non-collinear points , , use the three-point circle fitting method to calculate the center coordinates of the temporary circle and radius ;

[0025] S34, calculation All points Temporary circle fitting error , if the temporary circle fitting error If the pixel value does not exceed the set pixel threshold, Mark as an interior point and add it to the interior point set ;

[0026] S35, repeat steps S33 to S34 m times to obtain m inner point sets; then retain the inner point set with the largest number of candidate circle inner points ;

[0027] S36. For each interior point set The least squares optimization is performed on the points in the circle, and the center of the circle is The center coordinates of each candidate circle after refinement and radius , the refined candidate circle is recorded as: ;

[0028] S37. Constructing a refined candidate circle set , calculate each Radius With standard radius The error is less than the threshold. The refined candidate circle is updated.

[0029] Furthermore, step S4 specifically includes:

[0030] S41. Randomly select four refined candidate circles from the updated refined candidate circle set, take the center of each circle, and sort them according to the adjacent relationship to obtain four refined candidate circle centers, which are expressed as: 、 、 、 ;remember To refine the candidate circle center combination, adjacent in sequence;

[0031] S42, calculation The length of each side of the quadrilateral formed by the centers of adjacent circles is expressed as:

[0032] ;

[0033] in, Represents the length of any side of a quadrilateral; calculate the lengths of the four sides in sequence to obtain the set of side lengths ;

[0034] S43, calculation The angle between the adjacent sides of the quadrilateral formed by the center of the circle; the formula is:

[0035] ;

[0036] in, 、 The lengths of any pair of adjacent sides of a quadrilateral, is the angle between any two adjacent sides of the quadrilateral; calculate the four angles in sequence to get the set of interior angles ; Indicates the number of the corresponding refined candidate circle center; is the horizontal coordinate corresponding to the center of the circle, is the ordinate corresponding to the center of the circle;

[0037] S44, according to The set of side lengths and the set of interior angles Calculate the comprehensive error of the square feature ,judge Is it a valid combination?

[0038] S45. From the updated set of refined candidate circles, obtain all selected combinations of four random refined candidate circles, repeat steps S41 to S44, and select the combination with the smallest comprehensive error as the positioning hole center combination. The comprehensive error of the positioning hole center combination is recorded as ,like Less than the comprehensive error threshold , then the positioning hole center combination is determined to be valid, and the coordinates of the four positioning hole centers are used as the current coordinates of the positioning hole centers, which are expressed as: ;

[0039] S46, if Not less than the comprehensive error threshold , the industrial camera is triggered to reacquire the image of the flange to be connected, and steps S2 to S45 are repeated until the current coordinates of the centers of the four positioning holes are obtained.

[0040] Furthermore, step S5 specifically includes:

[0041] S51, if the docking state coordinates of the center of the four positioning holes are not obtained, the loading and unloading arm is manually controlled to complete the docking, and the docking state coordinates of the center of the four positioning holes are obtained in the docking state, which are recorded as The docking coordinates of the centers of the four positioning holes of flanges at different positions only need to be obtained once. If the docking coordinates of the centers of the four positioning holes have been obtained, skip this step.

[0042] S52, under the premise of ensuring that the relative position of the positioning hole center remains unchanged, the coordinates of the four positioning hole centers are reordered according to the distance between the current coordinates of the positioning hole center and the docking state coordinates. The reordering is recorded as ;

[0043] in, The coordinates of the docking state with the coordinates of the center of the positioning hole The coordinates of the center of the circle closest to ;

[0044] S53, convert the current extraction coordinates and docking state coordinates of the centers of the four positioning holes into vectors and , and use this to calculate the deviation at the current moment .

[0045] Furthermore, step S6 specifically includes:

[0046] S61. Get the vector composed of the current joint angle values ​​of the loading and unloading arm ;

[0047] S62. Assume that the joint angle update formula is: ,in is the angular velocity control signal of each joint of the loading and unloading arm in step k, hour, ;

[0048] S63, constructing a summation function of the position deviation and the control signal amplitude in the time domain N ;

[0049] S64, introduce the minimum angular velocity limit of any joint i of the loading and unloading arm and maximum angular velocity limit , that is, to meet , correspond The angular velocity control signal of the i-th joint in is converted into the problem of solving the control signal to minimize the sum function Problems;

[0050] S65, use IPOPT nonlinear optimization solver to solve the above constrained minimization problem and obtain the optimal control sequence ;

[0051] S66, with the optimal control sequence The first control signal in Drive the joints of the loading and unloading arm to adjust their posture and update the joint angle vector of the loading and unloading arm ;

[0052] S67, repeat steps S2 to S5, calculate the deviation at the current moment, and determine whether the current deviation and the deviation change rate are both less than the corresponding thresholds;

[0053] S68. If the conditions in step S67 are not met at the same time, execute steps S2 to S67;

[0054] S69: If the conditions in step S67 are met at the same time, execute steps S61 to S66, drive the flange at the end of the loading and unloading arm to the docking position, and complete the docking by the quick docking device at the end of the loading and unloading arm.

[0055] In addition, the present invention also discloses an application system of the above-mentioned fluid loading and unloading arm dynamic flange automatic docking method, which specifically includes: a visual perception module, an image processing unit, a motion control unit and a loading and unloading arm actuator;

[0056] The visual perception module is composed of an industrial camera installed at the end of the loading and unloading arm, and is used to realize the image acquisition in step S2;

[0057] The image processing unit processes the image data acquired by the visual perception module to implement the Hough circle detection method in step S2 and steps S3-S5;

[0058] The motion control unit generates a joint angle motion instruction based on the real-time position of the positioning hole coordinates and the position deviation of the docking state, so as to implement the instruction solution in step S6;

[0059] The loading and unloading arm actuator executes the motion instructions issued by the motion control unit to realize the driving of the joints of the loading and unloading arm to adjust the posture in steps S1 and S6, and complete the docking of the end flange of the loading and unloading arm with the flange to be docked.

[0060] Based on the above technical solution, the present invention has at least the following beneficial effects:

[0061] 1. Through the initial feature extraction of Hough circle detection and the refined fitting of RANSAC circle features, sub-pixel feature extraction of images is achieved, effectively suppressing image noise and feature offset caused by ship shaking, and enhancing the stability of flange hole positioning in complex marine environments;

[0062] 2. Introducing orthogonal distribution constraints of flange positioning hole side length consistency and angle constraints, and screening effective circle combinations through comprehensive geometric error thresholds, significantly reducing the false detection rate caused by environmental interference and ensuring the reliability of feature recognition;

[0063] 3. Using a nonlinear optimization solver, with the goal of minimizing time-domain deviation, the optimal control sequence is generated while satisfying the physical constraints of the joints, achieving real-time posture compensation for ship sway and significantly improving system response efficiency.

[0064] 4. A complete closed loop is formed from initial positioning to final docking, without the need for manual calibration or path teaching. The system supports a "one-time calibration, multiple reuse" mechanism. For flanges to be docked at different positions, only the docking status coordinates need to be obtained once, significantly reducing deployment complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0066] Figure 1 This is a flow chart of the method for automatic docking of dynamic flanges of fluid loading and unloading arms based on image visual servoing proposed by the present invention;

[0067] Figure 2This is a schematic diagram of the system composition and information transmission relationship of the dynamic flange automatic docking system for fluid loading and unloading arms based on image visual servoing proposed in the present invention;

[0068] Figure 3 This is a simplified structural diagram of the fluid loading and unloading arm and its important components for installation;

[0069] Figure 3 Middle: 1. Loading arm joint; 2. Industrial camera; 3. Quick docking device at the end of the loading arm; 4. Flange at the end of the loading arm. DETAILED DESCRIPTION

[0070] In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, the following Figure 1-3 The present invention is further described in detail with specific implementation methods, so that the application can fully understand how to use technical means to solve technical problems and achieve technical effects and implement them accordingly.

[0071] Those skilled in the art will appreciate that all or part of the steps in the above-mentioned embodiment methods can be accomplished by instructing the relevant hardware through a program. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] Please refer to Figure 1 This embodiment proposes a method for automatic docking of dynamic flanges of fluid loading and unloading arms based on image visual servoing, which includes the following steps:

[0073] S1. Manually initialize the end position of the loading and unloading arm and control the loading and unloading arm to move until the industrial camera at its end can collect complete image information of the flange to be connected;

[0074] S2. At any time t, an image of the flange to be connected is collected, and the Hough circle detection algorithm is used to extract initial candidate circles with circular features in the image;

[0075] As a preferred embodiment, step S2 specifically includes:

[0076] S21. At any time t, collect the image of the flange to be connected and obtain RGB image data , and then Convert to grayscale In this embodiment, The grayscale value of each pixel in the image is calculated by weighted average method. The calculation formula is:

[0077] ;

[0078] in, are the pixel values ​​corresponding to the three channels of the color image;

[0079] S22, use Canny edge detector to detect grayscale images Perform edge detection to obtain a set of edge points consisting of non-zero pixels and its corresponding gradient direction set ;

[0080] S23, according to the Hough circle detection method, the edge point set Each edge point in , according to its gradient direction , Calculate possible circle center coordinates , the calculation formula is as follows:

[0081] ;

[0082] in, According to the prior size of the flange positioning hole, the range is set The inner circle is sampled according to the set step size, which is also the radius of the candidate circle; Indicates the edge points, Indicates the Sampling step length;

[0083] S24. Construct a two-dimensional accumulator matrix , for every possible coordinate of the circle center Count the number of votes, based on the set voting threshold , retain the candidate circle center coordinates whose votes exceed the threshold and the corresponding candidate circle radius ;

[0084] S26, the radius of the candidate circle With standard radius The relative error exceeds the preset radius threshold The candidate circles with the highest number of votes are eliminated, and the remaining candidate circles are sorted in descending order according to the number of votes, and the top candidate circles are retained. (Usually take ) candidate circles.

[0085] S3. Using the RANSAC algorithm that combines geometric constraints with statistical verification, the initial candidate circle is refined to obtain a refined candidate circle.

[0086] As a preferred embodiment, step S3 specifically includes:

[0087] S31, define each candidate circle as is the center of the circle, An annular area with a radius of ;

[0088] ;

[0089] S32, through the Zernike moment method Sub-pixel positioning of edge points within the region is performed to extract edge point sets that meet sub-pixel accuracy ;

[0090] S33, from the edge point set Randomly select 3 non-collinear points , , use the three-point circle fitting method to calculate the center coordinates of the temporary circle and radius ;

[0091] S34, calculation All points Temporary circle fitting error , the specific calculation formula is: ; If the temporary circle fitting error If the pixel threshold does not exceed the set pixel threshold (the pixel threshold is set to 0.5 pixels in this embodiment), Mark as an interior point and add it to the interior point set ;

[0092] S35, repeat steps S33 to S34 m times to obtain m inner point sets; then retain the inner point set with the largest number of candidate circle inner points , that is, The set of interior points with the most interior points of the candidate circle of the circle center;

[0093] S36. For each interior point set The least squares optimization is performed on the points in the circle, and the center of the circle is The center coordinates of each candidate circle after refinement and radius , the refined candidate circle is recorded as: ;

[0094] S37. Constructing a refined candidate circle set , calculate each Radius With standard radius The error is less than the threshold. The refined candidate circle is updated.

[0095] S4. Extracting the center coordinates of four positioning holes from the refined candidate circles according to the geometric characteristics of the positioning holes;

[0096] As a preferred embodiment, step S4 specifically includes:

[0097] S41. Randomly select four refined candidate circles from the updated refined candidate circle set, take the center of each circle, and sort them according to the adjacent relationship to obtain four refined candidate circle centers, which are expressed as: 、 、 、 ;remember To refine the candidate circle center combination, adjacent in sequence;

[0098] S42, calculation The length of each side of the quadrilateral formed by the centers of adjacent circles is expressed as:

[0099] ;

[0100] in, Represents the length of any side of a quadrilateral; calculate the lengths of the four sides in sequence to obtain the set of side lengths ;

[0101] S43, calculation The angle between the adjacent sides of the quadrilateral formed by the center of the circle; the formula is:

[0102] ;

[0103] in, 、 The lengths of any pair of adjacent sides of a quadrilateral, is the angle between any two adjacent sides of the quadrilateral; calculate the four angles in sequence to get the set of interior angles ; Indicates the number of the corresponding refined candidate circle center; is the horizontal coordinate corresponding to the center of the circle, is the ordinate corresponding to the center of the circle;

[0104] S44, according to The set of side lengths and the set of interior angles Calculate the comprehensive error of the square feature ,judge Is it a valid combination? In this embodiment, the comprehensive error ,in 、 、 It is the standard side length set according to the flange geometry.

[0105] In addition, it should be noted that this embodiment is aimed at the 12-hole evenly distributed flange that is most widely used in loading and unloading arms. Therefore, a square with the centers of the four flange hole end faces as vertices can be found on the flange design model, which is consistent with the centers of the four positioning holes extracted in this application.

[0106] S45. From the updated set of refined candidate circles, obtain all selected combinations of four random refined candidate circles, repeat steps S41 to S44, and select the combination with the smallest comprehensive error as the positioning hole center combination. The comprehensive error of the positioning hole center combination is recorded as ,like Less than the comprehensive error threshold , then the positioning hole center combination is determined to be valid, and the coordinates of the four positioning hole centers are used as the current coordinates of the positioning hole centers, which are expressed as: ;

[0107] S46, if Not less than the comprehensive error threshold , the industrial camera is triggered to reacquire the image of the flange to be connected, and steps S2 to S45 are repeated until the current coordinates of the centers of the four positioning holes are obtained.

[0108] S5. Calculate the current moment deviation based on the extracted center coordinates of the four positioning holes and their docking state coordinate positions;

[0109] As a preferred embodiment, step S5 specifically includes:

[0110] S51, if the docking state coordinates of the center of the four positioning holes are not obtained, the loading and unloading arm is manually controlled to complete the docking, and the docking state coordinates of the center of the four positioning holes are obtained in the docking state, which are recorded as The docking coordinates of the centers of the four positioning holes of flanges at different positions only need to be obtained once. If the docking coordinates of the centers of the four positioning holes have been obtained, skip this step.

[0111] It should be noted that the method for obtaining the docking state coordinates of the center of the positioning hole in this application can refer to steps S2 to S46, which are obtained under the ideal state of completely precise docking.

[0112] S52, under the premise of ensuring that the relative position of the positioning hole center remains unchanged, the coordinates of the four positioning hole centers are reordered according to the distance between the current coordinates of the positioning hole center and the docking state coordinates. The reordering is recorded as ;

[0113] in, The coordinates of the docking state with the coordinates of the center of the positioning hole The coordinates of the center of the circle closest to ;

[0114] S53, convert the current extraction coordinates and docking state coordinates of the centers of the four positioning holes into vectors and , and use this to calculate the deviation at the current moment ; and Specifically expressed as:

[0115] ; .

[0116] S6, to minimize The deviation at the moment is taken as the target, and the angular velocity instructions of the joints of the loading and unloading arm are solved. The joints of the loading and unloading arm are driven to adjust their posture until the deviation converges, and the end flange of the loading and unloading arm is docked with the flange to be docked.

[0117] As a preferred embodiment, step S6 specifically includes:

[0118] S61. Get the vector composed of the current joint angle values ​​of the loading and unloading arm ;

[0119] S62. Assume that the joint angle update formula is: ,in is the angular velocity control signal of each joint of the loading and unloading arm in step k, hour, ;

[0120] S63, constructing a summation function of the position deviation and the control signal amplitude in the time domain N ; In this embodiment, the summation function ;in, In the time domain N An array of sequences; It is a hyperparameter and can be set based on engineering experience; , is the image Jacques matrix, which is calculated by the center coordinate information of the four positioning holes extracted in step S4 and the depth estimation method based on the Kalman filter;

[0121] S64, introduce the minimum angular velocity limit of any joint i of the loading and unloading arm and maximum angular velocity limit , that is, to meet , correspond The angular velocity control signal of the i-th joint in is converted into the problem of solving the control signal to minimize the sum function the problem; that is ;

[0122] S65, use IPOPT nonlinear optimization solver to solve the above constrained minimization problem and obtain the optimal control sequence ;

[0123] S66, with the optimal control sequence The first control signal in Drive the joints of the loading and unloading arm to adjust their posture and update the joint angle vector of the loading and unloading arm ; The loading and unloading arm under the optimal control sequence obtains a relatively accurate docking posture. Next, it is necessary to calculate the deviation between the current coordinates of the center of the positioning hole and the docking state coordinates to determine whether the docking task is completed.

[0124] S67, repeat steps S2 to S5 to calculate the deviation at the current moment , to determine whether the current deviation and the deviation change rate are both less than the corresponding threshold, that is, whether they both meet and ,in, is the deviation threshold, is the deviation change rate threshold;

[0125] S68. If the conditions in step S67 are not met simultaneously, then execute steps S2 to S67; that is, readjust the posture of the loading and unloading arm, calculate the deviation at the current moment, and then re-determine whether the two conditions in step S67 are met simultaneously;

[0126] S69: If the conditions in step S67 are met at the same time, execute steps S61 to S66, drive the flange at the end of the loading and unloading arm to the docking position, and complete the docking by the quick docking device at the end of the loading and unloading arm.

[0127] It should be noted that when the two conditions in step S67 are met at the same time, it can be considered that a perfect and precise docking position has been reached. The execution of steps S61-S66 here is a safety redundant step to ensure that the docking can be completed.

[0128] In addition, if Figure 3 As shown, the present invention also discloses an application system of the above-mentioned fluid loading and unloading arm dynamic flange automatic docking method, which specifically includes: a visual perception module, an image processing unit, a motion control unit and a loading and unloading arm actuator; Figure 2 It shows the composition and information transmission relationship of the system, specifically:

[0129] The visual perception module is composed of an industrial camera installed at the end of the loading and unloading arm, and is used to realize the image acquisition in step S2;

[0130] The image processing unit processes the image data acquired by the visual perception module to implement the Hough circle detection method in step S2 and steps S3-S5;

[0131] The motion control unit generates a joint angle motion instruction based on the real-time position of the positioning hole coordinates and the position deviation of the docking state, so as to implement the instruction solution in step S6;

[0132] The loading and unloading arm actuator executes the motion instructions issued by the motion control unit to realize the driving of the joints of the loading and unloading arm to adjust the posture in steps S1 and S6, and complete the docking of the end flange of the loading and unloading arm with the flange to be docked.

[0133] In summary, the proposed method and application system for dynamic flange automatic docking of fluid loading and unloading arms based on image visual servoing achieves millimeter-level precision docking of loading and unloading arm flanges through real-time image data acquisition and feature detection, pose deviation calculation, dynamic motion planning, and multi-joint coordinated control, significantly improving operational efficiency and safety in complex environments.

[0134] Throughout this specification, terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" refer to at least one embodiment or example described in conjunction with specific features, structures, materials, or characteristics. These specific features, structures, materials, or characteristics may be combined in appropriate manners in one or more embodiments or examples. Furthermore, a skilled artisan may combine and integrate different embodiments or examples described in this specification and their features, unless they are mutually incompatible.

[0135] The logic and / or steps shown in the flowcharts or otherwise described can be considered a sequence of executable instructions for implementing the logical functions. These instructions can be embodied in any computer-readable medium for use by an instruction execution system, device, or apparatus. These systems, devices, or apparatuses include processor systems or other systems capable of receiving and executing instructions.

[0136] The above embodiments have detailed the principles and implementation methods of the present invention, and illustrated its working principles using specific examples. These examples are intended only to facilitate understanding of the method and core concepts of the present invention. Furthermore, actual implementation methods and applications may vary depending on the principles of the present invention. Therefore, this specification should not be construed as limiting the present invention.

Claims

1. A method for automatic docking of dynamic flanges of fluid loading and unloading arms based on image visual servoing, characterized in that: The specific steps include: S1. Manually initialize the end position of the loading and unloading arm and control the loading and unloading arm to move until the industrial camera at its end can collect complete image information of the flange to be connected; S2. At any time t, collect an image of the flange to be connected, and use the Hough circle detection algorithm to extract initial candidate circles with circular features in the image; specifically, the following steps are performed: S21. At any time t, collect the image of the flange to be connected and obtain RGB image data , and then Convert to grayscale ; S22, use Canny edge detector to detect grayscale images Perform edge detection to obtain a set of edge points consisting of non-zero pixels and its corresponding gradient direction set ; S23, according to the Hough circle detection method, the edge point set Each edge point in , according to its gradient direction , Calculate possible circle center coordinates , the calculation formula is as follows: ； in, According to the prior size of the flange positioning hole, the range is set The inner circle is sampled according to the set step size, which is also the radius of the candidate circle; Indicates the edge points, Indicates the Sampling step length; S24. Construct a two-dimensional accumulator matrix , for every possible coordinate of the circle center Count the number of votes, based on the set voting threshold , retain the candidate circle center coordinates whose votes exceed the threshold and the corresponding candidate circle radius ; S26, the radius of the candidate circle With standard radius The relative error exceeds the preset radius threshold The candidate circles with the highest number of votes are eliminated, and the remaining candidate circles are sorted in descending order according to the number of votes, and the top candidate circles are retained. candidate circles; S3. Using the RANSAC algorithm that combines geometric constraints with statistical verification, the initial candidate circle is refined to obtain a refined candidate circle. This includes: S31, define each candidate circle as is the center of the circle, An annular area with a radius of ; S32, through the Zernike moment method Sub-pixel positioning of edge points within the region is performed to extract edge point sets that meet sub-pixel accuracy ; S33, from the edge point set Randomly select 3 non-collinear points , , use the three-point circle fitting method to calculate the center coordinates of the temporary circle and radius ; S34, calculation All points Temporary circle fitting error , if the temporary circle fitting error If the pixel value does not exceed the set pixel threshold, Mark as an interior point and add it to the interior point set ; S35, repeat steps S33 to S34 m times to obtain m inner point sets; then retain the inner point set with the largest number of candidate circle inner points ; S36. For each interior point set The least squares optimization is performed on the points in the circle, and the center of the circle is The center coordinates of each candidate circle after refinement and radius , the refined candidate circle is recorded as: ; S37. Constructing a refined candidate circle set , calculate each Radius With standard radius The error is less than the threshold. Refined candidate circles of , update the refined candidate circle set; S4. Extracting the center coordinates of four positioning holes from the refined candidate circles according to the geometric characteristics of the positioning holes; S5. Calculate the current moment deviation based on the extracted center coordinates of the four positioning holes and their docking state coordinate positions; S6, to minimize The deviation at the moment is taken as the target, and the angular velocity instructions of the joints of the loading and unloading arm are solved. The joints of the loading and unloading arm are driven to adjust their posture until the deviation converges, and the end flange of the loading and unloading arm is docked with the flange to be docked.

2. The method for automatic dynamic flange docking of fluid loading and unloading arms based on image visual servoing according to claim 1 is characterized in that: Step S4 specifically includes: S41. Randomly select four refined candidate circles from the updated refined candidate circle set, take the center of each circle, and sort them according to the adjacent relationship to obtain four refined candidate circle centers, which are expressed as: 、 、 、 ;remember To refine the candidate circle center combination, adjacent in sequence; S42, calculation The length of each side of the quadrilateral formed by the centers of adjacent circles is expressed as: ； in, Represents the length of any side of a quadrilateral; calculate the lengths of the four sides in sequence to obtain the set of side lengths ; S43, calculation The angle between the adjacent sides of the quadrilateral formed by the center of the circle; the formula is: ； in, 、 The lengths of any pair of adjacent sides of a quadrilateral, is the angle between any two adjacent sides of the quadrilateral; calculate the four angles in sequence to get the set of interior angles ; Indicates the number of the corresponding refined candidate circle center; is the horizontal coordinate corresponding to the center of the circle, is the ordinate corresponding to the center of the circle; S44, according to The set of side lengths and the set of interior angles Calculate the comprehensive error of the square feature ,judge Is it a valid combination? S45. From the updated set of refined candidate circles, obtain all selected combinations of four random refined candidate circles, repeat steps S41 to S44, and select the combination with the smallest comprehensive error as the positioning hole center combination. The comprehensive error of the positioning hole center combination is recorded as ,like Less than the comprehensive error threshold , then the positioning hole center combination is determined to be valid, and the coordinates of the four positioning hole centers are used as the current coordinates of the positioning hole centers, which are expressed as: ; S46, if Not less than the comprehensive error threshold , the industrial camera is triggered to reacquire the image of the flange to be connected, and steps S2 to S45 are repeated until the current coordinates of the centers of the four positioning holes are obtained.

3. The method for automatic dynamic flange docking of fluid loading and unloading arms based on image visual servoing according to claim 2 is characterized in that: Step S5 specifically includes: S51, if the docking state coordinates of the center of the four positioning holes are not obtained, the loading and unloading arm is manually controlled to complete the docking, and the docking state coordinates of the center of the four positioning holes are obtained in the docking state, which are recorded as The docking coordinates of the centers of the four positioning holes of flanges at different positions only need to be obtained once. If the docking coordinates of the centers of the four positioning holes have been obtained, skip this step. S52, under the premise of ensuring that the relative position of the positioning hole center remains unchanged, the coordinates of the four positioning hole centers are reordered according to the distance between the current coordinates of the positioning hole center and the docking state coordinates. The reordering is recorded as ; in, The coordinates of the docking state with the coordinates of the center of the positioning hole The coordinates of the center of the circle closest to ; S53, convert the current extraction coordinates and docking state coordinates of the centers of the four positioning holes into vectors and , and use this to calculate the deviation at the current moment .

4. According to the method for automatic dynamic flange docking of a fluid loading and unloading arm based on image visual servoing according to claim 1, step S6 specifically comprises: S61. Get the vector composed of the current joint angle values ​​of the loading and unloading arm ; S62. Assume that the joint angle update formula is: ,in is the angular velocity control signal of each joint of the loading and unloading arm in step k, hour, ; S63, constructing a summation function of the position deviation and the control signal amplitude in the time domain N ; S64, introduce the minimum angular velocity limit of any joint i of the loading and unloading arm and maximum angular velocity limit , that is, to meet , correspond The angular velocity control signal of the i-th joint in is converted into the problem of solving the control signal to minimize the sum function Problems; S65, use IPOPT nonlinear optimization solver to solve the above constrained minimization problem and obtain the optimal control sequence ; S66, with the optimal control sequence The first control signal in Drive the joints of the loading and unloading arm to adjust their posture and update the joint angle vector of the loading and unloading arm ; S67, repeat steps S2 to S5, calculate the deviation at the current moment, and determine whether the current deviation and the deviation change rate are both less than the corresponding thresholds; S68. If the conditions in step S67 are not met at the same time, execute steps S2 to S67; S69: If the conditions in step S67 are met at the same time, execute steps S61 to S66, drive the flange at the end of the loading and unloading arm to the docking position, and complete the docking by the quick docking device at the end of the loading and unloading arm.

5. An application system of the method for automatic docking of dynamic flanges of fluid loading and unloading arms according to any one of claims 1 to 4, characterized in that: Specifically include: Visual perception module, image processing unit, motion control unit and loading and unloading arm actuator; The visual perception module is composed of an industrial camera installed at the end of the loading and unloading arm, and is used to realize the image acquisition in step S2; The image processing unit processes the image data acquired by the visual perception module to implement the Hough circle detection method in step S2 and steps S3-S5; The motion control unit generates a joint angle motion instruction based on the real-time position of the positioning hole coordinates and the position deviation of the docking state, so as to implement the instruction solution in step S6; The loading and unloading arm actuator executes the motion instructions issued by the motion control unit to realize the driving of the joints of the loading and unloading arm to adjust the posture in steps S1 and S6, and complete the docking of the end flange of the loading and unloading arm with the flange to be docked.

Images