Automatic hexagonal sleeve and outer hexagonal bolt alignment method and system based on coaxial vision at tail end of mechanical arm, electronic equipment and storage medium

Abstract

The invention belongs to the technical field of machine vision, and provides a hexagonal sleeve and outer hexagonal bolt automatic alignment method and system based on mechanical arm tail end coaxial vision, electronic equipment and a storage medium, and the method comprises the steps of image collection, contour detection, plane alignment, center axis alignment, minimum rotation angle calculation and rotation adjustment. According to the method, through contour detection, plane alignment, center axis alignment and minimum rotation angle calculation, a pre-training process is avoided, the computing power requirement and deployment cost are reduced, and the execution efficiency is improved; and plane alignment and center axis alignment are carried out by using a dichotomy, so that the alignment error is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a method, system, electronic equipment and storage medium for automatically aligning a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a robotic arm. Background Art

[0002] Hexagonal bolts are a common connecting component in industrial applications. However, they can be easily loosened by factors such as wind and thermal expansion and contraction, posing a safety hazard. Therefore, proper maintenance and repair of these bolts is crucial. During the repair process, the precise alignment of the hexagonal socket and the hexagonal bolt is crucial for automated tightening of the hexagonal socket at the end of the robotic arm. This is a prerequisite for reliable bolt tightening.

[0003] Publication number CN119515783A proposes a 6D pose detection and visual servoing method for bolts based on 3D template matching. This method requires a CAD model of the bolts and the use of a depth camera, making its implementation costly. Publication number CN118809153A proposes a force-controlled bolt alignment method using a robotic arm based on a rotational search strategy. This method achieves a low bolt alignment error of 1mm to 3mm. Summary of the Invention

[0004] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a method, system, electronic equipment and storage medium for automatic alignment of a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm, so as to solve the problems of high cost and low precision of the existing methods.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for automatically aligning a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm, comprising:

[0007] The robotic arm is moved to a pre-alignment position, and an RGB camera on the robotic arm is used to capture an image of the pre-alignment position to obtain an image to be analyzed; the RGB camera is kept coaxial with the hexagonal sleeve at the end of the robotic arm;

[0008] Performing contour detection on the image to be analyzed using the Hough circle transform algorithm, the Canny algorithm, and the Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt;

[0009] Performing planar alignment between the RGB camera and the plane where the target hexagonal bolt is located using a dichotomy method according to the outline of the external hexagonal bolt;

[0010] Performing central axis alignment between the RGB camera and the target hexagonal bolt using a bisection method according to the outline of the external hexagonal bolt;

[0011] Performing hexagonal alignment calculation on the hexagonal sleeve and the target hexagonal bolt to obtain a minimum rotation angle;

[0012] The position of the hexagonal sleeve is adjusted according to the minimum rotation angle to obtain the hexagonal sleeve aligned with the target hexagonal bolt.

[0013] Preferably, the contour detection of the image to be analyzed is performed using a Hough circle transform algorithm, a Canny algorithm, or a Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt, including:

[0014] Converting the image to be analyzed into a grayscale image to obtain a grayscale image to be analyzed;

[0015] Using a Gaussian filter to smooth the grayscale image to be analyzed to obtain a noise-reduced grayscale image;

[0016] The noise reduction grayscale image is detected using the Hough circle transform algorithm to obtain the gasket center coordinates and gasket radius of the circular gasket at the target hexagonal bolt position.

[0017] Preferably, the contour detection of the image to be analyzed is performed using a Hough circle transform algorithm, a Canny algorithm, or a Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt, including:

[0018] Using the Canny algorithm to perform edge detection and extraction on the denoised grayscale image to obtain several candidate contours;

[0019] Performing polygonal approximation processing on each candidate contour using the Douglas-Peucker algorithm to obtain the coordinates of the polygon corner points corresponding to the candidate contour;

[0020] The points whose distance from the center coordinates of the gasket is between 1 / 4 of the gasket radius and 3 / 4 of the gasket radius in the polygon corner point coordinates are filtered to obtain filtered corner point coordinates.

[0021] Preferably, the contour detection of the image to be analyzed is performed using a Hough circle transform algorithm, a Canny algorithm, or a Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt, including:

[0022] Generate hexagonal coordinates using a preset coordinate recursion formula with the coordinates of each filtered corner point as a starting point to obtain several candidate hexagonal coordinate combinations;

[0023] The six coordinates of each candidate hexagonal coordinate combination are subjected to error calculation and summation according to the filtered corner point coordinates to obtain a total hexagonal coordinate error, and the regular hexagon formed by the candidate hexagonal coordinate combination with the smallest total hexagonal coordinate error is determined as the outer hexagon bolt profile.

[0024] Preferably, performing planar alignment on the RGB camera and the plane where the target hexagonal bolt is located by using a dichotomy method according to the outline of the external hexagonal bolt comprises:

[0025] Calculating the Laplace operator variance value of each vertex of the external hexagonal bolt profile to obtain six sharpness variance values;

[0026] When the difference between the maximum and minimum values ​​of the sharpness variance exceeds a first tolerance threshold, the RGB cameras are controlled to perform plane alignment using a dichotomy method according to the angle adjustment parameter until the difference between the maximum and minimum values ​​of the sharpness variance is less than the first tolerance threshold, at which time the adjustment of the RGB cameras is stopped.

[0027] Preferably, performing central axis alignment on the RGB camera and the target hexagonal bolt using a dichotomy method according to the profile of the external hexagonal bolt comprises:

[0028] Re-collecting the image to be analyzed, and obtaining new coordinates of the center of the gasket and the radius of the gasket according to the image to be analyzed;

[0029] The deviations between the coordinates of the center point of the image to be analyzed and the coordinates of the center of the gasket in the horizontal and vertical directions are calculated to obtain a horizontal deviation and a vertical deviation.

[0030] Preferably, performing central axis alignment on the RGB camera and the target hexagonal bolt using a dichotomy method according to the profile of the external hexagonal bolt comprises:

[0031] When either the lateral deviation or the longitudinal deviation exceeds a second tolerance threshold, setting a lateral translation offset and a longitudinal translation offset according to the lateral deviation and the longitudinal deviation respectively;

[0032] The RGB camera and the target hexagonal bolt are controlled to align their central axes using a dichotomy method according to the lateral translation offset and the longitudinal translation offset, and the adjustment of the RGB camera is stopped when both the lateral deviation and the longitudinal deviation meet a second tolerance threshold.

[0033] Preferably, a system for automatically aligning a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a robotic arm comprises:

[0034] An image acquisition module is used to move the robotic arm to a pre-alignment position and use an RGB camera on the robotic arm to acquire an image of the pre-alignment position to obtain an image to be analyzed; the RGB camera is coaxial with the hexagonal sleeve at the end of the robotic arm;

[0035] A contour detection module is used to perform contour detection on the image to be analyzed using a Hough circle transform algorithm, a Canny algorithm, and a Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt;

[0036] a plane alignment module, configured to perform plane alignment between the RGB camera and the plane where the target hexagonal bolt is located using a dichotomy method according to the profile of the external hexagonal bolt;

[0037] a central axis alignment module, configured to perform central axis alignment between the RGB camera and the target hexagonal bolt using a dichotomy method according to the profile of the external hexagonal bolt;

[0038] An angle calculation module, used to perform hexagonal alignment calculation on the hexagonal sleeve and the target hexagonal bolt to obtain a minimum rotation angle;

[0039] The rotation adjustment module is used to adjust the position of the hexagonal sleeve according to the minimum rotation angle to obtain the hexagonal sleeve aligned with the target hexagonal bolt.

[0040] Preferably, an electronic device comprises: at least one processor, and a memory communicatively connected to the processor; wherein the memory stores instructions that can be executed by the processor, and the instructions are executed by the processor so that the processor can execute the aforementioned method of automatic alignment of a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm.

[0041] Preferably, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the aforementioned method for automatically aligning a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm.

[0042] The present invention discloses the following technical effects:

[0043] The present invention provides a method, system, electronic device and storage medium for automatic alignment of a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm. Through contour detection, plane alignment, central axis alignment and minimum rotation angle calculation, the present invention solves the problem that conventional models require a pre-training process, thereby reducing computing power and deployment costs. By using the dichotomy method for plane alignment and central axis alignment, the present invention solves the problem of low precision of conventional methods and reduces alignment errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 A schematic diagram of a process for automatically aligning a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm according to an embodiment of the present invention;

[0046] Figure 2 A flowchart of automatic alignment of a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm provided in an embodiment of the present invention;

[0047] Figure 3 Before and after comparison diagram of plane alignment provided by an embodiment of the present invention;

[0048] Figure 4 A before-and-after comparison diagram of the central axis alignment provided by an embodiment of the present invention;

[0049] Figure 5 Before and after comparison of hexagonal alignment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0051] The purpose of the present invention is to provide a method, system, electronic device and storage medium for automatic alignment of a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm, so as to solve the problems of high cost and low precision in existing methods.

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0053] Figure 1 The present invention provides a schematic diagram of the automatic alignment process of the hexagonal sleeve and the external hexagonal bolt based on the coaxial vision of the end of the robot arm, as shown in FIG. Figure 1 As shown, the present invention provides a method for automatically aligning a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a robotic arm, comprising:

[0054] Step 100: Move the robotic arm to a pre-alignment position, and use an RGB camera on the robotic arm to capture an image of the pre-alignment position to obtain an image to be analyzed; the RGB camera is coaxial with the hexagonal sleeve at the end of the robotic arm;

[0055] Step 200: Performing contour detection on the image to be analyzed using a Hough circle transform algorithm, a Canny algorithm, and a Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt;

[0056] Step 300: performing planar alignment between the RGB camera and the plane where the target hexagonal bolt is located using a bisection method according to the outline of the external hexagonal bolt;

[0057] Step 400: performing central axis alignment between the RGB camera and the target hexagonal bolt using a bisection method according to the outer hexagonal bolt profile;

[0058] Step 500: performing hexagonal alignment calculation on the hexagonal sleeve and the target hexagonal bolt to obtain a minimum rotation angle;

[0059] Step 600: Adjust the position of the hexagonal socket according to the minimum rotation angle to obtain the hexagonal socket aligned with the target hexagonal bolt.

[0060] Specifically, the Hough circle transform algorithm, the Canny algorithm, and the Douglas-Peucker algorithm are used to perform contour detection on the image to be analyzed to obtain the outer hexagonal bolt contour of the target hexagonal bolt, including:

[0061] Converting the image to be analyzed into a grayscale image to obtain a grayscale image to be analyzed;

[0062] Using a Gaussian filter to smooth the grayscale image to be analyzed to obtain a noise-reduced grayscale image;

[0063] The noise reduction grayscale image is detected using the Hough circle transform algorithm to obtain the gasket center coordinates and gasket radius of the circular gasket at the target hexagonal bolt position.

[0064] Furthermore, the image to be analyzed is subjected to contour detection using the Hough circle transform algorithm, the Canny algorithm, and the Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt, including:

[0065] Using the Canny algorithm to perform edge detection and extraction on the denoised grayscale image to obtain several candidate contours;

[0066] Performing polygonal approximation processing on each candidate contour using the Douglas-Peucker algorithm to obtain the coordinates of the polygon corner points corresponding to the candidate contour;

[0067] The points whose distance from the center coordinates of the gasket is between 1 / 4 of the gasket radius and 3 / 4 of the gasket radius in the polygon corner point coordinates are filtered to obtain filtered corner point coordinates.

[0068] Specifically, the Hough circle transform algorithm, the Canny algorithm, and the Douglas-Peucker algorithm are used to perform contour detection on the image to be analyzed to obtain the outer hexagonal bolt contour of the target hexagonal bolt, including:

[0069] Generate hexagonal coordinates using a preset coordinate recursion formula with the coordinates of each filtered corner point as a starting point to obtain several candidate hexagonal coordinate combinations;

[0070] The six coordinates of each candidate hexagonal coordinate combination are subjected to error calculation and summation according to the filtered corner point coordinates to obtain a total hexagonal coordinate error, and the regular hexagon formed by the candidate hexagonal coordinate combination with the smallest total hexagonal coordinate error is determined as the outer hexagon bolt profile.

[0071] Furthermore, the RGB camera and the plane where the target hexagonal bolt is located are aligned using a dichotomy method according to the outline of the external hexagonal bolt, including:

[0072] Calculating the Laplace operator variance value of each vertex of the external hexagonal bolt profile to obtain six sharpness variance values;

[0073] When the difference between the maximum and minimum values ​​of the sharpness variance exceeds a first tolerance threshold, the RGB cameras are controlled to perform plane alignment using a dichotomy method according to the angle adjustment parameter until the difference between the maximum and minimum values ​​of the sharpness variance is less than the first tolerance threshold, at which time the adjustment of the RGB cameras is stopped.

[0074] Specifically, performing central axis alignment on the RGB camera and the target hexagonal bolt using a dichotomy method according to the profile of the external hexagonal bolt includes:

[0075] Re-collecting the image to be analyzed, and obtaining new coordinates of the center of the gasket and the radius of the gasket according to the image to be analyzed;

[0076] The deviations between the coordinates of the center point of the image to be analyzed and the coordinates of the center of the gasket in the horizontal and vertical directions are calculated to obtain a horizontal deviation and a vertical deviation.

[0077] refer to Figure 2, performing central axis alignment between the RGB camera and the target hexagonal bolt using a bisection method according to the outer hexagonal bolt profile, including:

[0078] When either the lateral deviation or the longitudinal deviation exceeds a second tolerance threshold, setting a lateral translation offset and a longitudinal translation offset according to the lateral deviation and the longitudinal deviation respectively;

[0079] The RGB camera and the target hexagonal bolt are controlled to align their central axes using a dichotomy method according to the lateral translation offset and the longitudinal translation offset, and the adjustment of the RGB camera is stopped when both the lateral deviation and the longitudinal deviation meet a second tolerance threshold.

[0080] Specifically, the RGB camera is kept coaxial with the hexagonal sleeve at the end of the robotic arm. The robotic arm is moved to the pre-alignment position so that a clear image of the bolt appears in the camera image and the bolt is brought close to the center of the image. This completes the rough alignment process, and the fine alignment of the bolt will then be carried out.

[0081] Perform plane alignment and adjust the posture of the robotic arm so that the camera's imaging plane is parallel to the plane where the bolt head is located, that is, the camera's optical axis is perpendicular to the plane where the bolt head is located;

[0082] Perform axis alignment and adjust the robot arm's posture so that the camera's optical axis coincides with the plane normal vector passing through the center of the bolt head, that is, the bolt appears in the center of the camera image;

[0083] Perform hexagon alignment and calculate the minimum rotation angle required to align the hexagon of the hexagonal socket at the end of the robot arm with the hexagon of the external hexagonal bolt. This step only calculates the rotation direction and angle value, but does not perform posture adjustment of the robot arm.

[0084] The hand-eye calibration technique is used to calculate the transformation matrix from the camera to the hexagonal sleeve at the end of the robotic arm. Because the central axis alignment process aligns the camera optical axis with the center point of the bolt, a certain offset adjustment is required to align the hexagonal sleeve at the end of the robotic arm with the center of the bolt. This offset is the offset between the camera optical axis and the hexagonal sleeve at the end of the robotic arm, and this offset needs to be calibrated.

[0085] According to the previously calculated transformation matrix, the robot arm posture is adjusted so that the center of the sleeve is aligned with the bolt, and then it is rotated according to the previously calculated angle value to align the hexagonal sleeve with the outer hexagonal bolt, and the subsequent bolt tightening operation can be performed.

[0086] Furthermore, the external hexagonal bolt profile detection method needs to extract the profile of the external hexagonal bolt before performing bolt alignment. The method flow is as follows:

[0087] Convert the RGB image captured by the camera into a grayscale image and use a Gaussian filter to smooth the image to reduce noise;

[0088] Since hexagonal bolts are usually used with circular washers, Hough circle transform is used here to detect the center coordinates of the circular washers. and radius , where the center coordinates are the center coordinates of the bolt head;

[0089] Use the Canny algorithm to perform edge detection on the processed image, and then perform contour extraction on the edge results;

[0090] Traverse all contours, use the Douglas-Peucker algorithm to perform polygonal approximation on each contour, and calculate the coordinates of the corner points of the polygon corresponding to each contour;

[0091] Since the distance from a point on the bolt head contour to the center point of the bolt head is usually arrive Based on this conclusion, the polygon corner coordinates calculated previously are filtered;

[0092] For each corner point in a contour that passes the filter , center coordinates of the bolt head According to the property of the regular hexagon that the corner point, the center of the bolt head and the adjacent corner points of the regular hexagon can form an equilateral triangle, the other five corner points of the regular hexagon can be deduced, and the corner point can be calculated. The corresponding regular hexagon data is calculated as follows:

[0093]

[0094] Then we can solve and Value:

[0095]

[0096]

[0097] in, is the center coordinate of the bolt head, are the coordinates of the first corner point, is The corner points of a regular hexagon obtained by recursion will be solved Then substitute the value of into the equation system and replace The value of can be recursively deduced, and after 4 recursions, the corner point can be obtained. The coordinates of the six points on the corresponding regular hexagon;

[0098] According to all corner points The coordinates of the six points in the corresponding regular hexagon are calculated, and the total error of these six points is calculated as the minimum distance between the coordinates of the point and the coordinates of all corner points. Finally, the six error values ​​are summed up. In this way, the regular hexagon with the smallest total error is selected, which is the outline of the external hexagon bolt.

[0099] Preferably, the plane alignment is to adjust the posture of the robot arm so that the imaging plane of the camera is parallel to the plane where the bolt head is located, that is, the optical axis of the camera is perpendicular to the plane where the bolt head is located. The specific steps are:

[0100] The above-mentioned external hexagonal bolt profile detection method is used to detect the hexagonal profile of the bolt;

[0101] When the plane of the bolt head is not parallel to the imaging plane, the sharpness of the six corners of the bolt head in the image will be unevenly distributed due to the influence of the camera's depth of field and perspective. Usually, the part closer to the camera (such as the front end of the bolt head after tilting) is imaged more clearly and sharply, while the part farther away from the camera (such as the rear end of the bolt head after tilting) has reduced sharpness. This embodiment calculates the coordinates of the six vertices based on the hexagonal outline, and based on the grayscale image, calculates the Laplace operator variance value in the local small square area centered on the six vertices. This value is used to define the image sharpness near each vertex. The larger the variance value, the clearer and the higher the sharpness, and the smaller the variance value, the blurrier and the lower the sharpness. The discrete calculation formula of the Laplace operator:

[0102] x-axis direction:

[0103]

[0104] Y-axis direction:

[0105]

[0106] Summary:

[0107]

[0108] in, Refers to pixel coordinates The pixel value at , after calculating each pixel point with the Laplace operator, the variance can be calculated;

[0109] If the difference between the maximum and minimum variance values ​​calculated for the six vertices is within a tolerance threshold, the plane alignment is completed. Usually, the alignment accuracy of this method is within ±1°, and the process ends.

[0110] Otherwise, the variance values ​​calculated based on the six vertices are combined with the angle adjustment parameters. Adjust the robot arm's posture so that the camera rotates toward the rear end of the bolt head in the picture. degrees, whenever the direction of the rotation deviation changes, After halving and adjusting the robot arm posture once, return to step "Use the above-mentioned external hexagonal bolt contour detection method to detect the hexagonal contour of the bolt" and continue execution. Figure 3 The left picture is before and after the plane alignment, and the right picture is after the plane alignment.

[0111] Specifically, the center axis alignment aims to adjust the posture of the robot arm so that the optical axis of the camera coincides with the plane normal vector of the center point of the bolt head, that is, the bolt appears in the center of the camera image. The specific steps are:

[0112] Convert the RGB image captured by the camera into a grayscale image and use a Gaussian filter to smooth the image to reduce noise;

[0113] Since hexagonal bolts are usually used with circular washers, Hough circle transform is used here to detect the center coordinates and radius of the circular washer, where the center coordinates are also the center coordinates of the bolt head.

[0114] Since the center of the camera image is usually aligned with the optical axis of the lens, if the deviation between the horizontal and vertical axes of the center coordinates of the bolt head and the center coordinates of the camera image is within a tolerance threshold, the center axis alignment is completed and the process ends. The alignment error of this method is ±0.5mm;

[0115] Otherwise, a translation offset is maintained on the horizontal and vertical axes respectively according to the deviation between the center coordinates of the hexagonal bolt head and the center coordinates of the camera image. and , adjust the robot arm to move on the horizontal and vertical axes and If the sign of the horizontal or vertical axis deviation changes, or Halve the value and make corresponding adjustments to the robotic arm. After that, return to step "Convert the RGB image taken by the camera into a grayscale image and use a Gaussian filter to smooth the image to reduce noise" and continue. Figure 4 This is a before and after comparison of the central axis alignment. The left picture is before the central axis alignment, and the right picture is after the central axis alignment.

[0116] Furthermore, the hexagon alignment is intended to calculate the minimum angle of rotation required to align the hexagon of the hexagonal sleeve at the end of the robot arm with the hexagon of the external hexagonal bolt to ensure that the sleeve at the end of the robot arm can cover the bolt. The specific steps are:

[0117] The above-mentioned external hexagonal bolt profile detection method is used to detect the hexagonal profile of the bolt;

[0118] The minimum rotation angle and direction are calculated based on the hexagonal profile of the hexagonal bolt and the hexagon of the sleeve at the end of the robotic arm, and then the end of the robotic arm is controlled to rotate accordingly. Figure 5 The left picture is before and after the hexagon alignment, and the right picture is after the hexagon alignment.

[0119] As an optional implementation, this embodiment further provides a system for automatically aligning a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm, comprising:

[0120] An image acquisition module is used to move the robotic arm to a pre-alignment position and use an RGB camera on the robotic arm to acquire an image of the pre-alignment position to obtain an image to be analyzed; the RGB camera is coaxial with the hexagonal sleeve at the end of the robotic arm;

[0121] A contour detection module is used to perform contour detection on the image to be analyzed using a Hough circle transform algorithm, a Canny algorithm, and a Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt;

[0122] a plane alignment module, configured to perform plane alignment between the RGB camera and the plane where the target hexagonal bolt is located using a dichotomy method according to the profile of the external hexagonal bolt;

[0123] a central axis alignment module, configured to perform central axis alignment between the RGB camera and the target hexagonal bolt using a dichotomy method according to the profile of the external hexagonal bolt;

[0124] An angle calculation module, used to perform hexagonal alignment calculation on the hexagonal sleeve and the target hexagonal bolt to obtain a minimum rotation angle;

[0125] The rotation adjustment module is used to adjust the position of the hexagonal sleeve according to the minimum rotation angle to obtain the hexagonal sleeve aligned with the target hexagonal bolt.

[0126] As an optional implementation, this embodiment also provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the processor; wherein the memory stores instructions that can be executed by the processor, and the instructions are executed by the processor so that the processor can execute the aforementioned method of automatic alignment of a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm.

[0127] As an optional implementation, this embodiment also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the aforementioned method for automatically aligning a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm.

[0128] The beneficial effects of the present invention are as follows:

[0129] The present invention avoids the pre-training process, reduces computing power requirements and deployment costs, and improves execution efficiency through contour detection, plane alignment, central axis alignment, and minimum rotation angle calculation; and reduces alignment errors by using the dichotomy method for plane alignment and central axis alignment.

[0130] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0131] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A method for automatically aligning a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a robotic arm, characterized in that: include: The robotic arm is moved to a pre-alignment position, and an RGB camera on the robotic arm is used to capture an image of the pre-alignment position to obtain an image to be analyzed; the RGB camera is kept coaxial with the hexagonal sleeve at the end of the robotic arm; Performing contour detection on the image to be analyzed using the Hough circle transform algorithm, the Canny algorithm, and the Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt; Performing planar alignment between the RGB camera and the plane where the target hexagonal bolt is located using a dichotomy method according to the outline of the external hexagonal bolt; Performing central axis alignment between the RGB camera and the target hexagonal bolt using a bisection method according to the outline of the external hexagonal bolt; Performing hexagonal alignment calculation on the hexagonal sleeve and the target hexagonal bolt to obtain a minimum rotation angle; The position of the hexagonal sleeve is adjusted according to the minimum rotation angle to obtain the hexagonal sleeve aligned with the target hexagonal bolt.

2. The method for automatically aligning a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a robotic arm according to claim 1, characterized in that: The image to be analyzed is subjected to contour detection using the Hough circle transform algorithm, the Canny algorithm, and the Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt, including: Converting the image to be analyzed into a grayscale image to obtain a grayscale image to be analyzed; Using a Gaussian filter to smooth the grayscale image to be analyzed to obtain a noise-reduced grayscale image; The noise reduction grayscale image is detected using the Hough circle transform algorithm to obtain the gasket center coordinates and gasket radius of the circular gasket at the target hexagonal bolt position.

3. The method for automatically aligning a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a robotic arm according to claim 2, characterized in that: The image to be analyzed is subjected to contour detection using the Hough circle transform algorithm, the Canny algorithm, and the Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt, including: Using the Canny algorithm to perform edge detection and extraction on the denoised grayscale image to obtain several candidate contours; Performing polygonal approximation processing on each candidate contour using the Douglas-Peucker algorithm to obtain the coordinates of the polygon corner points corresponding to the candidate contour; The points whose distance from the center coordinates of the gasket is between 1 / 4 of the gasket radius and 3 / 4 of the gasket radius in the polygon corner point coordinates are filtered to obtain filtered corner point coordinates.

4. The method for automatically aligning a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a robotic arm according to claim 3, characterized in that: The image to be analyzed is subjected to contour detection using the Hough circle transform algorithm, the Canny algorithm, and the Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt, including: Generate hexagonal coordinates using a preset coordinate recursion formula with the coordinates of each filtered corner point as a starting point to obtain several candidate hexagonal coordinate combinations; The six coordinates of each candidate hexagonal coordinate combination are subjected to error calculation and summation according to the filtered corner point coordinates to obtain a total hexagonal coordinate error, and the regular hexagon formed by the candidate hexagonal coordinate combination with the smallest total hexagonal coordinate error is determined as the outer hexagon bolt profile.

5. The method for automatically aligning a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a robotic arm according to claim 4, characterized in that: Performing planar alignment on the RGB camera and the plane where the target hexagonal bolt is located by using a bisection method according to the outline of the external hexagonal bolt includes: Calculating the Laplace operator variance value of each vertex of the external hexagonal bolt profile to obtain six sharpness variance values; When the difference between the maximum and minimum values ​​of the sharpness variance exceeds a first tolerance threshold, the RGB cameras are controlled to perform plane alignment using a dichotomy method according to the angle adjustment parameter until the difference between the maximum and minimum values ​​of the sharpness variance is less than the first tolerance threshold, at which time the adjustment of the RGB cameras is stopped.

6. The method for automatically aligning a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a robotic arm according to claim 5, characterized in that: Performing central axis alignment on the RGB camera and the target hexagonal bolt using a bisection method according to the outer hexagonal bolt profile includes: Re-collecting the image to be analyzed, and obtaining new coordinates of the center of the gasket and the radius of the gasket according to the image to be analyzed; The deviations between the coordinates of the center point of the image to be analyzed and the coordinates of the center of the gasket in the horizontal and vertical directions are calculated to obtain a horizontal deviation and a vertical deviation.

7. The method for automatically aligning a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a robotic arm according to claim 6, characterized in that: Performing central axis alignment on the RGB camera and the target hexagonal bolt using a bisection method according to the outer hexagonal bolt profile includes: When either the lateral deviation or the longitudinal deviation exceeds a second tolerance threshold, setting a lateral translation offset and a longitudinal translation offset according to the lateral deviation and the longitudinal deviation respectively; The RGB camera and the target hexagonal bolt are controlled to align their central axes using a dichotomy method according to the lateral translation offset and the longitudinal translation offset, and the adjustment of the RGB camera is stopped when both the lateral deviation and the longitudinal deviation meet a second tolerance threshold.

8. An automatic alignment system for hexagonal sockets and external hexagonal bolts based on coaxial vision at the end of a robotic arm, characterized in that: include: An image acquisition module is used to move the robotic arm to a pre-alignment position and use an RGB camera on the robotic arm to acquire an image of the pre-alignment position to obtain an image to be analyzed; the RGB camera is coaxial with the hexagonal sleeve at the end of the robotic arm; A contour detection module is used to perform contour detection on the image to be analyzed using a Hough circle transform algorithm, a Canny algorithm, and a Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt; a plane alignment module, configured to perform plane alignment between the RGB camera and the plane where the target hexagonal bolt is located using a dichotomy method according to the profile of the external hexagonal bolt; a central axis alignment module, configured to perform central axis alignment between the RGB camera and the target hexagonal bolt using a dichotomy method according to the profile of the external hexagonal bolt; An angle calculation module, used to perform hexagonal alignment calculation on the hexagonal sleeve and the target hexagonal bolt to obtain a minimum rotation angle; The rotation adjustment module is used to adjust the position of the hexagonal sleeve according to the minimum rotation angle to obtain the hexagonal sleeve aligned with the target hexagonal bolt.

9. An electronic device, characterized in that: include: At least one processor and a memory communicatively connected to the processor; wherein the memory stores instructions that can be executed by the processor, and the instructions are executed by the processor so that the processor can execute the method for automatically aligning a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute a method for automatically aligning a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm according to any one of claims 1 to 7.

Images