An automatic alignment method and system for a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a mechanical arm, an electronic device, and a storage medium

Abstract

The application belongs to the technical field of machine vision, and provides a hexagonal sleeve and external hexagonal bolt automatic alignment method and system based on mechanical arm end coaxial vision, an electronic device and a storage medium, which comprise image acquisition, contour detection, plane alignment, central axis alignment, minimum rotation angle calculation and rotation adjustment; the application avoids the pre-training process, reduces the algorithm requirement and deployment cost, and improves the execution efficiency through contour detection, plane alignment, central axis alignment and minimum rotation angle calculation; the alignment error is reduced through plane alignment and central axis alignment by using dichotomy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, in particular to a hexagonal sleeve and external hexagonal bolt automatic alignment method, system, electronic device and storage medium based on mechanical arm end coaxial vision. BACKGROUND

[0002] In an industrial scene, an external hexagonal bolt is a commonly used connecting component, but the external hexagonal bolt is prone to loosening due to factors such as wind force and thermal expansion and contraction of the bolt itself, thereby causing safety hazards, so it is particularly important to maintain and overhaul the bolt. In the maintenance process, to achieve automatic tightening of the hexagonal sleeve at the end of the mechanical arm, the key is to complete the accurate alignment of the hexagonal sleeve and the external hexagonal bolt, which is a prerequisite for reliable bolt tightening.

[0003] The technology with publication number CN119515783A proposes a 6D pose detection and visual servo method for a bolt based on 3D template matching, which requires establishing a CAD model of the bolt and using a depth camera, so the implementation cost of this method is relatively high. The technology with publication number CN118809153A proposes a mechanical arm force control bolt alignment method based on a rotation search strategy, and the bolt alignment error of this method is 1mm to 3mm, so the alignment accuracy is low. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present application is to provide a hexagonal sleeve and external hexagonal bolt automatic alignment method, system, electronic device and storage medium based on mechanical arm end coaxial vision, which solves the problems of high cost and low precision existing in the prior art.

[0005] To achieve the above-mentioned purpose, the present application provides the following solutions:

[0006] A hexagonal sleeve and external hexagonal bolt automatic alignment method based on mechanical arm end coaxial vision, comprising:

[0007] moving the mechanical arm to a pre-alignment position, and using an RGB camera on the mechanical arm to collect images of the pre-alignment position to obtain an image to be analyzed; the RGB camera is coaxial with a hexagonal sleeve at the end of the mechanical arm;

[0008] using a Hough circle transformation algorithm, a Canny algorithm and a Douglas-Peucker algorithm to detect the contour of the image to be analyzed to obtain an external hexagonal bolt contour of a target hexagonal bolt;

[0009] aligning the plane of the RGB camera and the target hexagonal bolt using bisection method according to the external hexagonal bolt contour;

[0010] Performing central axis alignment between the RGB camera and the target hexagonal bolt using a bisection method according to the outline of the external hexagonal bolt;

[0011] Performing hexagonal alignment calculation on the hexagonal sleeve and the target hexagonal bolt to obtain a minimum rotation angle;

[0012] The position of the hexagonal sleeve is adjusted according to the minimum rotation angle to obtain the hexagonal sleeve aligned with the target hexagonal bolt.

[0013] Preferably, the contour detection of the image to be analyzed is performed using a Hough circle transform algorithm, a Canny algorithm, or a Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt, including:

[0014] Converting the image to be analyzed into a grayscale image to obtain a grayscale image to be analyzed;

[0015] Using a Gaussian filter to smooth the grayscale image to be analyzed to obtain a noise-reduced grayscale image;

[0016] The noise reduction grayscale image is detected using the Hough circle transform algorithm to obtain the gasket center coordinates and gasket radius of the circular gasket at the target hexagonal bolt position.

[0017] Preferably, the contour detection of the image to be analyzed is performed using a Hough circle transform algorithm, a Canny algorithm, or a Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt, including:

[0018] Using the Canny algorithm to perform edge detection and extraction on the denoised grayscale image to obtain several candidate contours;

[0019] Performing polygonal approximation processing on each candidate contour using the Douglas-Peucker algorithm to obtain the coordinates of the polygon corner points corresponding to the candidate contour;

[0020] The points whose distance from the center coordinates of the gasket is between 1 / 4 of the gasket radius and 3 / 4 of the gasket radius in the polygon corner point coordinates are filtered to obtain filtered corner point coordinates.

[0021] Preferably, the contour detection of the image to be analyzed is performed using a Hough circle transform algorithm, a Canny algorithm, or a Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt, including:

[0022] Generate hexagonal coordinates using a preset coordinate recursion formula with the coordinates of each filtered corner point as a starting point to obtain several candidate hexagonal coordinate combinations;

[0023] According to the filter corner point coordinates, error calculation and summation are performed on six coordinates of each candidate hexagon coordinate combination to obtain a total error of hexagon coordinates, and a regular hexagon composed of the candidate hexagon coordinate combination with the minimum total error of hexagon coordinates is determined as the outer hexagon bolt profile.

[0024] Preferably, according to the outer hexagon bolt profile, bisection is used for plane alignment of the RGB camera and the plane where the target hexagon bolt is located, including:

[0025] A Laplacian operator variance value of each vertex of the outer hexagon bolt profile is calculated to obtain six sharpness variance values.

[0026] When the difference between the maximum value and the minimum value of the sharpness variance values exceeds a first tolerance threshold, bisection is used to control the RGB camera for plane alignment according to an angle adjustment parameter, and when the difference between the maximum value and the minimum value of the sharpness variance values is less than the first tolerance threshold, the adjustment of the RGB camera is stopped.

[0027] Preferably, according to the outer hexagon bolt profile, bisection is used for central axis alignment of the RGB camera and the target hexagon bolt, including:

[0028] The to-be-analyzed image is re-acquired, and new gasket center coordinates and gasket radii are obtained according to the to-be-analyzed image.

[0029] The deviation of the center point coordinates of the to-be-analyzed image and the gasket center coordinates in the horizontal and vertical directions is calculated to obtain a horizontal deviation and a vertical deviation.

[0030] Preferably, according to the outer hexagon bolt profile, bisection is used for central axis alignment of the RGB camera and the target hexagon bolt, including:

[0031] When either the horizontal deviation or the vertical deviation exceeds a second tolerance threshold, a horizontal translation offset and a vertical translation offset are respectively set according to the horizontal deviation and the vertical deviation.

[0032] Bisection is used to control the RGB camera and the target hexagon bolt for central axis alignment according to the horizontal translation offset and the vertical translation offset, and when the horizontal deviation and the vertical deviation both meet the second tolerance threshold, the adjustment of the RGB camera is stopped.

[0033] Preferably, a hexagon sleeve and outer hexagon bolt automatic alignment system based on mechanical arm end coaxial vision includes:

[0034] An image acquisition module is configured to move the mechanical arm to a pre-alignment position and acquire an image of the pre-alignment position by using an RGB camera on the mechanical arm, so as to obtain an image to be analyzed; the RGB camera is coaxial with a hexagonal sleeve at the end of the mechanical arm.

[0035] A contour detection module is configured to detect a contour of the image to be analyzed by using a Hough circle transformation algorithm, a Canny algorithm and a Douglas-Peucker algorithm, so as to obtain an outer hexagonal bolt contour of a target hexagonal bolt.

[0036] A plane alignment module is configured to align a plane of the RGB camera and the target hexagonal bolt by using a bisection method according to the outer hexagonal bolt contour.

[0037] A central axis alignment module is configured to align a central axis of the RGB camera and the target hexagonal bolt by using a bisection method according to the outer hexagonal bolt contour.

[0038] An angle calculation module is configured to calculate a hexagonal alignment of the hexagonal sleeve and the target hexagonal bolt, so as to obtain a minimum rotation angle.

[0039] A rotation adjustment module is configured to adjust a position of the hexagonal sleeve according to the minimum rotation angle, so as to obtain the hexagonal sleeve aligned with the target hexagonal bolt.

[0040] Preferably, an electronic device comprises at least one processor and a memory connected to the processor in communication; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor to enable the processor to execute the foregoing hexagonal sleeve and outer hexagonal bolt automatic alignment method based on coaxial vision at the end of a mechanical arm.

[0041] Preferably, a non-transitory computer readable storage medium stores computer instructions for enabling a computer to execute the foregoing hexagonal sleeve and outer hexagonal bolt automatic alignment method based on coaxial vision at the end of a mechanical arm.

[0042] The present application discloses the following technical effects:

[0043] The present application provides a hexagonal sleeve and outer hexagonal bolt automatic alignment method, system, electronic device and storage medium based on coaxial vision at the end of a mechanical arm, which solves the problem of the need for a pre-training process of a conventional model by contour detection, plane alignment, central axis alignment and minimum rotation angle calculation, reduces the computing power and deployment cost; the bisection method is used for plane alignment and central axis alignment, which solves the problem of low accuracy of conventional methods and reduces the alignment error. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 A schematic diagram of a process for automatically aligning a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm according to an embodiment of the present invention;

[0046] Figure 2 A flowchart of automatic alignment of a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm provided in an embodiment of the present invention;

[0047] Figure 3 Before and after comparison diagram of plane alignment provided by an embodiment of the present invention;

[0048] Figure 4 A before-and-after comparison diagram of the central axis alignment provided by an embodiment of the present invention;

[0049] Figure 5 Before and after comparison of hexagonal alignment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0051] The purpose of the present invention is to provide a method, system, electronic device and storage medium for automatic alignment of a hexagonal socket and an external hexagonal bolt based on coaxial vision at the end of a robotic arm, so as to solve the problems of high cost and low precision in existing methods.

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0053] Figure 1 The present invention provides a schematic diagram of the automatic alignment process of the hexagonal sleeve and the external hexagonal bolt based on the coaxial vision of the end of the robot arm, as shown in FIG. Figure 1 As shown, the present invention provides a method for automatically aligning a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a robotic arm, comprising:

[0054] Step 100: moving a mechanical arm to a pre-alignment position and collecting an image of the pre-alignment position by using an RGB camera on the mechanical arm, to obtain an image to be analyzed; the RGB camera is coaxial with a hexagonal sleeve at the end of the mechanical arm;

[0055] Step 200: performing contour detection on the image to be analyzed by using a Hough circle transformation algorithm, a Canny algorithm and a Douglas-Peucker algorithm, to obtain an outer hexagonal bolt contour of a target hexagonal bolt;

[0056] Step 300: performing plane alignment on a plane where the RGB camera and the target hexagonal bolt are located by using a bisection method according to the outer hexagonal bolt contour;

[0057] Step 400: performing central axis alignment on the RGB camera and the target hexagonal bolt by using a bisection method according to the outer hexagonal bolt contour;

[0058] Step 500: performing hexagon alignment calculation on the hexagonal sleeve and the target hexagonal bolt, to obtain a minimum rotation angle;

[0059] Step 600: adjusting the position of the hexagonal sleeve according to the minimum rotation angle, to obtain the hexagonal sleeve aligned with the target hexagonal bolt.

[0060] Specifically, the contour detection on the image to be analyzed by using the Hough circle transformation algorithm, the Canny algorithm and the Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt includes:

[0061] Converting the image to be analyzed into a gray-scale image, to obtain a gray-scale image to be analyzed;

[0062] Performing smoothing processing on the gray-scale image to be analyzed by using a Gaussian filter, to obtain a denoising gray-scale image;

[0063] Detecting the denoising gray-scale image by using the Hough circle transformation algorithm, to obtain a washer center coordinate and a washer radius of a target hexagonal bolt position washer.

[0064] Further, the contour detection on the image to be analyzed by using the Hough circle transformation algorithm, the Canny algorithm and the Douglas-Peucker algorithm to obtain the outer hexagonal bolt contour of the target hexagonal bolt includes:

[0065] Performing edge detection and extraction on the denoising gray-scale image by using the Canny algorithm, to obtain a plurality of candidate contours;

[0066] perform polygon approximation processing on each of the candidate contours by using a Douglas-Peucker algorithm to obtain polygon corner point coordinates corresponding to the candidate contours;

[0067] filter the polygon corner point coordinates to obtain filtered corner point coordinates, the filtered corner point coordinates being points in the polygon corner point coordinates that are between 1 / 4 of the gasket radius and 3 / 4 of the gasket radius from the gasket center point coordinates.

[0068] Specifically, the outer hexagonal bolt contour of the target hexagonal bolt is obtained by performing contour detection on the image to be analyzed by using a Hough circle transformation algorithm, a Canny algorithm, and a Douglas-Peucker algorithm.

[0069] Hexagonal coordinates are generated by using a preset coordinate recursion formula, with each of the filtered corner point coordinates as a starting point, to obtain a plurality of candidate hexagonal coordinate combinations.

[0070] The six coordinates of each of the candidate hexagonal coordinate combinations are subjected to error calculation and summation according to the filtered corner point coordinates to obtain a total hexagonal coordinate error, and a regular hexagon formed by the candidate hexagonal coordinate combination with the smallest total hexagonal coordinate error is determined as the outer hexagonal bolt contour.

[0071] Further, the plane alignment of the RGB camera and the plane on which the target hexagonal bolt is located is performed by using a bisection method according to the outer hexagonal bolt contour, including:

[0072] The Laplacian operator variance values of each vertex of the outer hexagonal bolt contour are calculated to obtain six sharpness variance values.

[0073] When the difference between the maximum value and the minimum value of the sharpness variance values exceeds a first tolerance threshold, the bisection method is used to control the plane alignment of the RGB camera according to an angle adjustment parameter until the difference between the maximum value and the minimum value of the sharpness variance values is less than the first tolerance threshold, and the adjustment of the RGB camera is stopped.

[0074] Specifically, the bisection method is used to perform the central axis alignment of the RGB camera and the target hexagonal bolt according to the outer hexagonal bolt contour, including:

[0075] The image to be analyzed is reacquired, and new gasket center point coordinates and a gasket radius are obtained from the image to be analyzed.

[0076] The deviations of the center point coordinates of the image to be analyzed and the gasket center point coordinates in the horizontal and vertical directions are calculated to obtain a horizontal deviation and a vertical deviation.

[0077] Reference Figure 2, the RGB camera and the target hexagonal bolt are aligned by bisection according to the outer hexagonal bolt profile, including:

[0078] when any one of the lateral deviation and the longitudinal deviation exceeds a second tolerance threshold, a lateral translation offset and a longitudinal translation offset are set according to the lateral deviation and the longitudinal deviation, respectively;

[0079] the RGB camera and the target hexagonal bolt are controlled by bisection according to the lateral translation offset and the longitudinal translation offset to be aligned, until the lateral deviation and the longitudinal deviation both meet the second tolerance threshold, and the adjustment of the RGB camera is stopped.

[0080] Specifically, the RGB camera is coaxial with the hexagonal sleeve at the end of the mechanical arm, the mechanical arm is moved to a pre-alignment position, a clear bolt image is presented in the camera image, and the bolt is close to the image center, the coarse alignment is completed, and then the fine alignment process of the bolt is performed;

[0081] plane alignment is performed, the pose of the mechanical arm is adjusted so that the imaging plane of the camera is parallel to the plane where the bolt head is located, that is, the optical axis of the camera is perpendicular to the plane where the bolt head is located;

[0082] the central axis alignment is performed, the pose of the mechanical arm is adjusted so that the optical axis of the camera coincides with the normal vector of the plane passing through the center point of the bolt head, that is, the bolt appears in the center of the camera image;

[0083] hexagon alignment is performed, the minimum angle of rotation required for the hexagon of the hexagonal sleeve at the end of the mechanical arm to align with the hexagon of the outer hexagonal bolt is calculated, this step only needs to calculate the rotation direction and angle value, but does not perform the pose adjustment of the mechanical arm;

[0084] The conversion matrix from the camera to the hexagonal sleeve at the end of the mechanical arm is calculated by hand-eye calibration technology; because the central axis alignment process realizes the alignment of the camera optical axis and the bolt center point, if the hexagonal sleeve at the end of the mechanical arm is to be aligned with the bolt center, a certain offset adjustment is required, which is the offset between the camera optical axis and the hexagonal sleeve at the end of the mechanical arm, and the offset needs to be calibrated;

[0085] The pose of the mechanical arm is adjusted according to the conversion matrix calculated in the foregoing, so that the center of the sleeve aligns with the bolt, and then the rotation is performed according to the angle value calculated in the foregoing, so that the hexagonal sleeve aligns with the outer hexagonal bolt, and the subsequent bolt tightening operation can be performed.

[0086] Further, the outer hexagonal bolt profile detection method needs to extract the profile of the outer hexagonal bolt before performing the bolt alignment, and the method flow is as follows:

[0087] Convert the RGB image captured by the camera into a grayscale image and smooth the image using a Gaussian filter to reduce noise;

[0088] Since the outer hexagonal bolt is usually matched with a circular washer, a Hough circle transform is used to detect the center coordinates of the circular washer and the radius , where the center coordinates are the coordinates of the center of the bolt head;

[0089] The Canny algorithm is used to detect the edges of the processed image, and the edge results are extracted;

[0090] All contours are traversed, and the Douglas-Peucker algorithm is used to approximate each contour to a polygon, and the coordinates of the polygon corresponding to each contour are calculated;

[0091] Since the distance from the points on the contour of the bolt head to the center point of the bolt head is usually between and , the polygon vertex coordinates calculated above are filtered according to this conclusion;

[0092] For each vertex in a contour that passes the screening , according to the property of the regular hexagon that the vertex, the center of the bolt head, and the adjacent vertices of the regular hexagon can form an equilateral triangle, the other five vertices of the regular hexagon can be recursively obtained, i.e., the vertex corresponding to the regular hexagon data can be calculated, and the calculation method is as follows:

[0093]

[0094] Then the values of and can be solved:

[0095]

[0096]

[0097] where is the center coordinate of the bolt head, is the coordinate of the first vertex, is a vertex of a regular hexagon recursively obtained from , and the value of solved is substituted into the equation set to replace the value of , and the recursion is performed again. After four recursive iterations, the coordinates of the six vertices of the regular hexagon corresponding to the vertex are obtained;

[0098] According to all the corner points The coordinates of the six points in the calculated corresponding regular hexagon are calculated, the total error of the six points is calculated, the error is calculated in the form of the minimum distance between the coordinates of the point and all the corner points, and finally the six error values are summed. In this way, the regular hexagon with the smallest total error is selected, that is, the contour of the external hexagonal bolt.

[0099] Preferably, the plane alignment aims to adjust the pose of the mechanical arm, so that the imaging plane of the camera is parallel to the plane where the bolt head is located, that is, the optical axis of the camera is perpendicular to the plane where the bolt head is located. The specific steps are:

[0100] The above-mentioned external hexagonal bolt contour detection method is used to detect the hexagonal contour of the bolt;

[0101] When the plane of the bolt head is not parallel to the imaging plane, due to the influence of the camera depth of field and perspective, the sharpness of the six corners of the bolt head in the image will present uneven distribution. Generally, the part closer to the camera (such as the front end of the inclined bolt head) is imaged more clearly and sharply, while the part farther away from the camera (such as the rear end of the inclined bolt head) has lower sharpness. In this embodiment, the coordinates of the six vertices of the hexagonal contour are calculated, and on the basis of the gray-scale image, the Laplacian operator variance value in the local small square region centered on the six vertices is calculated respectively. This value is used to define the image sharpness near each vertex. The larger the variance value, the clearer and sharper it is. The smaller the variance value, the more blurred and lower the sharpness. The discrete calculation formula of the Laplacian operator is:

[0102] x-axis direction:

[0103]

[0104] y-axis direction:

[0105]

[0106] Summary:

[0107]

[0108] wherein, refers to the pixel value at the pixel coordinates After calculating the variance for each pixel point using the Laplacian operator;

[0109] If the difference between the maximum value and the minimum value of the variance values calculated for the six vertices is within a tolerance threshold, it means that the plane alignment has been completed. The alignment accuracy of this method is usually ±1°, and the process ends;

[0110] Otherwise, according to the variance values calculated for the six vertices, and in combination with the angle adjustment parameter Adjust the robot arm's posture so that the camera rotates toward the rear end of the bolt head in the picture. degrees, whenever the direction of the rotation deviation changes, After halving and adjusting the robot arm posture once, return to step "Use the above-mentioned external hexagonal bolt contour detection method to detect the hexagonal contour of the bolt" and continue execution. Figure 3 The left picture is before and after the plane alignment, and the right picture is after the plane alignment.

[0111] Specifically, the center axis alignment aims to adjust the posture of the robot arm so that the optical axis of the camera coincides with the plane normal vector of the center point of the bolt head, that is, the bolt appears in the center of the camera image. The specific steps are:

[0112] Convert the RGB image captured by the camera into a grayscale image and use a Gaussian filter to smooth the image to reduce noise;

[0113] Since hexagonal bolts are usually used with circular washers, Hough circle transform is used here to detect the center coordinates and radius of the circular washer, where the center coordinates are also the center coordinates of the bolt head.

[0114] Since the center of the camera image is usually aligned with the optical axis of the lens, if the deviation between the horizontal and vertical axes of the center coordinates of the bolt head and the center coordinates of the camera image is within a tolerance threshold, the center axis alignment is completed and the process ends. The alignment error of this method is ±0.5mm;

[0115] Otherwise, a translation offset is maintained on the horizontal and vertical axes respectively according to the deviation between the center coordinates of the hexagonal bolt head and the center coordinates of the camera image. and , adjust the robot arm to move on the horizontal and vertical axes and If the sign of the horizontal or vertical axis deviation changes, or Halve the value and make corresponding adjustments to the robotic arm. After that, return to step "Convert the RGB image taken by the camera into a grayscale image and use a Gaussian filter to smooth the image to reduce noise" and continue. Figure 4 This is a before and after comparison of the central axis alignment. The left picture is before the central axis alignment, and the right picture is after the central axis alignment.

[0116] Furthermore, the hexagon alignment is intended to calculate the minimum angle of rotation required to align the hexagon of the hexagonal sleeve at the end of the robot arm with the hexagon of the external hexagonal bolt to ensure that the sleeve at the end of the robot arm can cover the bolt. The specific steps are:

[0117] The above-mentioned external hexagonal bolt profile detection method is used to detect the hexagonal profile of the bolt;

[0118] According to the hexagonal contour of the external hexagonal bolt and the hexagonal calculation of the mechanical arm end sleeve, the minimum rotation angle value and the rotation direction are calculated, and then the mechanical arm end is controlled to rotate accordingly. Figure 5 The left picture is before the hexagonal alignment, and the right picture is after the hexagonal alignment.

[0119] As an optional implementation, the embodiment also provides a hexagonal sleeve and external hexagonal bolt automatic alignment system based on mechanical arm end coaxial vision, comprising:

[0120] An image acquisition module is configured to move the mechanical arm to a pre-alignment position, and acquire images of the pre-alignment position by using an RGB camera on the mechanical arm, so as to obtain a to-be-analyzed image; the RGB camera is coaxial with a hexagonal sleeve at the end of the mechanical arm;

[0121] A contour detection module is configured to detect a contour of the to-be-analyzed image by using a Hough circle transformation algorithm, a Canny algorithm and a Douglas-Peucker algorithm, so as to obtain an external hexagonal bolt contour of a target hexagonal bolt;

[0122] A plane alignment module is configured to align a plane on which the RGB camera and the target hexagonal bolt are located by using a dichotomy method according to the external hexagonal bolt contour;

[0123] A central axis alignment module is configured to align a central axis of the RGB camera and the target hexagonal bolt by using a dichotomy method according to the external hexagonal bolt contour;

[0124] An angle calculation module is configured to calculate a hexagonal alignment of the hexagonal sleeve and the target hexagonal bolt, so as to obtain a minimum rotation angle;

[0125] A rotation adjustment module is configured to adjust a position of the hexagonal sleeve according to the minimum rotation angle, so as to obtain the hexagonal sleeve aligned with the target hexagonal bolt.

[0126] As an optional implementation, the embodiment also provides an electronic device, comprising at least one processor and a memory connected with the processor in communication; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor to enable the processor to execute the aforementioned hexagonal sleeve and external hexagonal bolt automatic alignment method based on mechanical arm end coaxial vision.

[0127] As an optional implementation, the embodiment also provides a non-transient computer readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the aforementioned hexagonal sleeve and external hexagonal bolt automatic alignment method based on mechanical arm end coaxial vision.

[0128] The beneficial effects of the present application are as follows:

[0129] The present application avoids the pre-training process, reduces the algorithm requirement and deployment cost, and improves the execution efficiency through contour detection, plane alignment, central axis alignment and minimum rotation angle calculation. The alignment error is reduced by using the bisection method for plane alignment and central axis alignment.

[0130] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be mutually referred to.

[0131] The principles and implementation modes of the present application are described by applying specific examples herein. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation modes and application ranges can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for automatically aligning a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a robotic arm, characterized in that: The method comprises the following steps: moving a mechanical arm to a pre-alignment position and collecting an image of the pre-alignment position by using an RGB camera on the mechanical arm, to obtain an image to be analyzed; the RGB camera is coaxial with a hexagonal sleeve at the end of the mechanical arm; performing contour detection on the image to be analyzed by using a Hough circle transformation algorithm, a Canny algorithm and a Douglas-Peucker algorithm, to obtain an outer hexagonal bolt contour of a target hexagonal bolt; aligning a plane of the RGB camera and the target hexagonal bolt by using a dichotomy method according to the outer hexagonal bolt contour; aligning a center axis of the RGB camera and the target hexagonal bolt by using a dichotomy method according to the outer hexagonal bolt contour; performing hexagon alignment calculation on the hexagonal sleeve and the target hexagonal bolt, to obtain a minimum rotation angle; adjusting the position of the hexagonal sleeve according to the minimum rotation angle, to obtain the hexagonal sleeve aligned with the target hexagonal bolt; aligning a plane of the RGB camera and the target hexagonal bolt by using a dichotomy method according to the outer hexagonal bolt contour, comprising: calculating Laplacian operator variance values of each vertex of the outer hexagonal bolt contour, to obtain six sharpness variance values; when the difference between the maximum value and the minimum value of the sharpness variance values exceeds a first tolerance threshold, controlling the RGB camera to perform plane alignment by using a dichotomy method according to an angle adjustment parameter, until the difference between the maximum value and the minimum value of the sharpness variance values is less than the first tolerance threshold, and then stopping adjusting the RGB camera; aligning a center axis of the RGB camera and the target hexagonal bolt by using a dichotomy method according to the outer hexagonal bolt contour, comprising: recollecting the image to be analyzed, and obtaining a new gasket center coordinate and a gasket radius according to the image to be analyzed; calculating deviations of a center point coordinate of the image to be analyzed and the gasket center coordinate in horizontal and vertical directions, to obtain a horizontal deviation and a vertical deviation; when either the horizontal deviation or the vertical deviation exceeds a second tolerance threshold, setting a horizontal translation offset and a vertical translation offset according to the horizontal deviation and the vertical deviation, respectively; controlling the RGB camera and the target hexagonal bolt to perform center axis alignment by using a dichotomy method according to the horizontal translation offset and the vertical translation offset, until the horizontal deviation and the vertical deviation both meet the second tolerance threshold, and then stopping adjusting the RGB camera.

2. The method of claim 1, wherein, performing contour detection on the image to be analyzed by using a Hough circle transformation algorithm, a Canny algorithm and a Douglas-Peucker algorithm, to obtain an outer hexagonal bolt contour of a target hexagonal bolt, comprising: converting the image to be analyzed into a grayscale image, to obtain a grayscale image to be analyzed; performing smoothing processing on the grayscale image to be analyzed by using a Gaussian filter, to obtain a denoising grayscale image; detecting the denoising grayscale image by using a Hough circle transformation algorithm, to obtain the gasket center coordinate and the gasket radius of the circular gasket of the target hexagonal bolt.

3. The method for automatically aligning a hexagonal sleeve and an external hexagonal bolt based on coaxial vision at the end of a robotic arm according to claim 2, characterized in that: The Hough circle transformation algorithm, the Canny algorithm, and the Douglas-Peucker algorithm are used for contour detection on the image to be analyzed, so as to obtain an outer hexagonal bolt contour of a target hexagonal bolt, including: The Canny algorithm is used for edge detection and extraction on the denoising grayscale image, so as to obtain a plurality of candidate contours. The Douglas-Peucker algorithm is used for polygon approximation processing on each candidate contour, so as to obtain polygon vertex coordinates corresponding to the candidate contour. The polygon vertex coordinates that are between 1 / 4 and 3 / 4 of the washer radius from the washer center coordinates are screened, so as to obtain filtered vertex coordinates.

4. The method of claim 3, wherein, The Hough circle transformation algorithm, the Canny algorithm, and the Douglas-Peucker algorithm are used for contour detection on the image to be analyzed, so as to obtain an outer hexagonal bolt contour of a target hexagonal bolt, including: A preset coordinate recursion formula is used to generate hexagonal coordinates from each filtered vertex coordinate, so as to obtain a plurality of candidate hexagonal coordinate combinations. Error calculation and summation are performed on six coordinates of each candidate hexagonal coordinate combination according to the filtered vertex coordinates, so as to obtain a total hexagonal coordinate error, and a regular hexagon composed of the candidate hexagonal coordinate combination with the smallest total hexagonal coordinate error is determined as the outer hexagonal bolt contour.

5. A hex socket and hex head bolt automatic alignment system based on end of arm coaxial vision, characterized in that, A hexagonal sleeve and outer hexagonal bolt automatic alignment method based on coaxial vision at the end of a mechanical arm is used to implement claim 1, and the system includes: An image acquisition module is configured to move a mechanical arm to a pre-alignment position, and acquire an image of the pre-alignment position by using an RGB camera on the mechanical arm, so as to obtain an image to be analyzed; the RGB camera is coaxial with a hexagonal sleeve at the end of the mechanical arm; A contour detection module is configured to use the Hough circle transformation algorithm, the Canny algorithm, and the Douglas-Peucker algorithm to perform contour detection on the image to be analyzed, so as to obtain an outer hexagonal bolt contour of a target hexagonal bolt; A plane alignment module is configured to use a bisection method to perform plane alignment on a plane where the RGB camera and the target hexagonal bolt are located according to the outer hexagonal bolt contour; A central axis alignment module is configured to use a bisection method to perform central axis alignment on the RGB camera and the target hexagonal bolt according to the outer hexagonal bolt contour; An angle calculation module is configured to perform hexagonal alignment calculation on the hexagonal sleeve and the target hexagonal bolt, so as to obtain a minimum rotation angle; A rotation adjustment module is configured to adjust the position of the hexagonal sleeve according to the minimum rotation angle, so as to obtain the hexagonal sleeve aligned with the target hexagonal bolt.

6. An electronic device, comprising: It includes: At least one processor and a memory connected to the processor in communication; wherein the memory stores instructions executable by the processor, the instructions executed by the processor to enable the processor to perform any one of claims 1 to 4, a hexagonal sleeve and outer hexagonal bolt automatic alignment method based on coaxial vision at the end of a mechanical arm.

7. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to make the computer execute the method for automatically aligning the hexagonal sleeve with the external hexagonal bolt based on the coaxial vision of the end of the mechanical arm according to any one of claims 1 to 4.

Images