A robot fixed pose grasping method based on ArUco pseudo QR code

Abstract

The present invention provides a robot fixed-pose grasping method based on ArUco pseudo-QR code, which relates to the fields of computer vision and image processing, and solves the limitations of ArUco pseudo-QR code applications, such as illumination sensitivity, corner misalignment, and pose calculation errors. The method includes: obtaining an input image and converting it into a grayscale image, calculating a threshold set and then binarizing it; extracting candidate corner points from the binary image, selecting and purifying the candidate corner points based on the prior information of the ArUco pseudo-QR code, and determining the target corner points and their corner point directions; further determining the information area, matching it with the marking template, calculating the pixel 3D coordinates after a successful match, and fitting them to the plane equation corresponding to the information area; generating 6DOF pose data for use by the robot based on the corner point directions and the plane equation. The present invention effectively improves the accuracy of generating the robot's 6-DOF pose, enabling the robot to accurately grasp the target object.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and image processing, and is applied to grasping robots. In particular, it relates to a robot fixed-pose grasping method based on ArUco pseudo-two-dimensional code. Background Art

[0002] As an important branch of robotic perception, visual positioning technology based on artificial markers plays a key role in scenarios such as industrial automation, intelligent warehousing, and service robotics. ArUco pseudo-QR codes, a typical coding and marking system, combine binary coded information with geometric patterns to create a composite identification system that combines information carrying and spatial positioning capabilities. Its operating principle is based on feature point detection and perspective projection models in computer vision. By extracting the coordinates of the QR code corners and establishing a correspondence with a preset template, it uses the PnP (Perspective-n-Point) algorithm combined with camera intrinsic parameters to calculate the target's three-dimensional pose relative to the camera. Due to its convenient deployment and high computational efficiency, this technology is widely used in robot calibration, scene mapping, mobile navigation, and other fields. It has become a mainstream positioning solution in fixed-pose grasping scenarios, where the robotic arm end effector must approach the target in a specific spatial pose, typically with pose error controlled within millimeter-level and angle-graded accuracy.

[0003] However, existing ArUco pseudo-QR code pose estimation technology has significant limitations in practical industrial applications. Traditional corner detection algorithms are highly sensitive to changes in image perspective. When the angle between the QR code plane and the camera's optical axis exceeds 30 degrees, perspective distortion increases the deviation in corner coordinate extraction, and in extreme cases, even corner mismatches may occur. Experimental data shows that at a pitch angle of 45 degrees, the average offset of corner detection can reach 3 to 5 pixels, directly increasing the translation error in pose estimation to several times the baseline value.

[0004] A more severe challenge comes from the dynamic interference of ambient light. Existing algorithms rely on binarization processing with a fixed threshold. In low-light environments, the pseudo-QR code boundary is easily blurred due to image noise, while pseudo-edge detection will occur under strong reflective conditions. Tests show that when the ambient illumination is lower than 100 lux, the success rate of mark detection drops to 72%, and the false detection rate on highly reflective materials such as metal surfaces is as high as 40%. This type of optical interference not only affects positioning accuracy, but also causes the system to trigger abnormal protection mechanisms and interrupt the operation process, seriously restricting the universal application of the technology. In addition, local occlusion problems that are common in industrial sites (such as oil adhesion, mechanical parts occlusion, etc.) will destroy the geometric integrity of the pseudo-QR code. Traditional algorithms lack an effective feature compensation mechanism for this, further exacerbating the instability of posture solution.

[0005] Existing technologies also overly rely on idealized visual models and fail to effectively integrate multi-dimensional physical constraints and robustness enhancement mechanisms. These technical shortcomings lead to high positioning drift, poor system robustness, and weak environmental adaptability when faced with high-precision, strongly constrained fixed-pose grasping tasks. This severely restricts the demand for advanced precision grasping technology in the field of intelligent robotics.

[0006] In summary, how to achieve highly robust pose solution through innovative breakthroughs at the algorithm level has become a technical challenge that needs to be overcome urgently in the field of industrial robot visual positioning. Summary of the Invention

[0007] Based on the current status of the background technology, the purpose of the present invention is to solve the limitations of ArUco pseudo-QR codes in actual industrial applications, such as illumination sensitivity, corner misalignment, and pose calculation errors. Therefore, a robot fixed-pose grasping method based on ArUco pseudo-QR codes is proposed. The present invention solves the above problems by using image statistical prior information, multi-image thresholds, and multi-view template matching. At the same time, it combines prior information such as the size, angle, and plane fitting of the ArUco pseudo-QR code to purify the corners, thereby improving the accuracy of the robot's 6-DOF pose generation, enabling the robot to accurately grasp the target object.

[0008] The present invention adopts the following technical solutions to achieve the purpose:

[0009] A robot fixed-pose grasping method based on ArUco pseudo-QR code, the method comprising the following steps:

[0010] S1. Obtain an input image containing an ArUco pseudo-QR code, convert the input image into a corresponding grayscale image, and calculate a corresponding threshold set based on the grayscale image;

[0011] S2. Binarize the grayscale image using the threshold value set to obtain a binary image corresponding to the grayscale image, and extract candidate corner points corresponding to the ArUco pseudo-QR code in the binary image;

[0012] S3. Select and purify the extracted candidate corner points based on the input image and the prior information of the ArUco pseudo-QR code, and determine the target corner point and its corresponding corner point direction;

[0013] S4. Determine the information area of ​​the ArUco pseudo-QR code based on the target corner point, and match the information area with a preset marking template;

[0014] S5. After the matching is successful, the 3D coordinates of the pixels in the information area are calculated, and the plane equation corresponding to the information area is obtained by fitting based on the 3D coordinates of the pixels;

[0015] S6. Generate 6DOF pose data based on the corner point direction and the plane equation. The robot grasps the target object provided with the ArUco pseudo-QR code based on the 6DOF pose data.

[0016] Specifically, in step S1, the input image is an RGB image; after converting the RGB image into the grayscale image, the average grayscale and median grayscale of the grayscale image are calculated, and then the corresponding threshold value set is obtained.

[0017] Furthermore, after calculating the average grayscale and the median grayscale, a starting threshold is first determined based on a preset reference value L. Subsequently, a threshold step size is calculated based on the brightness deviation and the grayscale dynamic range. The grayscale level corresponding to the starting threshold is used as the first grayscale threshold in the threshold set. Subsequently, the remaining grayscale thresholds are determined based on the first grayscale threshold according to the threshold step size until the maximum grayscale level of 255 is reached. The resulting multiple grayscale thresholds together constitute the threshold set.

[0018] Specifically, in step S2, the threshold set has multiple grayscale thresholds, and the grayscale image is binarized under the action of the multiple grayscale thresholds to obtain multiple binary images; an edge extraction operation is performed on each of the binary images, and the extracted edge polyline segments are fitted using the RDP algorithm; if the fitted polygon is not a quadrilateral, the polygon is discarded; if the fitted polygon is a quadrilateral, the four corner points of the quadrilateral are used as candidate corner points of the ArUco pseudo-QR code.

[0019] Specifically, in step S3, the prior information includes the side length information and angle information of the ArUco pseudo-QR code; after obtaining the prior information in advance, for each quadrilateral fitted in the binary image and the candidate corner points extracted therefrom, determine in turn whether the angle of each candidate corner point matches the angle information, and during matching, further determine whether at least one of the two side lengths corresponding to the candidate corner point matches the side length information; from multiple candidate corner points in the multiple binary images that successfully match both the angle information and the side length information, select the optimal candidate corner point as the target corner point, and the direction corresponding to at least one side length of the target corner point that successfully matches is the corner point direction.

[0020] Specifically, in step S4, a perspective transformation matrix is ​​used to transform the information area of ​​the ArUco pseudo-QR code into a bit mask image with a preset shape; the bit mask image obtained after the transformation is rotated and matched with a preset marking template in multiple directions. The marking template includes multiple preset correct ArUco pseudo-QR codes. When the match is successful, it means that the ID of the ArUco pseudo-QR code corresponding to the bit mask image is correct, and the ArUco pseudo-QR code is successfully identified.

[0021] Specifically, in step S5, the depth map information corresponding to the input image is obtained in advance, and the 3D coordinates of the pixels in the information area are calculated based on the depth map information to obtain a 3D coordinate set; based on the 3D coordinate set, the plane equation corresponding to the information area is fitted.

[0022] Specifically, in step S6, a corresponding normal vector n is obtained based on the plane equation; a first direction vector X is determined based on the vector corresponding to the corner point direction and the direction rotation during information region matching, and the normal vector n is recorded as a second direction vector Z; based on the first direction vector X and the second direction vector Z, a third direction vector Y is calculated using the right-hand rule;

[0023] Since the plane equation represents the spatial plane where the ArUco pseudo-QR code information area is located after successful matching and identification, the three-dimensional direction vector corresponding to the ArUco pseudo-QR code is formed according to the first direction vector X, the second direction vector Z and the third direction vector Y corresponding to the spatial plane, representing the fixed spatial position of the target object; the robot generates 6DOF pose data based on the preset relative position relationship with the three-dimensional direction vector, and grasps the target object based on the 6DOF pose data.

[0024] In summary, due to the adoption of this technical solution, the beneficial effects of the present invention are as follows:

[0025] By integrating image statistical prior information with multi-image threshold processing, this method effectively overcomes the technical deficiency of traditional ArUco pseudo-QR code detection, which is sensitive to lighting conditions. This significantly improves the algorithm's robustness in complex industrial scenarios and reduces the incidence of missed detections and false detections due to uneven lighting. A corner point purification strategy that utilizes multi-dimensional prior information, combined with characteristic parameters such as the QR code's physical size, spatial angle, and plane fitting, achieves precise correction and optimization of detected corner points, fundamentally resolving the corner point positioning offset problem that exists in traditional methods.

[0026] The pose calculation and verification system constructed by this invention can effectively improve the accuracy of the robot's 6-DOF pose solution, reducing the plane fitting error of the ArUco pseudo-QR code to the sub-pixel level, ensuring that the robot's end-effector can accurately locate and complete the grasping operation of the target object. Through its composite technical approach, this invention enhances the system's adaptability in dynamic industrial environments. It retains the inherent advantage of ArUco pseudo-QR code rapid recognition and breaks through the original technical bottleneck through innovative algorithm integration, thus providing a high-precision and high-reliability solution for visual positioning applications in the field of intelligent robot control. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of the overall process of the method of the present invention;

[0028] Figure 2 This is a schematic diagram of the effect of fitting a quadrilateral and extracting candidate corner points in the method of the present invention;

[0029] Figure 3 This is a schematic diagram of the effect of finally obtaining the three-dimensional direction vector in the method of the present invention. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0031] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0032] Example

[0033] A robot fixed pose grasping method based on ArUco pseudo QR code. The overall process of this method can be found in Figure 1 In this embodiment, the method is first summarized as follows:

[0034] S1. Obtain an input image containing an ArUco pseudo-QR code, convert the input image into a corresponding grayscale image, and calculate a corresponding threshold set based on the grayscale image;

[0035] S2. Binarize the grayscale image using the threshold value set to obtain a binary image corresponding to the grayscale image, and extract candidate corner points corresponding to the ArUco pseudo-QR code in the binary image;

[0036] S3. Select and purify the extracted candidate corner points based on the input image and the prior information of the ArUco pseudo-QR code, and determine the target corner point and its corresponding corner point direction;

[0037] S4. Determine the information area of ​​the ArUco pseudo-QR code based on the target corner point, and match the information area with a preset marking template;

[0038] S5. After the matching is successful, the 3D coordinates of the pixels in the information area are calculated, and the plane equation corresponding to the information area is fitted based on the 3D coordinates of the pixels;

[0039] S6. Generate 6DOF pose data based on the corner point direction and the plane equation. The robot grasps the target object provided with the ArUco pseudo-QR code based on the 6DOF pose data.

[0040] This embodiment will introduce the details of each step in detail according to the above step sequence.

[0041] In step S1, the input image is an RGB image. After converting the RGB image into a grayscale image, the average grayscale of the grayscale image is first calculated as follows:

[0042]

[0043] Where aver_gray represents the average grayscale, and imgGray[i][j] represents the grayscale value of the pixel in the i-th row and j-th column in the valid area of ​​the grayscale image. The average grayscale is obtained by summing the grayscale values ​​of all pixels in the valid area and dividing it by the total number of pixels in the valid area.

[0044] Continuing to calculate the median grayscale of the grayscale image, in this embodiment, the grayscale probability corresponding to each grayscale level in the grayscale image is first calculated as follows:

[0045]

[0046] Where gray_prob[I] represents the grayscale probability corresponding to grayscale level I, where I∈[0,255]; gray_pixel_num[I] represents the number of pixels at grayscale level I, and pixel_num represents the total number of pixels in the active area of ​​the grayscale image. After calculating the grayscale probabilities, a total of 256 probability values ​​are obtained, each corresponding to a specific grayscale level, such as the probability values ​​for grayscale levels 1, 10, and 100.

[0047] Then, based on the grayscale probability, the cumulative probability from 0 to each grayscale level is calculated, and the minimum grayscale level I with a cumulative probability exceeding 50% is taken as the median grayscale, as shown in the following formula:

[0048] mid_gray=argmin I {gray_distribution[I-1]≤0.5 <gray_distribution[I]}

[0049] Where mid_gray represents the median grayscale, argmin I represents the minimum value of all I values ​​that meet the conditions; gray_distribution[I] represents the cumulative probability from 0 to gray level I.

[0050] For example, if the probability of gray level 0 is 1%, the probability of gray level 1 is 2%, and the probability of gray level 2 is 4%, then the cumulative probability of gray level 2 is the sum of all probabilities from 0 to 2, that is, 1% + 2% + 4% = 7%. Therefore, if the cumulative probability of gray level I exceeds 50% and the cumulative probability of gray level I-1 is less than or equal to 50%, then gray level I is the median gray level to be found.

[0051] In this embodiment, after calculating the average grayscale aver_gray and the median grayscale mid_gray, the threshold set calculation can be performed. This part will focus on solving the problem of ArUco pseudo-QR code recognition errors caused by factors such as the lighting environment. First, a preset reference value L is obtained. In this embodiment, it is set to 127.5. When the average grayscale is greater than or equal to the preset reference value L, it means that the input image is bright, and the starting threshold start_thres is determined as follows:

[0052] start_thres=2×mid_gray-aver_gray

[0053] Furthermore, when the average grayscale is less than the preset reference value L, it indicates that the input image is dark, and the following starting threshold value start_thres is determined:

[0054] start_thres=2×aver_gray-mid_gray

[0055] Then, the threshold step length stride is calculated based on the brightness deviation (i.e., the difference from the preset reference value L) and the grayscale dynamic range (i.e., the difference between the average grayscale and the median grayscale), as follows:

[0056]

[0057] The grayscale level corresponding to the starting threshold is used as the first grayscale threshold in the threshold set. Then, according to the threshold step, the remaining grayscale thresholds are determined based on the first grayscale threshold until the maximum grayscale level of 255 is reached. The multiple grayscale thresholds obtained together constitute the threshold set.

[0058] In step S2, each grayscale threshold in the threshold set can binarize the grayscale image into a corresponding binary image. Therefore, after the grayscale image is binarized under the action of multiple grayscale thresholds, multiple binary images can be obtained. For different lighting conditions, this method obtains all situations in which the ArUco pseudo-QR code can be correctly extracted and identified as much as possible, making it very likely to ensure that at least one of the binary images can obtain the correct ArUco pseudo-QR code during the extraction and identification process.

[0059] This embodiment then performs an edge extraction operation on each binary image and fits the extracted edge polyline segments using the RDP algorithm (Ramer-Douglas-Peucker). In this embodiment, the shape of the ArUco pseudo-QR code is typically a square, which is a type of quadrilateral. Therefore, during the fitting process, if the fitted polygon is not a quadrilateral, the polygon is discarded; if a quadrilateral is fitted, the four corner points of the quadrilateral are used as candidate corner points of the ArUco pseudo-QR code.

[0060] In this embodiment, after quadrilateral fitting is performed on each binary image, it may contain multiple quadrilaterals. The quadrilaterals presented by different binary images may be different, which may lead to errors between the corresponding candidate corner points and the actual situation. Therefore, it is necessary to select the target corner point that is most likely to be one of the corner points of the ArUco pseudo-QR code from the multiple candidate corner points of multiple quadrilaterals in multiple binary images, and then obtain the corner point direction corresponding to the target corner point.

[0061] Therefore, in step S3, the prior information includes the side length and angle information of the ArUco pseudo-QR code. After obtaining the prior information, for each quadrilateral fitted in the binary image and its extracted candidate corner points, the angle of each candidate corner point is determined to match the angle information. During the matching process, a further determination is made as to whether at least one of the two side lengths corresponding to the candidate corner point matches the side length information. In this way, the optimal candidate corner point can be selected as the target corner point from among multiple candidate corner points that successfully match both the angle information and the side length information in multiple binary images. The direction corresponding to the at least one successfully matched side length of the target corner point is the corner point direction.

[0062] Before matching the angle information and side length information, this embodiment pre-acquires the depth map information corresponding to the input image; based on the depth map information, a 3D point cloud transformation is performed on the quadrilateral fitted in each binary image, and the angle information and side length information of each quadrilateral and its candidate corner points are matched in the transformed 3D point cloud space to achieve the selection and purification of the candidate corner points.

[0063] It should be noted here that, taking the square ArUco pseudo-QR code as an example, since different binary images are obtained based on different grayscale thresholds, the fitted quadrilateral will still have errors even after the 3D point cloud transformation. For example, the transformed quadrilateral is not a square, but a right-angled trapezoid, or none of the four corners are right angles. Moreover, the side lengths corresponding to the four candidate corner points in such a quadrilateral will also be different. Therefore, the selection and purification of candidate corner points is equivalent to finding the corner point that is closest to a right angle from multiple quadrilaterals. At the same time, we can also consider whether the entire quadrilateral is closest to a square. Then, starting from the found corner point, we check whether there is at least one side length closest to the prior side length information. If both the angle and side length match successfully, it means that the corner point and the corresponding side length are one of the sides of the ArUco pseudo-QR code. From this, we can accurately obtain the corner point and the corresponding corner point direction, that is, the target corner point, which can be used together with the remaining three candidate corner points of the quadrilateral to form the information area of ​​the subsequent ArUco pseudo-QR code.

[0064] See Figure 2 The effect is shown in the figure. For the two target objects, the pot lid and the pot handle, the robot needs to grasp them. The pot lid and the pot handle are both posted with corresponding ArUco pseudo QR codes. Figure 2 When the viewing angle is the input image viewing angle, the image processing method described above accurately marks the four corner points of the ArUco pseudo-QR code, and the center of the ArUco pseudo-QR code can be determined based on the positions of the four corner points. The corner point identified as the target corner point and its direction are used in the subsequent determination of the 3D direction vector.

[0065] After determining the information area of ​​the ArUco pseudo-QR code in the binary image based on the fitted quadrilateral and its target corners, in step S4, the perspective transformation matrix can be used to transform the information area into a bit mask image with a preset shape, which in this embodiment is a square image identical to the square ArUco pseudo-QR code. The perspective transformation matrix is ​​as follows:

[0066]

[0067] Where (x, y) represents the coordinates of the input pixel in the information area of ​​the ArUco pseudo-QR code, (x′, y′) represents the coordinates of the pixel in the bit mask image corresponding to the input pixel after transformation, ω′ represents the normalization factor, and h (…) Represents each matrix coefficient.

[0068] After the bit mask image obtained by the transformation is rotated, it is matched with a preset marking template in the up, down, left, and right directions. The marking template includes multiple preset correct ArUco pseudo-QR codes. For example, in this embodiment, the ArUco pseudo-QR code images corresponding to the pot lid and the pot handle under the straight-on condition are pre-recorded as marking templates. When the match is successful, it means that the ID of the ArUco pseudo-QR code corresponding to the bit mask image is correct, and the ArUco pseudo-QR code is successfully identified. At the same time, it is recorded whether the bit mask image has been rotated in which direction. If the match fails, it indicates that the information area may not be an ArUco pseudo-QR code, or it may not be the ArUco pseudo-QR code of the target object. It may also be that the ArUco pseudo-QR code is contaminated, blocked or damaged, and it is necessary to re-acquire images from other perspectives or take other measures.

[0069] When the matching is successful and the ArUco pseudo-QR code is correctly identified, step S5 can be entered. In step S5, the depth map information corresponding to the input image is obtained in advance, and the 3D coordinates of the pixels in the information area are calculated based on the depth map information to obtain a 3D coordinate set, as shown in the following formula:

[0070]

[0071]

[0072] In the above formulas, P represents a 3D coordinate set, depth[u,v] represents the pixel coordinates in the depth map information, which are u and v respectively; depth_scalar represents the scalar coefficient for normalizing the data in the depth map information to meters (if the unit of the depth map information is millimeters, the value of depth_scalar can be 1000; otherwise, if the unit of the depth map information is meters, the value of depth_scalar can be 1); c xand c y is the principal point coordinate in the camera intrinsic parameter, f x and f y is the X-axis and Y-axis focal length in the camera intrinsic parameters.

[0073] Then, based on the 3D coordinate set, the plane equation corresponding to the information area is fitted as follows:

[0074] ax+by+cz+d=0

[0075] Where a, b, c, and d are the coefficients of the plane equation, which are calculated after fitting the 3D coordinate set.

[0076] Finally, in step S6, based on the obtained plane equation corresponding to the information area, the corresponding normal vector n can be obtained as follows:

[0077] n=(a,b,c)

[0078] In this embodiment, the first direction vector X is determined based on the vector corresponding to the corner point direction and the direction rotation during information area matching, and the normal vector n is recorded as the second direction vector Z. In this embodiment, the normal vector n can always be recorded as facing the outside or upper side of the image during the determination process, and the first direction vector X, while ensuring its perpendicular relationship with the corner point direction, takes into account whether there is rotation when matching with the preset marker template, and can always be determined as the corresponding vector directly above when looking directly at the ArUco pseudo-QR code.

[0079] Based on the first direction vector X and the second direction vector Z, the right-hand rule is used to calculate the third direction vector Y. The third direction vector Y is the vector cross product of X and Z, as shown below:

[0080] Y=X×Z

[0081] Since the plane equation represents the spatial plane where the ArUco pseudo-QR code information area is located after successful matching and recognition, the three-dimensional direction vector corresponding to the ArUco pseudo-QR code is formed according to the first direction vector X, the second direction vector Z and the third direction vector Y corresponding to the spatial plane, which represents the fixed spatial position of the target object. After fitting the quadrilateral, its center point can be determined according to its four corner points. After the vectors X, Y and Z have a common intersection, the common intersection is marked at the center point, and the quadrilateral can be formed. Figure 3 The three-dimensional direction vector presentation effect shown corresponds to the ArUco pseudo-QR code at the lid and handle of the pot, where the green arrow represents the first direction vector X of the two, the blue arrow represents the second direction vector Z, and the red arrow represents the third direction vector Y.

[0082] Combine Figure 2 and Figure 3As shown in the diagram, it can be seen that the solution of this embodiment has a very accurate effect on the corner point extraction and three-dimensional direction vector recognition of the ArUco pseudo-QR code under low light conditions. Therefore, the robot finally grasps the target object based on the corresponding three-dimensional direction vector. Since the ArUco pseudo-QR code has the corresponding preset fixed position information, the robot also presets a standard grasping position for the target object when performing the grasping operation, and the grasping position can be determined based on the relative position relationship with the ArUco pseudo-QR code; when the three-dimensional direction vector of the ArUco pseudo-QR code is correctly identified, the robot can generate its corresponding 6DOF pose data and grasp the target object based on the 6DOF pose data.

Claims

1. A robot fixed-pose grasping method based on ArUco pseudo-QR code, characterized in that: The method comprises the following steps: S1. Obtain an input image containing an ArUco pseudo-QR code, convert the input image into a corresponding grayscale image, and calculate a corresponding threshold set based on the grayscale image; S2. Binarize the grayscale image using the threshold value set to obtain a binary image corresponding to the grayscale image, and extract candidate corner points corresponding to the ArUco pseudo-QR code in the binary image; S3. Select and purify the extracted candidate corner points based on the input image and the prior information of the ArUco pseudo-QR code, and determine the target corner point and its corresponding corner point direction; S4. Determine the information area of ​​the ArUco pseudo-QR code based on the target corner point, and match the information area with a preset marking template; S5. After the matching is successful, the 3D coordinates of the pixels in the information area are calculated, and the plane equation corresponding to the information area is fitted based on the 3D coordinates of the pixels; S6. Generate 6DOF pose data based on the corner point direction and the plane equation. The robot grasps the target object provided with the ArUco pseudo-QR code based on the 6DOF pose data.

2. The robot fixed-pose grasping method according to claim 1, characterized in that: In step S1, the input image is an RGB image; after converting the RGB image into the grayscale image, the average grayscale and median grayscale of the grayscale image are calculated, and then the corresponding threshold value set is obtained.

3. The robot fixed-pose grasping method according to claim 2, characterized in that: The calculation formula of the average grayscale is as follows: Where aver_gray represents the average grayscale, and imgGray[i][j] represents the grayscale value of the pixel in the i-th row and j-th column in the valid area of ​​the grayscale image. The average grayscale is obtained by summing the grayscale values ​​of all pixels in the valid area and dividing the sum by the total number of pixels in the valid area. In the process of calculating the median grayscale, the grayscale probability corresponding to each grayscale level in the grayscale image is first calculated as follows: Where gray_prob[I] represents the grayscale probability corresponding to grayscale level I, I∈[0,255]; gray_pixel_num[I] represents the number of pixels with grayscale level I, and pixel_num represents the total number of pixels in the valid area of ​​the grayscale image; then, based on the grayscale probability, the cumulative probability from 0 to the grayscale level corresponding to each grayscale level is calculated, and the minimum grayscale level I with a cumulative probability exceeding 50% is taken as the median grayscale, as shown in the following formula: mid_gray=argmin I {gray_distribution[I-1]≤0.5<gray_distribution[I]} Where mid_gray represents the median grayscale, argmin I represents the minimum value of all I values ​​that meet the conditions; gray_distribution[I] represents the cumulative probability from 0 to gray level I.

4. The robot fixed-pose grasping method according to claim 2, characterized in that: After calculating the average grayscale and the median grayscale, first determine the starting threshold as follows: When the average grayscale is greater than or equal to the preset reference value L, the calculation formula for the starting threshold is: start_thres=2×mid_gray-aver_gray When the average grayscale is less than the preset reference value L, the calculation formula for the starting threshold is: start_thres=2×aver_gray-mid_gray In the above two formulas, start_thres represents the starting threshold, aver_gray represents the average grayscale, and mid_gray represents the median grayscale. Subsequently, the threshold step size is calculated based on the brightness deviation and the grayscale dynamic range, as shown in the following formula: Where stride represents the threshold step size, and L represents the preset reference value; The grayscale level corresponding to the starting threshold is used as the first grayscale threshold in the threshold set. Then, according to the threshold step, the remaining grayscale thresholds are determined based on the first grayscale threshold until the maximum grayscale level of 255 is reached. The multiple grayscale thresholds obtained together constitute the threshold set.

5. The robot fixed-pose grasping method according to claim 1, characterized in that: In step S2, the threshold value set includes a plurality of grayscale thresholds, and the grayscale image is binarized under the action of the plurality of grayscale thresholds to obtain a plurality of binary images; Performing an edge extraction operation on each of the binary images, and fitting the extracted edge polyline segments using the RDP algorithm; if the polygon obtained by fitting is not a quadrilateral, discarding the polygon; If the fitted polygon is a quadrilateral, the four corner points of the quadrilateral are used as candidate corner points of the ArUco pseudo-two-dimensional code.

6. The robot fixed-pose grasping method according to claim 5, characterized in that: In step S3, the prior information includes the side length information and angle information of the ArUco pseudo-QR code; after obtaining the prior information in advance, for each quadrilateral fitted in the binary image and the candidate corner points extracted therefrom, determine in turn whether the angle of each candidate corner point matches the angle information, and during matching, further determine whether at least one of the two side lengths corresponding to the candidate corner point matches the side length information; from multiple candidate corner points in the multiple binary images that successfully match both the angle information and the side length information, select the optimal candidate corner point as the target corner point, and the direction corresponding to the at least one side length that successfully matches the target corner point is the corner point direction.

7. The robot fixed-pose grasping method according to claim 6, characterized in that: Before matching the angle information and the side length information, the depth map information corresponding to the input image is obtained in advance; based on the depth map information, a 3D point cloud transformation is performed on each quadrilateral fitted in the binary image, and the angle information and the side length information are matched for each quadrilateral and its candidate corner points in the transformed 3D point cloud space to achieve the selection and purification of candidate corner points.

8. The robot fixed-pose grasping method according to claim 1, characterized in that: In step S4, the information area of ​​the ArUco pseudo-QR code is transformed into a bit mask image with a preset shape using a perspective transformation matrix. The perspective transformation matrix is ​​as follows: Where (x, y) represents the coordinates of the input pixel in the information area of ​​the ArUco pseudo-QR code, (x′, y′) represents the coordinates of the pixel in the bit mask image corresponding to the input pixel after transformation, ω′ represents the normalization factor, and h (...) Represents each matrix coefficient; the bit mask image obtained after the transformation is rotated and matched with a preset marking template in multiple directions. The marking template includes multiple preset correct ArUco pseudo-QR codes. When the match is successful, it means that the ID of the ArUco pseudo-QR code corresponding to the bit mask image is correct, and the ArUco pseudo-QR code is successfully identified.

9. The robot fixed-pose grasping method according to claim 1, characterized in that: In step S5, the depth map information corresponding to the input image is obtained in advance, and the 3D coordinates of the pixels in the information area are calculated based on the depth map information to obtain a 3D coordinate set, as shown in the following formula: In the above formulas, P represents the 3D coordinate set, depth[u,v] represents the pixel coordinates in the depth map information, depth_scalar represents the scalar coefficient of the depth map information normalized to meters; c x and c y is the principal point coordinate in the camera intrinsic parameter, f x and f y is the X-axis and Y-axis focal length in the camera intrinsic parameters; According to the 3D coordinate set, the plane equation corresponding to the information area is fitted as follows: ax+by+cz+d=0 Where a, b, c, and d are the coefficients of the plane equation, which are calculated after fitting the 3D coordinate set.

10. The robot fixed-pose grasping method according to claim 9, characterized in that: In step S6, based on the plane equation, the corresponding normal vector n is obtained as follows: n=(a,b,c) Based on the vector corresponding to the corner point direction and the direction rotation during information region matching, the first direction vector X is determined, and the normal vector n is recorded as the second direction vector Z. Based on the first direction vector X and the second direction vector Z, the right-hand rule is used to calculate the third direction vector Y. The third direction vector Y is the vector cross product of X and Z, as shown in the following formula: Y=X×Z Since the plane equation represents the spatial plane where the ArUco pseudo-QR code information area is located after successful matching and identification, the three-dimensional direction vector corresponding to the ArUco pseudo-QR code is formed according to the first direction vector X, the second direction vector Z and the third direction vector Y corresponding to the spatial plane, representing the fixed spatial position of the target object; the robot generates 6DOF pose data based on the preset relative position relationship with the three-dimensional direction vector, and grasps the target object based on the 6DOF pose data.

Images