Robot workpiece grabbing pose adjusting method, device and equipment and storage medium

Abstract

The invention relates to a robot workpiece grabbing pose adjusting method, device and equipment and a storage medium. The method comprises the steps that pose information of a plurality of first feature points of a template workpiece, pose information of a plurality of second feature points of a to-be-grabbed workpiece and a template workpiece grabbing pose of a workpiece feeding robot are obtained; determining a first distance between the first feature points and a second distance between the second feature points; based on the first distance and the second distance, screening out multiple groups of matched first target feature points and second target feature points from the first feature points and the second feature points; based on the first pose information of the first target feature point and the second pose information of the second target feature point, the pose of the template workpiece and the pose of the to-be-grabbed workpiece are determined; and on the basis of the pose deviation between the template workpiece and the to-be-grabbed workpiece and the grabbing pose of the template workpiece, the workpiece grabbing pose of the workpiece feeding robot is adjusted. By the adoption of the method, the workpiece grabbing accuracy can be improved.

Description

Technical Field

[0001] This application relates to the field of image analysis technology, and in particular to a method, apparatus, computer device, computer-readable storage medium, and computer program product for adjusting the posture of a robot workpiece grasping. Background Technology

[0002] In automated workpiece gripping scenarios (such as automated gripping of exterior parts in the welding workshop of an automotive OEM), the gripping of related workpieces relies on high-precision racks or manual teaching, which have problems such as high cost and poor flexibility.

[0003] While using 3D vision to perform feature matching, positioning, and pose correction on workpieces for grasping can improve production efficiency, the related technologies still have the problem of low accuracy in feature matching for complex grasping scenarios (such as workpiece material deviations, positional offsets, posture tilts, or partial occlusions), resulting in low workpiece grasping accuracy. Summary of the Invention

[0004] Therefore, it is necessary to provide a robot workpiece gripping pose adjustment method, device, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy of workpiece gripping by addressing the above-mentioned technical problems.

[0005] In a first aspect, this application provides a method for adjusting the posture of a robot workpiece grasping, including:

[0006] Acquire the pose information of multiple first feature points of the template workpiece, the pose information of multiple second feature points of the workpiece to be grasped, and the template workpiece grasping pose of the workpiece loading robot.

[0007] Determine the first distance between each first feature point and the second distance between each second feature point;

[0008] Based on the first distance and the second distance, multiple sets of matching first target feature points and second target feature points are selected from multiple first feature points and multiple second feature points;

[0009] Based on the first pose information of the first target feature point and the second pose information of the second target feature point, the pose of the template workpiece and the pose of the workpiece to be grasped are determined respectively.

[0010] Based on the pose deviation between the pose of the template workpiece and the pose of the workpiece to be gripped, as well as the gripping pose of the template workpiece, the workpiece gripping pose of the workpiece loading robot is adjusted.

[0011] Secondly, this application also provides a robot workpiece gripping pose adjustment device, comprising:

[0012] The data acquisition module is used to acquire the pose information of multiple first feature points of the template workpiece, the pose information of multiple second feature points of the workpiece to be grasped, and the template workpiece grasping pose of the workpiece loading robot.

[0013] The distance determination module is used to determine the first distance between each first feature point and the second distance between each second feature point;

[0014] The feature point matching module is used to filter out multiple sets of matching first target feature points and second target feature points from multiple first feature points and multiple second feature points based on a first distance and a second distance;

[0015] The pose determination module is used to determine the pose of the template workpiece and the pose of the workpiece to be grasped based on the pose information of the first feature point and the second pose information of the second target feature point, respectively.

[0016] The pose adjustment module is used to adjust the workpiece grasping pose of the workpiece loading robot based on the pose deviation between the pose of the template workpiece and the pose of the workpiece to be grasped, as well as the grasping pose of the template workpiece.

[0017] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above embodiments of the robot workpiece grasping pose adjustment method.

[0018] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps in any of the above embodiments of the robot workpiece grasping pose adjustment method.

[0019] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the robot workpiece grasping pose adjustment method.

[0020] The aforementioned robot workpiece grasping pose adjustment method, apparatus, computer equipment, computer-readable storage medium, and computer program product first identify the first and second feature points of the template workpiece and the workpiece to be grasped, and determine the pose information of each feature point. Second, they determine the first distance between each first feature point and the second distance between each second feature point. This facilitates similarity distance search based on the first and second distances, allowing for the selection of multiple sets of matching first and second target feature points. Thus, even in complex grasping scenarios (such as workpiece material deviations, positional offsets, posture tilts, or partial occlusions), where the number of feature points on the workpiece to be grasped differs from the number of feature points on the template workpiece, the method can still achieve the desired results. It can identify feature points of a template workpiece that match the feature points of the workpiece to be gripped, improving the accuracy and adaptability of feature matching. Furthermore, based on the pose information of the matched feature points, coordinate systems are established for both the template workpiece and the workpiece to be gripped, determining their poses. Thus, in actual workpiece gripping scenarios, the workpiece gripping pose of the workpiece loading robot is adjusted based on the pose deviation between the template workpiece and the workpiece to be gripped, as well as the gripping pose of the template workpiece. In this way, by using the pose of the template workpiece as a reference to adjust the pose of the workpiece to be gripped, the accuracy and efficiency of the workpiece loading robot's pose correction are improved. Moreover, it eliminates the need to rely on high-precision material racks or manual teaching of material gripping, improving the flexibility of material gripping and loading, and reducing costs. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is an application environment diagram of a robot workpiece grasping pose adjustment method in one embodiment;

[0023] Figure 2 This is a flowchart illustrating a robot workpiece grasping pose adjustment method in one embodiment;

[0024] Figure 3 This is a flowchart illustrating the robot workpiece grasping pose adjustment method in another embodiment;

[0025] Figure 4 This is a flowchart illustrating the robot workpiece grasping pose adjustment method in yet another embodiment;

[0026] Figure 5This is a structural block diagram of a robot workpiece grasping posture adjustment device in one embodiment;

[0027] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] The robot workpiece grasping pose adjustment method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network, and server 104 communicates with the workpiece loading robot.

[0030] Specifically, the operator can upload the collected template workpiece image and the workpiece image to be grasped to the server 104 via the terminal 102. The workpiece loading robot sends the workpiece grasping pose to the server 104. The server 104 obtains the pose information of multiple first feature points in the template workpiece image, the pose information of multiple second feature points in the workpiece image to be grasped, and the template workpiece grasping pose of the workpiece loading robot. Next, it determines the first distance between each first feature point and the second distance between each second feature point. Based on the first and second distances, it selects multiple sets of matching first and second target feature points from the multiple first and second feature points. Then, based on the pose information of the first feature points and the second pose information of the second target feature points, it determines the pose of the template workpiece and the pose of the workpiece to be grasped. Finally, based on the pose deviation between the pose of the template workpiece and the pose of the workpiece to be grasped, and the template workpiece grasping pose, it adjusts the workpiece grasping pose of the workpiece loading robot.

[0031] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. The server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0032] In one exemplary embodiment, such as Figure 2 As shown, a method for adjusting the pose of a robot workpiece grasping is provided, which can be applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps (hereinafter referred to as S): S100 to S500. Wherein:

[0033] S100: Acquire the pose information of multiple first feature points of the template workpiece, the pose information of multiple second feature points of the workpiece to be grasped, and the template workpiece grasping pose of the workpiece loading robot.

[0034] The workpiece is determined based on the actual gripping scenario, and this application does not impose any limitations on it. In this embodiment, the gripping scenario of automotive exterior parts is used as an example for illustration. The workpiece may include, but is not limited to, door trim strips, bumpers, wheel arches, and side skirts.

[0035] The template workpiece can be selected from the workpieces to be gripped and gripped by the workpiece loading robot through manual teaching control. The template workpiece gripping pose of the workpiece loading robot is the pose of the robot's gripper tool when gripping the template workpiece.

[0036] The first feature point can be a natural feature such as a circular hole or an edge in the template workpiece image, and the second feature point can be a natural feature such as a circular hole or an edge in the workpiece image to be grasped. The template workpiece image and the workpiece image are acquired by the image acquisition device from the template workpiece and the workpiece to be grasped, respectively. The image acquisition device includes, but is not limited to, a binocular camera and a structured light camera.

[0037] The pose information of each first feature point includes the rotation matrix and translation vector of the first feature point, and the pose information of each second feature point includes the rotation matrix and translation vector of the first feature point.

[0038] Specifically, for both the template workpiece image and the workpiece image, identifying the edges and corners in the images can be achieved by detecting edge features in the image using an edge detection operator to obtain an edge detection map. Subsequently, a corner detection algorithm is used to identify the corners in the edge detection map, thus obtaining the corner points of the image's edges and corners. The corner detection algorithm can include the Harris corner detection algorithm, the Shi-Tomasi corner detection algorithm, and the scale-invariant feature transform algorithm, among others.

[0039] Specifically, for both the template workpiece image and the workpiece image, identifying circular holes in the image can be achieved by performing contour detection on the image, using an ellipse fitting algorithm to fit the detected contours to an ellipse, identifying the circular holes in the image, i.e., feature points, and obtaining the geometric parameters of the circular holes (such as the center point, major axis length, and minor axis length). The ellipse fitting algorithm can include, but is not limited to, the least squares method and the Hough transform.

[0040] Specifically, for the first feature point and the second feature point, the pose information of the feature point can be determined by acquiring point cloud data corresponding to the template workpiece image of the template workpiece and the workpiece image of the workpiece to be grasped, respectively. Then, the point cloud data corresponding to the circular hole is selected, and the pose of the feature point is estimated by principal component analysis and the point cloud data corresponding to the circular hole, respectively, to obtain the pose information of the feature point.

[0041] In practice, the operator can select a workpiece to be grasped as a template workpiece from a preset position in the material frame. A 3D point cloud camera scans the template workpiece and all other workpieces to be grasped in the material frame, excluding the template workpiece, to obtain images of the template workpiece, the workpieces, and their point cloud data. Subsequently, a structured light camera sends these images to a server. The template workpiece image and its point cloud data are corresponding, as are the workpiece images and their point cloud data.

[0042] After acquiring the template workpiece image and the workpiece image from the server, image preprocessing is performed on both images. Preprocessing may include grayscale conversion and filtering to reduce noise and enhance edges. Subsequently, for the template workpiece, contour detection is performed on the template workpiece image. The detected contour is fitted with an ellipse using the least squares method to identify the circular hole in the image (which may appear elliptical due to the image's viewing angle). The center point, major axis length, and minor axis length of the circular hole are obtained, and the first feature point includes the circular hole. Then, based on the center point, major axis length, and minor axis length of the circular hole, the corresponding point cloud data is selected to determine the three-dimensional coordinates of the center point of the circular hole. Finally, the pose of the circular hole is estimated using principal component analysis and the point cloud data to obtain the pose information of the circular hole in the template workpiece.

[0043] For the workpiece to be grasped, the method for obtaining the pose information of multiple second feature points of the workpiece is the same as the method for obtaining the pose information of multiple first feature points of the template workpiece described above, and will not be repeated here. For example, the material box contains 10 workpieces, one of which is selected as the template workpiece, and the remaining 9 are the workpieces to be grasped. Then, the workpiece loading robot is manually taught to grasp the template workpiece. The pose of the gripper tool of the workpiece loading robot when grasping the template workpiece is obtained from the teach pendant, and the grasping pose of the template workpiece is obtained. The grasping pose of the workpiece is then sent to the server.

[0044] S200, determine the first distance between each first feature point and the second distance between each second feature point.

[0045] Wherein, the first distance is the distance between each first feature point, and the second distance is the distance between each second feature point.

[0046] In specific implementation, following the above implementation steps, the server can determine the distance between each pair of first feature points based on the three-dimensional coordinates (i.e., the coordinates of the center point of the circular hole) in the pose information of each first feature point, thus obtaining the first distance between each first feature point. The server can then determine the distance between each pair of second feature points based on the three-dimensional coordinates (i.e., the coordinates of the center point of the circular hole) in the pose information of each second feature point, thus obtaining the second distance between each second feature point.

[0047] S300, based on the first distance and the second distance, selects multiple sets of matching first target feature points and second target feature points from multiple first feature points and multiple second feature points.

[0048] In this embodiment, considering the workpiece arrival offset, the workpiece image acquired by the image acquisition device may have positional offset, posture tilt, and partial occlusion. This can easily lead to a discrepancy between the number of second feature points and the number of first feature points in the workpiece image, thus affecting the accuracy of feature point matching. Therefore, in this embodiment, feature point matching is performed based on the similarity of the distances between feature points. Specifically, let the number of first feature points be N. For each first feature point Ai, the first distances between Ai and other first feature points (excluding itself) can be sorted, and a vector Ji is constructed based on the sorted first distances. Let the number of second feature points be M. For each second feature point Bi, the second distances between Bi and other second feature points (excluding itself) can be sorted, and a vector Ki is constructed based on the sorted second distances. Subsequently, for each second feature point Bi, the similarity between the vector Ki of the second feature point Bi and the vector of each first feature point is determined. This similarity represents the similarity between the second feature point and the first feature point. Then, the first and second feature points with the highest similarity are selected. If the highest similarity is higher than a preset similarity threshold, the first and second feature points with the highest similarity are determined to be mutually matched first and second target feature points. Thus, after traversing all the second feature points, multiple sets of mutually matched first and second target feature points are obtained. The similarity between two vectors can be calculated based on similarity calculation methods such as cosine similarity, Euclidean distance, or Manhattan distance.

[0049] S400 determines the pose of the template workpiece and the pose of the workpiece to be grasped based on the pose information of the first target feature point and the second pose information of the second target feature point, respectively.

[0050] In practical applications, the transformation matrix from the camera coordinate system to the robot base coordinate system can be obtained in advance through methods such as hand-eye calibration.

[0051] In specific implementation, since this embodiment uses the workpiece grasping pose of the template workpiece as a reference and determines the workpiece grasping pose of the workpiece to be grasped based on the offset calculated by vision, before determining the pose deviation between the pose of the template workpiece and the workpiece to be grasped, workpiece coordinates are established using matching feature points between the template workpiece and the workpiece to be grasped to obtain the poses of both the template workpiece and the workpiece to be grasped. Specifically, the pose information of the first target feature point is first transformed to the robot base coordinate system according to the transformation matrix, and the pose information of the second target feature point is then transformed to the robot base coordinate system according to the transformation matrix. Subsequently, for the template workpiece, a first target feature point can be selected from the first target feature points as the origin of the template workpiece coordinate system. Then, the normal vector of the selected first target feature point is determined as the Z-axis direction of the template workpiece coordinate system, and the vector perpendicular to the Z-axis direction is determined as the X-axis direction. The Y-axis vector is obtained by the cross product between the vectors in the Z-axis direction and the vectors in the X-axis direction, thereby obtaining the pose of the template workpiece. For the workpiece to be grasped, the pose of the workpiece to be grasped is obtained by using the above method of determining the pose of the template workpiece.

[0052] S500 adjusts the workpiece gripping pose of the workpiece loading robot based on the pose deviation between the pose of the template workpiece and the pose of the workpiece to be gripped, as well as the gripping pose of the template workpiece.

[0053] In practice, the poses of both the template workpiece and the workpiece to be grasped can be transformed to the robot's base coordinate system to facilitate the planning of the grasping path. Let the pose of the template workpiece in the robot's base coordinate system be X. t The real-time pose of the workpiece is X c Then determine the pose deviation X between the pose of the template workpiece and the pose of the workpiece to be grasped. o It can be obtained through the following formula:

[0054]

[0055] After obtaining the pose deviation, the pose of the gripper tool when the workpiece loading robot grasps the workpiece is determined based on the pose deviation and the template workpiece grasping pose. Specifically, the template workpiece grasping pose and the pose deviation are multiplied and transformed to obtain the pose of the gripper tool when the workpiece loading robot grasps the workpiece, thus enabling the robot control system to plan the grasping path. The robot base coordinate system is a coordinate system based on the robot mounting base, with its origin located on the robot base, used to determine the position and orientation of each axis of the robot.

[0056] In the aforementioned robot workpiece grasping pose adjustment method, firstly, feature points of the template workpiece and the workpiece to be grasped are identified, and the pose information of the feature points is determined. Secondly, the first distance between each first feature point and the second distance between each second feature point are determined. This facilitates similarity distance search based on the first and second distances, allowing for the selection of multiple sets of matching first and second target feature points from the first and second feature points. Thus, even in complex grasping scenarios (such as workpiece material deviations, positional offsets, posture tilts, or partial occlusions) where the number of feature points of the workpiece to be grasped differs from the number of feature points of the template workpiece, it can still identify feature points matching those of the workpiece to be grasped. The feature points of the template workpiece improve the accuracy and adaptability of feature matching. Furthermore, based on the pose information of the matched feature points, coordinate systems are established for both the template workpiece and the workpiece to be gripped, determining their respective poses. Thus, in actual workpiece gripping scenarios, the workpiece gripping pose of the workpiece loading robot is adjusted based on the pose deviation between the template workpiece and the workpiece to be gripped, as well as the gripping pose of the template workpiece. In this way, by using the pose of the template workpiece as a reference to adjust the pose of the workpiece to be gripped, the accuracy and efficiency of the workpiece loading robot's pose correction are improved. Moreover, it eliminates the need to rely on high-precision material racks or manual teaching of material gripping, improving the flexibility of material gripping and loading, and reducing costs.

[0057] In one exemplary embodiment, such as Figure 3 As shown, based on the first distance and the second distance, multiple sets of matching first target feature points and second target feature points are selected from multiple first feature points and multiple second feature points, including S320 to S380, wherein:

[0058] S320, construct a two-dimensional array based on the first distance between each first feature point. Each row in the two-dimensional array contains the distance value between each first feature point and other first feature points except itself. The row array corresponds one-to-one with the first feature point.

[0059] The row array is a one-dimensional array within a two-dimensional array.

[0060] In specific implementation, the server stores the calculated first distances between each first feature point (S1, S2, ..., Sn) in a two-dimensional array M. The row indices and column indices of this two-dimensional array M correspond to each first feature point; that is, each row of the array corresponds one-to-one with each first feature point. For example, the i-th row Mi of the two-dimensional data M corresponds to the first feature point Si. The i-th row M contains the distance values ​​(first distances) between the first feature point Si and other first feature points besides itself. It can be understood that this row array can also contain the distance value between the first feature point Si and itself (i.e., a distance value of 0).

[0061] For each second feature point, perform the following steps S340 to S380:

[0062] S340, construct a one-dimensional array based on the second distance between the second feature point and other feature points excluding itself.

[0063] For example, for a second feature point Tj in the second feature points (T1, T2, ..., Tm), the server stores the second distance between the second feature point Tj and other second feature points other than itself in a one-dimensional array Vj.

[0064] S360, determine the distance difference between each distance value in the one-dimensional array of the second feature point and the distance value in each row of the two-dimensional array, count the number of distance value pairs whose distance difference is less than a preset distance threshold, and obtain the number of distance value pairs corresponding to each row array. The number of distance value pairs in each row array corresponds one-to-one with the number of distance value pairs.

[0065] The distance values ​​are represented by a one-dimensional array containing the distance values ​​of the second feature point and a two-dimensional array containing the distance values ​​of the second feature point.

[0066] In practical applications, the preset distance threshold can be determined in advance based on the detection requirements.

[0067] In practice, the server compares the one-dimensional array Vj of the second feature point Tj with each row of the two-dimensional array M row by row. Specifically, it compares each distance value in the one-dimensional array Vj of the second feature point Tj with each distance value in the first row of the two-dimensional array M, calculates the distance difference between each pair of distance values, and compares whether the absolute value of the distance difference is less than a preset distance threshold. Then, it counts the number num of distance value pairs whose distance difference is less than the preset distance difference threshold. This number num represents the similarity index between the first feature point and the second feature point. Subsequently, following the above distance value pair comparison method, it traverses each row of the two-dimensional array to obtain the number num of distance value pairs corresponding to each row. It can be understood that if the distance difference between two distance values ​​is less than the preset distance threshold, it indicates that the two distance values ​​are similar. For the first feature point and the second feature point, the more distance value pairs num between the row array of the first feature point and the one-dimensional array of the second feature point, the higher the matching degree between the first feature point and the second feature point.

[0068] In other embodiments, it may also be possible to determine the total number of distance value pairs between each distance value in the one-dimensional array Vj of the second feature point Tj and each distance value in each row of the two-dimensional array M, and to determine the ratio of the number of distance value pairs num whose distance difference is less than a preset distance difference threshold to the total number, and use this ratio as the similarity index between the first feature point and the second feature point.

[0069] S380: Select the maximum value from the number of distance value pairs corresponding to multiple row arrays. If the maximum value is greater than the preset number threshold, mark the first feature point corresponding to the maximum value as the first target feature point, mark the second feature point as the second target feature point, and determine that the first target feature point and the second target feature point match.

[0070] In practical applications, the preset quantity threshold is used to detect whether there is a first feature point in the template workpiece that matches the second feature point. The quantity threshold can be determined by experiments or by the number of identified second feature points and a preset ratio (such as 80%) to accommodate situations where the number of feature points identified due to material deviation is not fixed.

[0071] In practice, after obtaining the number of distance value pairs num corresponding to each row array (i.e., there are n numbers num), the maximum value num-max is selected from the n numbers num, and it is checked whether the maximum value num-max is greater than the preset number threshold. If the maximum value num-max is greater than the preset number threshold, the second feature point Tj is determined as the second target feature point, and the first feature point Si corresponding to the maximum value num-max is determined as the first target feature point. It is determined that the second target feature point and the first target feature point match.

[0072] In other embodiments, the highest quantity ratio can be selected from all quantity ratios, and compared with a preset quantity ratio threshold. If the highest quantity ratio is greater than the preset quantity ratio threshold, the first feature point corresponding to the highest quantity ratio is determined as the first target feature point, and the second feature point corresponding to the highest quantity ratio is determined as the second target feature point. It is then determined that the second target feature point and the first target feature point match. The preset quantity ratio threshold can be set according to the matching accuracy requirements.

[0073] In this embodiment, the process of feature matching between the second feature point and the first feature point can be understood as comparing the distance values ​​in the one-dimensional array of the second feature point and the one-dimensional array corresponding to each first feature point. The number of distance value pairs whose distance difference between the two arrays is less than a preset distance threshold is used as the similarity index between the two feature points. Based on the highest similarity index value, the first feature point that matches the second feature point is determined.

[0074] In this embodiment, by using a feature matching method based on similarity distance search, matching feature point pairs between the workpiece to be grasped and the template workpiece are selected, which helps to improve the adaptability of feature matching in complex grasping scenarios and the accuracy of feature matching.

[0075] In one exemplary embodiment, such as Figure 4 As shown, the process of acquiring the pose information of multiple first feature points in the template workpiece image and the pose information of multiple second feature points in the workpiece image to be grasped includes steps S122 to S126, wherein:

[0076] S122, acquire the template workpiece image and depth map of the template workpiece, and the workpiece image and depth map of the workpiece to be grabbed.

[0077] In practical applications, the operator can select a workpiece to be grasped as a template workpiece from a preset position in the material frame. A structured light camera scans the template workpiece and all other workpieces to be grasped in the material frame, obtaining images of the template workpiece, the workpieces, and their depth maps. The structured light camera then sends these images to a server. The template workpiece image and its depth map are corresponding, as are the workpiece images and their depth maps.

[0078] S124, identify the first elliptical region in the template workpiece image, determine the center coordinates of the first elliptical region, and perform three-dimensional ellipse fitting on the first elliptical region based on the depth information matching the first elliptical region in the depth map, and determine the three-dimensional coordinates and normal vector of the center point of the fitted first three-dimensional elliptical region.

[0079] The first elliptical region represents the elliptical region in the template workpiece image, and the first three-dimensional elliptical region represents the three-dimensional elliptical region in the template workpiece.

[0080] In specific implementation, for the template workpiece image, grayscale processing can be performed on the template workpiece image, and edge features of the template workpiece image can be detected by an edge detection algorithm to obtain an edge detection map. Subsequently, contour detection can be performed on the edge detection map to obtain a contour detection map. Then, contour extraction can be performed on the contour detection map, and an ellipse fitting algorithm (such as OpenCV's fitEllipse function) can be used to identify the first elliptical region in the template workpiece image to obtain the geometric parameters of the first elliptical region. The geometric parameters may include the coordinates of the center point of the elliptical region, the length of the major axis, the length of the minor axis, and the eccentricity of the ellipse.

[0081] After identifying the first elliptical region, to improve the accuracy of feature recognition, misidentified elliptical regions can be filtered based on the geometric parameters of the first elliptical region. Specifically, threshold ranges can be set for the major axis length, minor axis length, and eccentricity of the ellipse, respectively. Then, the misidentified elliptical regions can be filtered to retain the first elliptical region whose major axis length and minor axis length are both within the major axis length threshold range; or it can be elliptical regions whose eccentricity is within the eccentricity threshold range, to eliminate overly flat ellipses.

[0082] If any geometric parameter in the first elliptical region does not meet the aforementioned threshold range, then the first elliptical region is determined to be a misidentified elliptical region and is filtered out.

[0083] After obtaining the first elliptical region, the ROI region of the first elliptical region in the template workpiece image is determined according to the preset pixel size and the center point of the first elliptical region. Then, for each ROI region, the local depth information corresponding to the ROI region is extracted from the depth map of the template workpiece. Through principal component analysis and local depth information, three-dimensional ellipse fitting is performed to obtain the three-dimensional coordinates and normal vector of the center point of the fitted first three-dimensional elliptical region.

[0084] S126, identify the second elliptical region in the workpiece image, determine the center coordinates of the second elliptical region, perform three-dimensional ellipse fitting on the second elliptical region based on the depth information matching the second elliptical region in the depth map, and determine the three-dimensional coordinates and normal vector of the center point of the fitted second three-dimensional elliptical region. The first feature point includes the first three-dimensional elliptical region, and the pose information of the first feature point includes the three-dimensional coordinates and normal vector of the center point of the first three-dimensional elliptical region. The second feature point includes the second three-dimensional elliptical region, and the pose information of the second feature point includes the three-dimensional coordinates and normal vector of the center point of the second three-dimensional elliptical region.

[0085] The second elliptical region represents the elliptical region in the workpiece image, and the second three-dimensional elliptical region represents the three-dimensional elliptical region in the workpiece to be grasped.

[0086] In specific implementation, the second elliptical region in the workpiece image is identified, the center coordinates of the second elliptical region are determined, and a three-dimensional ellipse fitting is performed on the second elliptical region based on the depth information matching the second elliptical region in the depth map to determine the three-dimensional coordinates and normal vector of the center point of the fitted second three-dimensional elliptical region. Referring to the above implementation of identifying the first elliptical region in the template workpiece image, the center coordinates of the first elliptical region are determined, and a three-dimensional ellipse fitting is performed on the first elliptical region based on the depth information matching the first elliptical region in the depth map to determine the three-dimensional coordinates and normal vector of the center point of the fitted first three-dimensional elliptical region, the implementation method will not be repeated here.

[0087] After obtaining the pose information of multiple first target feature points of the template workpiece, workpiece corpuscles are established based on the pose information of the multiple first target feature points. In an exemplary embodiment, determining the pose of the template workpiece based on the pose information of the first target feature points includes:

[0088] The mean three-dimensional coordinates of the center point of each first target feature point are determined to obtain the mean coordinates.

[0089] The mean coordinates represent the coordinates of the origin in the coordinate system of the template workpiece.

[0090] For example, suppose there are N circular holes (first target feature points) on the template workpiece, and the center coordinates of each first target feature point are Pi = (xi, yi, zi). Then, the average x-values ​​of the center coordinates of the N first target feature points are taken, the average y-values ​​of the center coordinates of the N first target feature points are taken, and the average z-values ​​of the center coordinates of the N first target feature points are taken to obtain the average coordinates.

[0091] Determine the average normal vector and the magnitude of the average normal vector of each first target feature point. The magnitude of the average normal vector represents the Z-axis direction of the template workpiece.

[0092] Here, the modulus vector can be a unit vector of vectors.

[0093] In specific implementation, let the normal vector of each first target feature point be (nxi, nyi, nzi). It can be obtained by the server taking the average x value of the normal vector of N first target feature points, the average y value of the normal vector of N first target feature points, and the average z value of the normal vector of N first target feature points to obtain the average normal vector. Then, the unit vector of the average normal vector is determined to obtain the vector representing the Z-axis direction of the template workpiece.

[0094] Determine the average reference vector from the three-dimensional coordinates of the center point of each first target feature point to the mean coordinates, and the magnitude vector of the average reference vector. The magnitude vector of the average reference vector represents the reference X-axis direction of the template workpiece.

[0095] In practice, the vector between the three-dimensional coordinates and the mean coordinates of the center point of each first target feature point is determined, and the mean of each vector is taken to obtain the average reference vector. The magnitude vector of the average reference vector is determined to obtain the reference X-axis direction of the template workpiece.

[0096] In other embodiments, determining the reference X-axis direction of the template workpiece can also involve determining the distance between the three-dimensional coordinates and the mean coordinates of the center points of each first target feature point, filtering out the three-dimensional coordinates of the center point of the farthest first target feature point, determining the vector between the three-dimensional coordinates and the mean coordinates of the center point of the farthest first target feature point, and determining the modulus vector of the vector as the target modulus vector.

[0097] The cross product of the magnitude vector of the average normal vector and the magnitude vector of the average reference vector is determined to obtain the first cross product vector, which represents the Y-axis direction of the template workpiece.

[0098] In practice, the cross product of the magnitude vector of the average normal vector and the magnitude vector of the average reference vector is determined to obtain the first cross product vector, which is perpendicular to the Z-axis direction and the reference X-axis direction.

[0099] The cross product of the first cross product vector and the magnitude vector of the average normal vector is determined to obtain the second cross product vector. The second cross product vector represents the X-axis direction of the template workpiece. The pose of the template workpiece includes the mean coordinates, the magnitude vector of the average normal vector, the first cross product vector, and the second cross product vector.

[0100] In practice, the cross product of the first cross product vector and the magnitude vector of the average normal vector is determined to obtain the second cross product vector, which is perpendicular to the Y-axis and Z-axis directions.

[0101] In this embodiment, the template workpiece is corpuscularly constructed based on the pose information of the first target feature point, and the pose of the template workpiece is determined. This is beneficial for determining the workpiece grasping pose of the workpiece to be grasped based on the pose of the template workpiece.

[0102] In an exemplary embodiment, determining the pose of the workpiece to be grasped based on the second pose information of the second target feature points includes:

[0103] The mean three-dimensional coordinates of the center points of each second target feature point are determined to obtain the mean coordinates.

[0104] The mean coordinates represent the coordinates of the origin in the coordinate system of the workpiece to be grasped.

[0105] For example, suppose there are M circular holes (second target feature points) on the workpiece to be gripped, and the center coordinates of each second target feature point are Pi = (xi, yi, zi). Then, the average x-values ​​of the center coordinates of the M second target feature points are taken, the average y-values ​​of the center coordinates of the M second target feature points are taken, and the average z-values ​​of the center coordinates of the M second target feature points are taken to obtain the average coordinates.

[0106] Determine the average normal vector of the normal vectors of each second target feature point, and determine the magnitude vector of the average normal vector. The magnitude vector of the average normal vector represents the Z-axis direction of the workpiece to be grasped.

[0107] Here, the modulus vector can be a unit vector of vectors.

[0108] In specific implementation, let the normal vector of each second target feature point be (nxi, nyi, nzi). It can be obtained by the server taking the average x value of the normal vector of M second target feature points, the average y value of the normal vector of N second target feature points, and the average z value of the normal vector of M second target feature points to obtain the average normal vector. Then, the unit vector of the average normal vector is determined to obtain the vector representing the Z-axis direction of the workpiece to be grasped.

[0109] Determine the average reference vector from the three-dimensional coordinates of the center point of each second target feature point to the mean coordinates, and the magnitude vector of the average reference vector. The magnitude vector of the average reference vector represents the reference X-axis direction of the workpiece to be grasped.

[0110] In practice, the vector between the three-dimensional coordinates and the mean coordinates of the center point of each second target feature point is determined, and the mean of each vector is taken to obtain the average reference vector. The magnitude vector of the average reference vector is determined to obtain the reference X-axis direction of the workpiece to be grasped.

[0111] In other embodiments, determining the reference X-axis direction of the workpiece to be grasped can also be done by determining the distance between the three-dimensional coordinates and the mean coordinates of the center points of each second target feature point, filtering out the three-dimensional coordinates of the center point of the second target feature point that is furthest away, determining the vector between the three-dimensional coordinates and the mean coordinates of the center point of the furthest second target feature point, and determining the modulus vector of the vector as the target modulus vector.

[0112] The cross product of the magnitude vector of the average normal vector and the magnitude vector of the average reference vector is determined to obtain the third cross product vector, which represents the Y-axis direction of the workpiece to be grasped.

[0113] In practice, the cross product of the magnitude vector of the average normal vector, the magnitude vector of the average reference vector, and the target magnitude vector is determined to obtain the third cross product vector, which is perpendicular to the Z-axis direction and the reference X-axis direction.

[0114] The cross product of the third cross product vector and the magnitude vector of the average normal vector is determined to obtain the fourth cross product vector. The fourth cross product vector represents the X-axis direction of the workpiece to be grasped. The pose of the workpiece to be grasped includes the mean coordinates, the magnitude vector of the average normal vector, the third cross product vector, and the fourth cross product vector.

[0115] In practice, the cross product of the third cross product vector and the magnitude vector of the average normal vector is determined to obtain the fourth cross product vector, which is perpendicular to the Y-axis and Z-axis directions.

[0116] In this embodiment, workpiece corpus is established based on the pose information of the second target feature point to determine the pose of the workpiece to be grasped. This is beneficial for determining the workpiece grasping pose of the workpiece to be grasped based on the pose of the template workpiece and the pose of the workpiece to be grasped.

[0117] In an exemplary embodiment, after obtaining the pose information of multiple first feature points of the template workpiece and the pose information of multiple second feature points of the workpiece to be grasped, the method further includes:

[0118] The pose information of multiple first feature points and multiple second feature points are respectively transformed into the flange coordinate system to obtain the pose information of multiple first feature points and multiple second feature points in the flange coordinate system.

[0119] The flange coordinate system is a three-dimensional coordinate system used for robot positioning and motion control. It consists of three axes: X, Y, and Z. The Z-axis is perpendicular to the flange, the X-axis is parallel to the flange's centerline, and the Y-axis, along with the X and Z axes, forms a right-handed coordinate system.

[0120] In practical applications, a homogeneous transformation matrix can be obtained beforehand through hand-eye calibration. This homogeneous transformation matrix represents the pose of the camera coordinate system relative to the robot flange coordinate system. In specific implementation, based on this homogeneous transformation matrix, the pose information of the first feature point and the second feature point in the camera coordinate system are transformed to the flange coordinate system.

[0121] The pose information of multiple first feature points and multiple second feature points in the flange coordinate system are respectively transformed into the robot base coordinate system to obtain the first target pose information of multiple first feature points and the second target pose information of multiple second feature points.

[0122] In practical applications, the pose of the robot flange in the robot base coordinate system can be obtained through the control system of the workpiece loading robot. Based on the pose of the robot flange in the robot base coordinate system and the pose information of the first feature point, the first target pose information of the first feature point can be obtained. Based on the pose of the robot flange in the robot base coordinate system and the pose information of the second feature point, the first target pose information of the second feature point can be obtained.

[0123] Based on the pose information of the first feature point and the second pose information of the second target feature point, the poses of the template workpiece and the workpiece to be grasped are determined, including:

[0124] Based on the first target pose information and the second target pose information, the poses of the template workpiece and the workpiece to be grasped are determined respectively.

[0125] In specific implementation, the implementation method of determining the pose of the template workpiece and the pose of the workpiece to be grasped is based on the first target pose information and the second target pose information, respectively. Referring to the above implementation method of determining the pose of the template workpiece and the pose of the workpiece to be grasped based on the pose information of the first feature point and the second pose information of the second target feature point, respectively, it will not be repeated here.

[0126] In this embodiment, converting the pose of the template workpiece and the pose of the workpiece to be gripped to the same coordinate system helps to improve the accuracy of workpiece gripping pose adjustment.

[0127] To provide a clearer explanation of the robot workpiece grasping pose adjustment method provided in this application, a specific embodiment and accompanying drawings are described below. Figure 6 The specific embodiment includes the following steps:

[0128] S1, acquire the template workpiece image and depth map, the workpiece image and depth map of the workpiece to be grasped, and the template workpiece grasping pose of the workpiece loading robot.

[0129] S2, identify the first elliptical region in the template workpiece image, determine the center coordinates of the first elliptical region, and perform three-dimensional ellipse fitting on the first elliptical region based on the depth information matching the first elliptical region in the depth map, and determine the three-dimensional coordinates and normal vector of the center point of the fitted first three-dimensional elliptical region.

[0130] S3, identify the second elliptical region in the workpiece image, determine the center coordinates of the second elliptical region, and perform three-dimensional ellipse fitting on the second elliptical region based on the depth information matching the second elliptical region in the depth map, and determine the three-dimensional coordinates and normal vector of the center point of the fitted second three-dimensional elliptical region. The first feature point includes the first three-dimensional elliptical region, and the pose information of the first feature point includes the three-dimensional coordinates and normal vector of the center point of the first three-dimensional elliptical region. The second feature point includes the second three-dimensional elliptical region, and the pose information of the second feature point includes the three-dimensional coordinates and normal vector of the center point of the second three-dimensional elliptical region.

[0131] S4, transform the pose information of multiple first feature points and multiple second feature points into the flange coordinate system respectively, to obtain the pose information of multiple first feature points and multiple second feature points in the flange coordinate system. Then transform the pose information of multiple first feature points and multiple second feature points in the flange coordinate system into the robot base coordinate system respectively, to obtain the first target pose information of multiple first feature points and the second target pose information of multiple second feature points.

[0132] S5, determine the first distance between each first feature point and the second distance between each second feature point.

[0133] S6. Based on the first distance between each first feature point, construct a two-dimensional array. Each row of the two-dimensional array contains the distance value between each first feature point and other first feature points except itself. The row array corresponds one-to-one with the first feature point.

[0134] S7. For each second feature point, perform the following processing: Based on the second distance between the second feature point and other feature points excluding itself, construct a one-dimensional array, determine the distance difference between each distance value in the one-dimensional array of the second feature point and the distance value in each row of the two-dimensional array, count the number of distance value pairs whose distance difference is less than a preset distance threshold, obtain the number of distance value pairs corresponding to each row array, and the number of distance value pairs corresponds one-to-one with the number of distance value pairs in the row array. Select the maximum value from the number of distance value pairs corresponding to multiple row arrays. If the maximum value is greater than the preset number threshold, mark the first feature point corresponding to the maximum value as the first target feature point, mark the second feature point as the second target feature point, and determine that the first target feature point and the second target feature point match.

[0135] S8. Determine the mean of the three-dimensional coordinates of the center point of each target first feature point to obtain the mean coordinates. Determine the average normal vector and the magnitude vector of the average normal vector of each target first feature point. The magnitude vector of the average normal vector represents the Z-axis direction of the template workpiece. Determine the average reference vector from the three-dimensional coordinates of the center point of each first target feature point to the mean coordinates, and the magnitude vector of the average reference vector. The magnitude vector of the average reference vector represents the reference X-axis direction of the workpiece to be grasped. Determine the cross product of the magnitude vector of the average normal vector and the magnitude vector of the average reference vector to obtain the first cross product vector. The first cross product vector represents the Y-axis direction of the template workpiece. Determine the cross product of the first cross product vector and the magnitude vector of the average normal vector to obtain the second cross product vector. The second cross product vector represents the X-axis direction of the template workpiece. The pose of the template workpiece includes the mean coordinates, the magnitude vector of the average normal vector, the first cross product vector, and the second cross product vector.

[0136] S9, determine the mean of the three-dimensional coordinates of the center point of each target's second feature point, obtain the mean coordinates, determine the average normal vector of the normal vector of each target's second feature point, determine the magnitude vector of the average normal vector, the magnitude vector of the average normal vector represents the Z-axis direction of the workpiece to be grasped, determine the average reference vector from the three-dimensional coordinates of each target's second feature point to the mean coordinates, and the magnitude vector of the average reference vector, the magnitude vector of the average reference vector represents the reference X-axis direction of the workpiece to be grasped, determine the cross product of the magnitude vector of the average normal vector and the magnitude vector of the average reference vector, obtain the third cross product vector, the third cross product vector represents the Y-axis direction of the workpiece to be grasped, determine the cross product of the third cross product vector and the magnitude vector of the average normal vector, obtain the fourth cross product vector, the fourth cross product vector represents the X-axis direction of the workpiece to be grasped, the pose of the workpiece to be grasped includes the mean coordinates, the magnitude vector of the average normal vector, the third cross product vector, and the fourth cross product vector.

[0137] S9, based on the pose deviation between the pose of the template workpiece and the pose of the workpiece to be gripped, and the gripping pose of the template workpiece, adjust the workpiece gripping pose of the workpiece loading robot.

[0138] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0139] In one exemplary embodiment, such as Figure 5 As shown, a robot workpiece grasping pose adjustment device 600 is provided, including: a data acquisition module 610, a distance determination module 620, a feature point matching module 630, a pose determination module 640, and a pose adjustment module 650, wherein:

[0140] The data acquisition module 610 is used to acquire the pose information of multiple first feature points of the template workpiece, the pose information of multiple second feature points of the workpiece to be grasped, and the template workpiece grasping pose of the workpiece loading robot.

[0141] The distance determination module 620 is used to determine the first distance between each first feature point and the second distance between each second feature point;

[0142] The feature point matching module 630 is used to filter out multiple sets of matching first target feature points and second target feature points from multiple first feature points and multiple second feature points based on a first distance and a second distance;

[0143] The pose determination module 640 is used to determine the pose of the template workpiece and the pose of the workpiece to be grasped based on the pose information of the first feature point and the second pose information of the second target feature point, respectively.

[0144] The pose adjustment module 650 is used to adjust the workpiece grasping pose of the workpiece loading robot based on the pose deviation between the pose of the template workpiece and the pose of the workpiece to be grasped, as well as the grasping pose of the template workpiece.

[0145] In an exemplary embodiment, the feature point matching module 630 is further configured to construct a two-dimensional array based on the first distance between each first feature point, wherein each row array in the two-dimensional array contains the distance values ​​between each first feature point and other first feature points excluding itself, and the row array corresponds one-to-one with the first feature points; for each second feature point, the following processing is performed: constructing a one-dimensional array based on the second distance between the second feature point and other feature points excluding itself; determining the distance difference between each distance value in the one-dimensional array of the second feature point and the distance value in each row array of the two-dimensional array, counting the number of distance value pairs whose distance difference is less than a preset distance threshold, obtaining the number of distance value pairs corresponding to each row array, and the row array corresponds one-to-one with the number of distance value pairs; filtering out the maximum value from the number of distance value pairs corresponding to multiple row arrays, and if the maximum value is greater than the preset number threshold, marking the first feature point corresponding to the maximum value as the first target feature point, marking the second feature point as the second target feature point, and determining that the first target feature point and the second target feature point match.

[0146] In an exemplary embodiment, the data acquisition module 610 is further configured to acquire a template workpiece image and a depth map of the template workpiece, and a workpiece image and a depth map of the workpiece to be grasped; identify a first elliptical region in the template workpiece image, determine the center coordinates of the first elliptical region, and perform three-dimensional ellipse fitting on the first elliptical region based on the depth information matching the first elliptical region in the depth map, thereby determining the three-dimensional coordinates and normal vector of the center point of the fitted first three-dimensional elliptical region; identify a second elliptical region in the workpiece image, determine the center coordinates of the second elliptical region, and perform three-dimensional ellipse fitting on the second elliptical region based on the depth information matching the second elliptical region in the depth map, thereby determining the three-dimensional coordinates and normal vector of the center point of the fitted second three-dimensional elliptical region; a first feature point includes the first three-dimensional elliptical region, and the pose information of the first feature point includes the three-dimensional coordinates and normal vector of the center point of the first three-dimensional elliptical region; a second feature point includes a second three-dimensional elliptical region, and the pose information of the second feature point includes the three-dimensional coordinates and normal vector of the center point of the second three-dimensional elliptical region.

[0147] In an exemplary embodiment, the pose determination module 640 is further configured to: determine the mean of the three-dimensional coordinates of the center points of each target first feature point to obtain mean coordinates; determine the average normal vector and the magnitude vector of the average normal vector of each target first feature point, wherein the magnitude vector of the average normal vector represents the Z-axis direction of the template workpiece; determine the average reference vector from the three-dimensional coordinates of the center points of each first target feature point to the mean coordinates, and the magnitude vector of the average reference vector, wherein the magnitude vector of the average reference vector represents the reference X-axis direction of the template workpiece; determine the cross product of the magnitude vector of the average normal vector and the magnitude vector of the average reference vector to obtain a first cross product vector, wherein the first cross product vector represents the Y-axis direction of the template workpiece; and determine the cross product of the first cross product vector and the magnitude vector of the average normal vector to obtain a second cross product vector, wherein the second cross product vector represents the X-axis direction of the template workpiece; the pose of the template workpiece includes mean coordinates, the magnitude vector of the average normal vector, the first cross product vector, and the second cross product vector.

[0148] In an exemplary embodiment, the pose determination module 640 is further configured to: determine the mean of the three-dimensional coordinates of the center points of each target second feature point to obtain mean coordinates; determine the average normal vector of the normal vectors of each target second feature point, determine the magnitude vector of the average normal vector, the magnitude vector of the average normal vector representing the Z-axis direction of the workpiece to be grasped; determine the average reference vector from the three-dimensional coordinates of the center points of each second target feature point to the mean coordinates, and the magnitude vector of the average reference vector, the magnitude vector of the average reference vector representing the reference X-axis direction of the template workpiece; determine the cross product of the magnitude vector of the average normal vector and the magnitude vector of the average reference vector to obtain a third cross product vector, the third cross product vector representing the Y-axis direction of the workpiece to be grasped; determine the cross product of the third cross product vector and the magnitude vector of the average normal vector to obtain a fourth cross product vector, the fourth cross product vector representing the X-axis direction of the workpiece to be grasped; the pose of the workpiece to be grasped includes the mean coordinates, the magnitude vector of the average normal vector, the third cross product vector, and the fourth cross product vector.

[0149] In an exemplary embodiment, the robot workpiece gripping pose adjustment device 600 further includes a pose transformation module 660, which is used to transform the pose information of a plurality of first feature points and the pose information of a plurality of second feature points to the flange coordinate system, respectively, to obtain the pose information of a plurality of first feature points and the pose information of a plurality of second feature points in the flange coordinate system; and to transform the pose information of a plurality of first feature points and the pose information of a plurality of second feature points in the flange coordinate system to the robot base coordinate system, respectively, to obtain the first target pose information of a plurality of first feature points and the second target pose information of a plurality of second feature points;

[0150] The pose determination module 640 is also used to determine the pose of the template workpiece and the pose of the workpiece to be grasped based on the first target pose information and the second target pose information, respectively.

[0151] Each module in the aforementioned robot workpiece gripping pose adjustment device 600 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0152] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When executed by the processor, the computer program implements a method for adjusting the pose of a robot workpiece grasping.

[0153] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0154] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above embodiments of the robot workpiece grasping pose adjustment method.

[0155] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps in any of the above embodiments of the robot workpiece grasping pose adjustment method.

[0156] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the robot workpiece grasping pose adjustment method.

[0157] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0158] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0159] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0160] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for adjusting the posture of a robot workpiece grasping, characterized in that, The method includes: Acquire the pose information of multiple first feature points of the template workpiece, the pose information of multiple second feature points of the workpiece to be grasped, and the template workpiece grasping pose of the workpiece loading robot. Determine the first distance between each first feature point and the second distance between each second feature point; Based on the first distance and the second distance, multiple sets of matching first target feature points and second target feature points are selected from multiple first feature points and multiple second feature points; Based on the first pose information of the first target feature point and the second pose information of the second target feature point, the pose of the template workpiece and the pose of the workpiece to be grasped are determined respectively. Based on the pose deviation between the pose of the template workpiece and the pose of the workpiece to be gripped, and the gripping pose of the template workpiece, the workpiece gripping pose of the workpiece loading robot is adjusted.

2. The method according to claim 1, characterized in that, The step of selecting multiple sets of matching first target feature points and second target feature points from multiple first feature points and multiple second feature points based on the first distance and the second distance includes: Based on the first distance between each first feature point, a two-dimensional array is constructed. Each row of the two-dimensional array contains the distance value between each first feature point and other first feature points excluding itself. Each row corresponds one-to-one with the first feature point. For each of the second feature points, the following processing is performed: Construct a one-dimensional array based on the second distance between the second feature point and other feature points excluding itself; Determine the distance difference between each distance value in the one-dimensional array of the second feature point and the distance value in each row of the two-dimensional array, count the number of distance value pairs whose distance difference is less than a preset distance threshold, and obtain the number of distance value pairs corresponding to each row of the array. The number of distance value pairs corresponds one-to-one with the number of row arrays. The maximum value is selected from the number of distance value pairs corresponding to the multiple row arrays. If the maximum value is greater than a preset number threshold, the first feature point corresponding to the maximum value is marked as the first target feature point, and the second feature point is marked as the second target feature point. It is determined that the first target feature point and the second target feature point match.

3. The method according to claim 1, characterized in that, The process of acquiring the pose information of multiple first feature points in the template workpiece image and the pose information of multiple second feature points in the workpiece image to be grasped includes: Obtain the template workpiece image and depth map, as well as the workpiece image and depth map of the workpiece to be grasped; Identify the first elliptical region in the template workpiece image, determine the center coordinates of the first elliptical region, and perform three-dimensional ellipse fitting on the first elliptical region based on the depth information that matches the first elliptical region in the depth map, and determine the three-dimensional coordinates and normal vector of the center point of the fitted first three-dimensional elliptical region. Identify the second elliptical region in the workpiece image, determine the center coordinates of the second elliptical region, and perform three-dimensional ellipse fitting on the second elliptical region based on the depth information that matches the second elliptical region in the depth map, and determine the three-dimensional coordinates and normal vector of the center point of the fitted second three-dimensional elliptical region. The first feature point includes the first three-dimensional elliptical region, and the pose information of the first feature point includes the three-dimensional coordinates and normal vector of the center point of the first three-dimensional elliptical region. The second feature point includes the second three-dimensional elliptical region, and the pose information of the second feature point includes the three-dimensional coordinates and normal vector of the center point of the second three-dimensional elliptical region.

4. The method according to claim 3, characterized in that, Based on the first pose information of the first target feature point, the pose of the template workpiece is determined, including: The mean three-dimensional coordinates of the center points of each of the first target feature points are determined to obtain the mean coordinates; Determine the average normal vector of the normal vectors of each of the first target feature points, and the magnitude vector of the average normal vector, wherein the magnitude vector of the average normal vector represents the Z-axis direction of the template workpiece; Determine the average reference vector from the three-dimensional coordinates of the center point of each first target feature point to the mean coordinates, and the magnitude vector of the average reference vector, wherein the magnitude vector of the average reference vector characterizes the reference X-axis direction of the template workpiece; The cross product of the magnitude vector of the average normal vector and the magnitude vector of the average reference vector is determined to obtain a first cross product vector, which represents the Y-axis direction of the template workpiece. The cross product of the first cross product vector and the magnitude vector of the average normal vector is determined to obtain the second cross product vector, which represents the X-axis direction of the template workpiece. The pose of the template workpiece includes the mean coordinates, the magnitude vector of the mean normal vector, the first cross product vector, and the second cross product vector.

5. The method according to claim 3, characterized in that, Based on the second pose information of the second target feature points, the pose of the workpiece to be grasped is determined, including: Determine the mean value of the three-dimensional coordinates of the center point of each of the second target feature points to obtain the mean coordinates; Determine the average normal vector of the normal vectors of each of the second target feature points, and determine the magnitude vector of the average normal vector, wherein the magnitude vector of the average normal vector represents the Z-axis direction of the workpiece to be grasped; Determine the average reference vector from the three-dimensional coordinates of the center point of each second target feature point to the mean coordinates, and the magnitude vector of the average reference vector, wherein the magnitude vector of the average reference vector represents the reference X-axis direction of the workpiece to be grasped; The cross product of the magnitude vector of the average normal vector and the magnitude vector of the average reference vector is determined to obtain the third cross product vector, which represents the Y-axis direction of the workpiece to be grasped. The cross product of the third cross product vector and the magnitude vector of the average normal vector is determined to obtain the fourth cross product vector, which represents the X-axis direction of the workpiece to be grasped. The pose of the workpiece to be grasped includes the mean coordinates, the magnitude vector of the mean normal vector, the third cross product vector, and the fourth cross product vector.

6. The method according to any one of claims 1 to 5, characterized in that, After acquiring the pose information of multiple first feature points of the template workpiece and the pose information of multiple second feature points of the workpiece to be grasped, the method further includes: The pose information of multiple first feature points and multiple second feature points are respectively transformed into the robot flange coordinate system to obtain the pose information of multiple first feature points and multiple second feature points in the robot flange coordinate system. The pose information of multiple first feature points and multiple second feature points in the robot flange coordinate system are respectively transformed to the robot base coordinate system to obtain the first target pose information of multiple first feature points and the second target pose information of multiple second feature points; The step of determining the pose of the template workpiece and the pose of the workpiece to be grasped based on the pose information of the first feature point and the second pose information of the second target feature point, respectively, includes: The poses of the template workpiece and the workpiece to be grasped are determined based on the first target pose information and the second target pose information, respectively.

7. A robot workpiece gripping posture adjustment device, characterized in that, The device includes: The data acquisition module is used to acquire the pose information of multiple first feature points of the template workpiece, the pose information of multiple second feature points of the workpiece to be grasped, and the template workpiece grasping pose of the workpiece loading robot. The distance determination module is used to determine the first distance between each first feature point and the second distance between each second feature point; The feature point matching module is used to filter out multiple sets of matching first target feature points and second target feature points from multiple first feature points and multiple second feature points based on the first distance and the second distance; The pose determination module is used to determine the pose of the template workpiece and the pose of the workpiece to be grasped based on the pose information of the first feature point and the second pose information of the second target feature point, respectively. The pose adjustment module is used to adjust the workpiece grasping pose of the workpiece loading robot based on the pose deviation between the pose of the template workpiece and the pose of the workpiece to be grasped, as well as the grasping pose of the template workpiece.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images