Self-adaptive workpiece grabbing pose correction method and system based on feature point matching

Abstract

The invention provides a self-adaptive workpiece grabbing pose correction method and system based on feature point matching. The method comprises the steps that a standard image of a workpiece which is successfully grabbed to a jig and an auxiliary image of the workpiece which is not successfully grabbed to the jig are shot; the template image and the auxiliary image are processed, feature points in the images are recognized and compared, matched feature points between the template image and the similar image are found out, and three-dimensional coordinates matched with the feature points are obtained; and based on the matched feature points and the three-dimensional coordinates of the feature points, the pose of the workpiece is solved through an RANSAC PNP algorithm, the pose of the current unsuccessfully grabbed workpiece relative to the position of the template drawing is recovered, and it is ensured that the workpiece can be accurately placed on the jig by adjusting the pose of the mechanical arm. And through feature extraction and matching of the SIFT algorithm and the LightGlue algorithm, accurate and effective correction of the workpiece placement pose is achieved, important industrial application value and market potential are achieved, and powerful technical support is provided for improving the placement precision and efficiency of the mechanical arm.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision positioning, and in particular to a method and system for adaptive workpiece grasping posture correction based on feature point matching. Background Art

[0002] In modern manufacturing and automated production processes, robotic arms are widely used in operations such as material handling, workpiece grasping and placement. However, due to the possible differences in the shape, size, material and posture of the workpiece, robotic arms face a series of challenges when grasping and placing workpieces, such as posture errors, grasping failures, collision risks, etc.

[0003] Especially in application scenarios where workpieces need to be accurately placed on fixtures, such as in assembly, inspection, and processing, the precise position and posture of the workpiece are particularly critical. Conventional robotic arm control methods often fail to meet the requirements of precise placement, and the main technical problems are as follows:

[0004] 1. Inaccurate grasping posture estimation: In the process of workpiece grasping by the robot arm, the accuracy of posture estimation is crucial. However, due to various factors, such as changes in lighting, surface characteristics of the workpiece, etc., there may be a certain degree of posture estimation error when the robot arm grasps the workpiece. If this error is not corrected in a timely and accurate manner, it may cause the workpiece to be unable to be accurately aligned and positioned on the fixture during the placement process. For example, even a slight posture deviation may cause the workpiece to be unable to be fully inserted or fixed in the fixture, thereby affecting the overall production efficiency, quality and safety. In addition, inaccurate posture estimation may also cause the robot arm to bump into the workpiece during the placement process, affecting the subsequent quality inspection process.

[0005] 2. Insufficient surface texture features of workpieces: In automated production and manufacturing processes, the texture features of the workpiece surface are often used as key information for visual inspection and positioning. However, since the surface texture features of some workpieces may be fewer or less obvious, this poses additional challenges for the robot arm to perform posture correction before placement. When faced with workpieces with fewer texture features, traditional posture correction methods, such as feature matching, edge detection, or template alignment, may not provide a universal and effective correction strategy. This requires the design and optimization of posture correction algorithms for the specific shape and size of each workpiece, which increases the complexity of development and application.

[0006] 3. Scale uncertainty problem: When performing posture correction, the captured template image and the current image to be corrected are usually used to solve the 2D-2D posture. At this time, the scale of the estimated translation amount is uncertain. This scale uncertainty will inevitably cause deviations in the robot arm when placing the workpiece. Summary of the invention

[0007] In response to the above technical problems, the present invention provides an adaptive workpiece grasping posture correction method and system based on feature point matching to solve the posture correction problem caused by insufficient surface texture features of the workpiece. The template image and auxiliary image are used in combination with SIFT and LightGlue algorithms to achieve accurate correction of the workpiece placement posture and improve the placement accuracy and efficiency of the robotic arm.

[0008] In order to achieve the above object, the present invention provides an adaptive workpiece grasping posture correction method based on feature point matching, comprising:

[0009] Take a standard image of the workpiece being successfully grasped onto the fixture and an auxiliary image of the workpiece being unsuccessfully grasped onto the fixture;

[0010] Process the template image and the auxiliary image, identify the feature points in the image and compare them, find the matching feature points between the template image and the similar image, and obtain the three-dimensional coordinates of the matching feature points;

[0011] Based on the matched feature points and their three-dimensional coordinates, the RANSAC PNP algorithm is used to solve the workpiece pose, restore the pose of the currently unsuccessfully grasped workpiece relative to the template position, and ensure that the workpiece can be accurately placed on the fixture by adjusting the robot arm pose.

[0012] Furthermore, in the process of processing the template image and the auxiliary image, the workpiece is annotated, the background area is removed, and the effective area of ​​the workpiece is retained.

[0013] Furthermore, SIFT feature point extraction is performed on the template image and the auxiliary image to identify key points in the image and generate descriptors for the key points.

[0014] Furthermore, the SIFT feature points are matched according to the LightGlue algorithm to find the corresponding feature points between the template image and the auxiliary image.

[0015] Furthermore, in the formal process, for a workpiece to be corrected grasped by a robotic arm, a picture of the workpiece in its current state is taken to extract SIFT position points, and the SIFT feature points are matched according to the LightGlue algorithm to determine the correspondence between the feature points of the workpiece image and the template image.

[0016] Furthermore, triangulation calculation is performed based on the feature points matched to the corresponding relationship, so as to convert the two-dimensional coordinates of the feature points into corresponding three-dimensional coordinates.

[0017] On the other hand, the present invention also provides an adaptive workpiece grasping posture correction system based on feature point matching, comprising:

[0018] The data acquisition module captures the standard image of the workpiece successfully grasped by the fixture and the auxiliary image of the workpiece not successfully grasped by the fixture;

[0019] The template image processing module annotates the workpiece, removes the background area, and retains the effective area of ​​the workpiece;

[0020] Feature extraction and matching module, which identifies and compares the feature points in the image to find the matching feature points between the template image and the similar image;

[0021] The coordinate triangulation calculation module performs triangulation calculation based on the feature points that match the corresponding relationship, thereby converting the two-dimensional coordinates of the feature points into the corresponding three-dimensional coordinates

[0022] The posture estimation and correction module captures the workpiece to be corrected grasped by the robotic arm, solves the workpiece posture through the RANSAC PNP algorithm based on the matched feature points and their three-dimensional coordinates, restores the posture of the currently unsuccessfully grasped workpiece relative to the template image position, and ensures that the workpiece can be accurately placed on the fixture by adjusting the robotic arm posture.

[0023] The present invention provides an adaptive workpiece grasping posture correction method and system based on feature point matching, which shoots a standard image of a workpiece successfully grasped to a fixture and an auxiliary image of an unsuccessful grasping of the fixture; processes a template image and an auxiliary image, identifies feature points in the image and compares them, finds out the matching feature points between the template image and the similar image, and obtains the three-dimensional coordinates of the matched feature points; based on the matched feature points and their three-dimensional coordinates, solves the workpiece posture through a RANSAC PNP algorithm, restores the posture of the currently unsuccessfully grasped workpiece relative to the position of the template image, and ensures that the workpiece can be accurately placed on the fixture by adjusting the posture of a robotic arm. The present invention realizes accurate and effective correction of the workpiece placement posture by combining data collection of the template image and the auxiliary image, as well as feature extraction and matching of SIFT and LightGlue algorithms, has important industrial application value and market potential, and provides strong technical support for improving the placement accuracy and efficiency of robotic arms. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0025] Figure 1 The present invention is a flowchart of a method for adaptive workpiece grasping posture correction based on feature point matching according to an embodiment of the present invention.

[0026] Figure 2This is a system framework diagram of an adaptive workpiece grasping posture correction system based on feature point matching according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0029] The purpose of the present invention is to provide an adaptive workpiece grasping posture correction method and system based on feature point matching to solve the posture correction problem caused by insufficient workpiece surface texture features.

[0030] Figure 1 The flowchart of the method of the present invention is a method for adaptive workpiece grasping posture correction based on feature point matching. Figure 1 As shown, an adaptive workpiece grasping posture correction method based on feature point matching of the present invention includes the following steps:.

[0031] S1, data acquisition.

[0032] Specifically, data collection includes template image collection and auxiliary image collection. Among them, template image collection first takes an image in a known state where the workpiece can be successfully placed on the fixture after being grasped as a template image. This image provides a reference standard for correct placement. Auxiliary image collection appropriately adjusts the robot arm to change the state of the workpiece, and takes a picture of the workpiece in its current state as an auxiliary image. In this state, the workpiece may not be accurately placed in the fixture.

[0033] S2, template graph processing

[0034] Specifically, the template image processing first marks the workpiece in the template image, removes the background area, and only retains the effective area of ​​the workpiece, ensuring that the feature points matching the template image are concentrated on the workpiece to avoid image segmentation processing.

[0035] S3, feature extraction and matching.

[0036] Specifically, SIFT feature points are extracted from the template image and the auxiliary image to identify key points in the image and generate descriptors for these points. The LightGlue algorithm is used to match the feature points extracted by SIFT to find the corresponding feature points between the template image and the auxiliary image.

[0037] In this embodiment, the feature points and feature descriptors extracted in this step are saved in the form of a two-dimensional array. For example, in one of our tests, 121 feature points were extracted in this step, and the feature points were saved as a (121,2)-dimensional python array; since the descriptor of each feature point is a 128-dimensional vector, the feature descriptor is saved as a (121,128)-dimensional python array.

[0038] S4, triangulate and calculate 3D points.

[0039] Specifically, based on the matched feature points, triangulation calculation is performed on them to determine the position of the 2D feature points in the 3D space and obtain the 3D coordinate points corresponding to the 2D feature points. The corresponding 3D coordinate points are calculated by triangulation and saved as a (121,3)-dimensional python array.

[0040] S5, pose estimation and correction.

[0041] In the formal process, for a workpiece to be calibrated that is grasped by a robotic arm, a picture of the workpiece in its current state is taken, and feature points are extracted using sift, and lightglue is used to match the feature points on the template image. The matched feature points and their corresponding 3D points can be obtained, and the workpiece posture can be solved using the RANSAC PNP algorithm, and the posture of the workpiece relative to the template position when the current picture was taken can be restored, and the posture of the robotic arm can be adjusted to ensure that the workpiece can be accurately placed on the fixture.

[0042] Figure 2 FIG. 1 is a system framework diagram of an adaptive workpiece grasping posture correction system based on feature point matching according to an embodiment of the present invention. Figure 2 As shown, the present invention provides an adaptive workpiece grasping posture correction system based on feature point matching, comprising:

[0043] The data acquisition module captures the standard image of the workpiece successfully grasped to the fixture and the auxiliary image of the workpiece not successfully grasped to the fixture.

[0044] The template image processing module annotates the workpiece, removes the background area, and retains the effective area of ​​the workpiece.

[0045] The feature extraction and matching module identifies and compares the feature points in the image to find the matching feature points between the template image and the similar image.

[0046] The coordinate triangulation calculation module performs triangulation calculation based on the feature points that match the corresponding relationship, thereby converting the two-dimensional coordinates of the feature points into corresponding three-dimensional coordinates.

[0047] The posture estimation and correction module captures the workpiece to be corrected grasped by the robotic arm, solves the workpiece posture through the RANSAC PNP algorithm based on the matched feature points and their three-dimensional coordinates, restores the posture of the currently unsuccessfully grasped workpiece relative to the template image position, and ensures that the workpiece can be accurately placed on the fixture by adjusting the robotic arm posture.

[0048] Although the embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art can change, modify, replace and modify the above embodiments within the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An adaptive workpiece grasping posture correction method based on feature point matching, characterized in that: include: Take a standard image of the workpiece being successfully grasped onto the fixture and an auxiliary image of the workpiece being unsuccessfully grasped onto the fixture; Process the template image and the auxiliary image, identify the feature points in the image and compare them, find the matching feature points between the template image and the similar image, and obtain the three-dimensional coordinates of the matching feature points; Based on the matched feature points and their three-dimensional coordinates, the RANSAC PNP algorithm is used to solve the workpiece pose, restore the pose of the currently unsuccessfully grasped workpiece relative to the template position, and ensure that the workpiece can be accurately placed on the fixture by adjusting the robot arm pose.

2. The method for adaptive workpiece grasping posture correction based on feature point matching according to claim 1, characterized in that: In the process of processing the template image and the auxiliary image, the workpiece is annotated, the background area is removed, and the effective area of ​​the workpiece is retained.

3. The method for adaptive workpiece grasping posture correction based on feature point matching as described in claim 1, characterized in that: SIFT feature point extraction is performed on the template image and the auxiliary image to identify key points in the image and generate descriptors for the key points.

4. The method for adaptive workpiece grasping posture correction based on feature point matching according to claim 1, characterized in that: The SIFT feature points are matched according to the LightGlue algorithm to find the corresponding feature points between the template image and the auxiliary image.

5. The method for adaptive workpiece grasping posture correction based on feature point matching according to claim 1, characterized in that: In the formal process, for a workpiece to be calibrated that is grasped by a robotic arm, a picture of the workpiece in its current state is taken to extract SIFT position points, and the SIFT feature points are matched according to the LightGlue algorithm to determine the correspondence between the feature points of the workpiece image and the template image.

6. The method for adaptive workpiece grasping posture correction based on feature point matching according to any one of claims 4 or 5, characterized in that: Triangulation calculation is performed based on the feature points that match the corresponding relationship, so as to convert the two-dimensional coordinates of the feature points into corresponding three-dimensional coordinates.

7. The adaptive workpiece grasping posture correction system based on feature point matching according to claim 1, characterized in that: include: The data acquisition module captures the standard image of the workpiece successfully grasped by the fixture and the auxiliary image of the workpiece not successfully grasped by the fixture; The template image processing module annotates the workpiece, removes the background area, and retains the effective area of ​​the workpiece; Feature extraction and matching module, which identifies and compares the feature points in the image to find the matching feature points between the template image and the similar image; The coordinate triangulation calculation module performs triangulation calculation based on the feature points that match the corresponding relationship, thereby converting the two-dimensional coordinates of the feature points into the corresponding three-dimensional coordinates The posture estimation and correction module captures the workpiece to be corrected grasped by the robotic arm, solves the workpiece posture through the RANSAC PNP algorithm based on the matched feature points and their three-dimensional coordinates, restores the posture of the currently unsuccessfully grasped workpiece relative to the template image position, and ensures that the workpiece can be accurately placed on the fixture by adjusting the robotic arm posture.

Images