Workpiece visual positioning method and system based on contour key point matching

Abstract

The present application relates to the technical field of image processing, more particularly, the present application relates to a kind of workpiece visual positioning method and system based on contour key point matching, comprising: obtaining current image containing workpiece, and extracting the contour of the current image, based on Harris operator, extract multiple key points on the contour;The stability of each key point is calculated.The present application effectively eliminates the interference of light noise by constructing a stability evaluation model containing gradient, curvature and Harris response;Topological correlation strength combining geometric distance and polar angle constraint is first created, which quantifies the support ability of feature points to workpiece structure;Further, adaptive matching weight is constructed and applied to weighted RANSAC algorithm, which changes the traditional uniform sampling strategy, and realizes high robustness, high precision and high efficiency positioning of workpiece under complex industrial scene (such as occlusion, reflection).

Description

Technical Field

[0001] This invention relates to the field of [field / industry]. More specifically, this invention relates to a workpiece visual positioning method and system based on contour key point matching. Background Technology

[0002] With the improvement of industrial automation, machine vision-based workpiece positioning technology plays a core role in intelligent manufacturing, especially in the robotic grasping stage of conveyor belt production lines. Accurately acquiring the workpiece's pose is a prerequisite for achieving automated operations. The lighting environment in actual industrial settings is complex, and the surfaces of metal or smooth workpieces are prone to glare. Furthermore, workpieces often appear at arbitrary angles and sizes, placing extremely high demands on the robustness of vision systems.

[0003] Existing technologies typically employ contour feature point matching to achieve localization. The specific process generally includes: acquiring a workpiece image and extracting the contour; using the Harris operator to detect corner points on the contour as key points; combining feature descriptors to perform point-to-point matching; and finally using the Random Sample Consensus (RANSAC) algorithm to remove mismatched points and calculate the workpiece pose.

[0004] However, directly applying the above algorithms in complex industrial scenarios has significant drawbacks: First, the traditional Harris operator extracts corner points based solely on changes in local pixel grayscale gradients, failing to effectively distinguish between genuine geometric inflection points of the workpiece and pseudo-keypoints (i.e., noise points) caused by light reflection, oil stains, or textures. This results in a large amount of unreliable information being mixed into the feature point set, leading to poor stability. Second, the standard RANSAC algorithm employs a uniform sampling strategy during iteration, treating all feature points equally and ignoring the quality differences in stability and geometric structure of the keypoints themselves. When there are a large number of pseudo-feature points in the image or when the workpiece is partially occluded, this indiscriminate sampling method causes a surge in the number of algorithm iterations, severely reducing computational efficiency and easily leading to pose calculation deviations or even matching failures due to excessively low inlier rates, making it difficult to meet the high efficiency and high reliability requirements of industrial production. Summary of the Invention

[0005] This invention provides a workpiece visual positioning method and system based on contour key point matching, aiming to solve the problems in related technologies.

[0006] In a first aspect, the present invention provides a workpiece visual localization method based on contour keypoint matching, comprising: acquiring a current image containing the workpiece and extracting the contour of the current image; extracting multiple keypoints on the contour based on a Harris operator; calculating the stability of each keypoint, wherein the stability is positively correlated with the Harris corner response value of the corresponding keypoint and with the local curvature at the keypoint. and the average curvature of the contour points in the neighborhood The absolute values ​​of the differences are negatively correlated and are related to the gradient intensity along the contour normal direction. The topological correlation strength between any two key points is positively correlated with the product of the stability of the two key points and the Euclidean distance between them. The average distance from all keypoints to the geometric center of the contour is positively correlated with the absolute value of the polar angle difference between two keypoints in the polar coordinate system with the geometric center as the origin. The matching weight of each keypoint is calculated, and this matching weight is positively correlated with the mean of the topological association strength of the keypoint and other keypoints in its neighborhood, the distance from the keypoint to the geometric center of the contour, and the stability. Based on the matching weight, keypoints in the template image are matched with keypoints in the current image, and the pose of the workpiece in the current image is solved using a weighted RANSAC algorithm. In the weighted RANSAC algorithm, the weight of each matching point pair is its corresponding matching weight. By introducing a stability evaluation combining local curvature, gradient, and Harris response, false keypoints caused by illumination reflection or texture are effectively eliminated. Simultaneously, a topological association strength reflecting the geometric structure's support capability is constructed, and this is used to calculate the matching weight to improve the RANSAC sampling strategy. This allows the algorithm to prioritize the use of high-quality feature points for pose calculation in complex industrial environments (such as those with significant noise or local occlusion), significantly improving the robustness, accuracy, and computational efficiency of the localization.

[0007] Furthermore, the stability of each key point is calculated using the following formula: In the formula, Indicates the first The stability of each key point; For the first Harris corner response values ​​at key points; The maximum response value of all keypoints in the current image, used for normalization; Indicates the first Local curvature at key points; Indicates the first The average curvature of contour points within the neighborhood of a key point, for example, a neighborhood of size . ; Indicates the first Gradient strength at each key point along the normal direction; The values ​​are set to extremely small positive numbers to prevent the denominator from being zero. A specific mathematical formula quantifies the stability of keypoints. Utilizing the properties of the exponential function, scores are automatically suppressed when gradient strength is weak or local curvature differs significantly from the neighborhood (typically corresponding to pseudo-keypoints or noise). Conversely, high scores are awarded when gradients are strong and curvature changes conform to physical laws (corresponding to true geometric inflection points). This achieves precise screening of feature point quality at the algorithm's underlying level, providing a reliable data foundation resistant to illumination interference for subsequent matching.

[0008] Furthermore, the formula for calculating the topological association strength is as follows: In the formula, Indicates the first The key point and the first The topological correlation strength of each key point Indicates the first The stability of each key point Indicates the first The stability of each key point Indicates the first The key point and the first Euclidean distance between key points This represents the average distance from all key points to the geometric center of the workpiece. This indicates that in the polar coordinate system with the workpiece's center of gravity as the origin, the first... The key point and the first The absolute value of the polar angle difference at each key point. This represents the adjustment coefficient. By constructing a topological correlation strength model, the product logic of two-point stability is used to ensure that connections are established only between reliable points. The distance factor is combined to simulate the physical lever arm effect, giving higher weights to long-distance point pairs to enhance the constraint on the workpiece rotation angle. At the same time, the polar angle difference is used to suppress redundant information with overly close directions, thereby selecting the feature point combination that provides the strongest support for the overall rigid structure of the workpiece.

[0009] Furthermore, the formula for calculating the matching weight is as follows: In the formula: Indicates the first Matching weights for each key point; Indicates the first The distance from each key point to the geometric center of the profile; Indicates the first The mean of the topological association strength between each key point and other key points in its neighborhood; This represents the average distance from all keypoints to the center. For the first The stability of each key point. This method integrates the inherent stability of the key points, their importance in the overall topology, and their geometric distribution (long lever arm) relative to the workpiece center. This allows key points located at the workpiece edge and with high structural support to receive higher priority in matching, and even in the event of local occlusion, the reliability of pose calculation can still be maintained by relying on the remaining high-weight key points.

[0010] Furthermore, extracting the contour of the current image includes: performing noise reduction processing on the current image using a bilateral filter, extracting edges using the Canny operator, and using morphological closing operations on the extracted edges to obtain closed contours. The bilateral filter removes Gaussian and salt-and-pepper noise from the image while preserving the gradient information of the workpiece edges to the greatest extent. Combined with the Canny operator and morphological closing operations to obtain continuous closed contours, this effectively solves the problem of contour breaks caused by uneven lighting or reflections on the workpiece surface in industrial settings, ensuring the integrity of subsequent feature extraction.

[0011] Furthermore, after solving the workpiece pose in the current image, the method further includes: refining the pose using weighted least squares on the set of inliers selected by the weighted RANSAC algorithm. After obtaining the initial pose through the weighted RANSAC algorithm, the selected set of inliers is further optimized using weighted least squares, which further reduces the residual of the pose solution and achieves sub-pixel-level refinement of the workpiece position and angle, meeting the needs of high-precision industrial grasping in precision manufacturing.

[0012] Furthermore, matching key points of the template image with key points of the current image based on the matching weight includes: sorting the key points of the template image and the current image in descending order according to the matching weight, selecting the top N key points as seed points and generating descriptors for them, matching the descriptors of the seed points, and incorporating the matching weight as part of the matching cost function, wherein the value of N ranges from 40 to 60. By using a weight-based sorting strategy, only the top N high-quality key points are selected as seed points for descriptor generation and matching, significantly reducing the computational overhead and risk of mismatches for invalid points. Introducing the weight into the matching cost function ensures that high-confidence feature points have higher priority in the search, thereby significantly improving the algorithm's running speed and matching success rate in environments with massive feature points.

[0013] Furthermore, the method also includes: controlling the robotic arm to move above the workpiece in the current image based on the pose, and controlling the end effector to perform rotational compensation to align the workpiece axially and perform a gripping operation.

[0014] In a second aspect, a workpiece visual positioning system based on contour key point matching includes a processor and a memory, the memory storing a computer program, the processor executing the computer program to implement the workpiece visual positioning method based on contour key point matching as described in any of the preceding claims.

[0015] Beneficial effects: By constructing a stability evaluation model that includes gradient, curvature and Harris response, the interference of illumination noise is effectively eliminated; the topological association strength that combines geometric distance and polar angle constraints is the first to quantify the support capability of feature points for the workpiece structure; furthermore, an adaptive matching weight is constructed and applied to the weighted RANSAC algorithm, which changes the traditional uniform sampling strategy and achieves high robustness, high accuracy and high efficiency positioning of workpieces in complex industrial scenarios (such as occlusion and reflection). Attached Figure Description

[0016] Figure 1 This is a schematic diagram illustrating the positioning process according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the comparison of pose calculation results according to an embodiment of the present invention. Detailed Implementation

[0017] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0018] S101: Image acquisition and preprocessing.

[0019] Images of the workpiece to be inspected on the conveyor belt are acquired using an industrial camera (such as a high-resolution CCD camera). Considering the complex lighting conditions in industrial settings, the original images often contain Gaussian noise and salt-and-pepper noise.

[0020] To remove noise while preserving the edge gradient information of the workpiece to the greatest extent, this embodiment preferably uses a bilateral filter for noise reduction. Unlike traditional Gaussian filtering, the bilateral filter considers both the spatial distance between pixels and the similarity of pixel values.

[0021] Subsequently, the Canny operator is used to extract edges, and the breakpoints are connected using morphological closing operations to obtain the closed contour of the workpiece. Finally, the Harris operator is used to extract a preliminary set of key points on the contour line.

[0022] S102: Calculate the stability of each key point.

[0023] To address pseudo-keypoints (such as light spot edges) caused by reflections in industrial scenarios, this step analyzes the curvature changes and grayscale contrast around the contour points to construct edge stability. This metric quantitatively describes whether a keypoint is a genuine physical geometric feature or a false feature point generated by light and shadow interference. The formula for constructing its stability is: In the formula, Indicates the first The stability of each key point; For the first Harris corner response values ​​at key points; The maximum response value of all keypoints in the current image, used for normalization; Indicates the first Local curvature at key points; Indicates the first The average curvature of contour points within the neighborhood of a key point, for example, a neighborhood of size . ; Indicates the first Gradient strength at each key point along the normal direction; Use extremely small positive numbers to prevent the denominator from being zero. The value is 0.01.

[0024] As can be seen from the above formula, the terms inside the exponential function reflect the ratio of the degree of shape abrupt change to the edge sharpness.

[0025] For pseudo-keypoints, such as reflective points: they are usually represented by gradient intensity. It is relatively weak, but due to the irregular shape of the light spot, its local curvature is... Possibly similar to the surrounding average The difference is significant. In this case, the denominator is small while the numerator is large, resulting in a large absolute value of the negative exponent. The result is close to Thus Significantly reduced.

[0026] For real key points, such as workpiece corners: their gradient intensity Typically, the value is large, and the curvature changes in accordance with physical laws. In this case, the denominator is large, and the exponential term approaches... , The result is close to , making Maintain a high value. This formula allows the algorithm to automatically suppress unstructured pseudo-keypoints, achieving initial screening.

[0027] S103: Construct the topological association strength between any two key points.

[0028] After obtaining key points with a certain degree of stability, a logic system with affine transformation invariance needs to be constructed to cope with the rotation and scaling of the workpiece on the conveyor belt. This step introduces topological correlation strength to evaluate the support capability of any combination of two key points for the overall structure of the workpiece.

[0029] In this embodiment, the formula for calculating the topological association strength is as follows: In the formula, Indicates the first The key point and the first The topological correlation strength of each key point Indicates the first The stability of each key point Indicates the first The stability of each key point Indicates the first The key point and the first Euclidean distance between key points This represents the average distance from all key points to the geometric center of the workpiece. This indicates that in the polar coordinate system with the workpiece's center of gravity as the origin, the first... The key point and the first The absolute value of the polar angle difference at each key point. This represents the adjustment coefficient.

[0030] In the above formula, the stability coupling term This is an AND operation. It only works if the points at both ends of the connector are... and Both have high stability. Value close to This term is only high when any point is noise. ), then the whole Zeroing out. This ensures that the topology chain is not built on an unreliable baseline. Lever arm effect term. In rigid body positioning, the farther apart two points are (long baseline), the stronger the constraint of their line on the rotation angle. This term assigns higher weights to point pairs on the longer side or diagonal, simulating the lever arm effect in physics, causing the algorithm to tend to capture the macroscopic contour of the workpiece rather than local minor burrs. Redundancy suppression term If the angles of two key points in the polar coordinate system are very close (i.e.) This indicates that the two key points are in the same direction, such as two pixels near the same rounded corner, which have information redundancy in terms of geometric constraints. In this case, the exponential term decreases sharply, suppressing the weight of the point pair.

[0031] S104: Construct adaptive matching weights for each key point.

[0032] To improve matching accuracy, this step integrates the aforementioned geometric and physical features to construct the final matching weights. These weights quantify the importance of each keypoint in pose calculation. The formula for calculating the matching weights is as follows: In the formula: Indicates the first Matching weights for each key point; Indicates the first The distance from each key point to the geometric center of the profile; Indicates the first The mean of the topological association strength between each key point and other key points in its neighborhood; This represents the average distance from all keypoints to the center. For the first The stability of each key point.

[0033] As can be seen from the above formula, through Eliminate false features caused by environmental noise; secondly, through Assign a higher rotation constraint weight to the distal endpoint; most importantly, through This achieves protection of the workpiece's structural features. Even if a critical point is slightly obstructed, affecting stability... The descent occurs as long as it occupies a critical position in the topological logic chain, i.e. Despite its relatively large size, the keypoint still maintains a certain matching weight, thus significantly improving the algorithm's resistance to occlusion and robustness in complex industrial environments. The formula reflects the coupling relationship between the distance of the keypoint from the center and its physical stability. Keypoints farther from the center (edge ​​feature points) contribute more to rotational localization. The reciprocal amplification of its sensitivity to noise ensures that only points that are both at the edge and have high stability receive high weights.

[0034] S105: Perform matching and pose calculation.

[0035] Using the calculated weights Hierarchical matching and pose calculation are performed. Specifically, the extracted feature points are first sorted using the matching weights of each keypoint, and only the features points before the matching weight sorting are selected. Using 10 points as seed points, and generating corresponding descriptors, in this embodiment, The value is 50. When performing descriptor matching, matching weights are introduced into the matching cost function, and points with higher weights have higher priority in the matching search.

[0036] Subsequently, the transformation matrix is ​​solved using the Weighted Randomized Random Consensus (RANSAC) algorithm. During the RANSAC iteration process, the sampling probability is no longer uniformly distributed, but rather proportional to the weights of the matching point pairs. The reason is that key points located on the long side or diagonal of the workpiece, possessing high topological support (high... The probability of being selected as a base point for model hypothesis increases significantly, thereby significantly reducing the number of RANSAC iterations and reducing the risk of mismatches caused by local occlusion or false detection.

[0037] Finally, for the set of interior points selected by the weighted random consensus algorithm, the weighted least squares method is used to solve for the two-dimensional homogeneous transformation matrix of the workpiece to be tested relative to the standard template, thereby obtaining the accurate workpiece position. and deflection angle The precise workpiece position and deflection angle This is used to provide critical visual guidance data for downstream actuators (such as industrial robots or robotic arms). Given the randomness of the position and orientation of workpieces during transport on the conveyor belt, the control system uses calibration relationships to determine the position in the image coordinate system. Mapping to motion coordinates in the robot arm base coordinate system This drives the robotic arm to move precisely above the workpiece; simultaneously, it utilizes the calculated deflection angle... The end effector (such as a gripper or suction cup) is rotated to compensate for rotation, ensuring its axis is aligned with the actual posture of the workpiece. This process effectively solves the technical problem that traditional blind gripping modes cannot adapt to randomly received materials, avoiding gripping slippage, positioning failure, or rigid collision damage caused by angular deviations, thereby significantly improving the accuracy, success rate, and safety of automated gripping operations.

[0038] like Figure 2 As shown, this figure is a comparison of pose calculation results. Figure 2 The left image (existing technology) shows the results of the traditional RANSAC algorithm. Because the pseudo-keypoints generated by the light spot are mistakenly identified as feature points, the red positioning box is significantly tilted and offset (angle error of about 10 degrees), and the center point deviates from the actual center, which will cause the robotic arm to fail to grasp.

[0039] Figure 2 The right image in this embodiment shows the results based on weighted RANSAC. The green positioning frame perfectly fits the workpiece edge, its center coordinates completely coincide with the center of the workpiece hole, and the displayed coordinate axis direction (green arrow) matches the actual workpiece deflection angle. Precise alignment. This demonstrates that even in the presence of strong noise and reflective interference, this solution can still provide high-precision guidance data to downstream actuators.

[0040] The present invention also provides a workpiece visual positioning system based on contour key point matching. The system includes a processor and a memory, the memory storing computer program instructions. When the processor executes the computer program instructions, it implements the workpiece visual positioning method based on contour key point matching according to the first aspect of the present invention.

[0041] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and therefore will not be described in detail here.

[0042] In this invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device. Any application or module described in this invention can be implemented using computer-readable / executable instructions stored or otherwise maintained on such a computer-readable medium.

[0043] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A workpiece visual positioning method based on contour key point matching, characterized in that, include: Obtain the current image containing the workpiece, extract the contour of the current image, and extract multiple key points on the contour based on the Harris operator; The stability of each keypoint is calculated, and the stability is positively correlated with the Harris corner response value of the corresponding keypoint and with the local curvature at the keypoint. and the average curvature of the contour points in the neighborhood The absolute values ​​of the differences are negatively correlated and are related to the gradient intensity along the contour normal direction. Positive correlation; Construct the topological association strength between any two key points. This topological association strength is positively correlated with the product of the stability of the two key points and the Euclidean distance between them. The average distance from all key points to the geometric center of the contour is positively correlated with the absolute value of the polar angle difference between two key points in the polar coordinate system with the geometric center as the origin; Calculate the matching weight of each key point. The matching weight is positively correlated with the mean of the topological association strength of the key point and other key points in its neighborhood, the distance from the key point to the geometric center of the contour, and the stability. Based on the matching weight, match the key points of the template image with the key points of the current image, and use the weighted RANSAC algorithm to solve the pose of the workpiece in the current image. In the weighted RANSAC algorithm, the weight of each matching point pair is its corresponding matching weight.

2. The workpiece visual positioning method based on contour key point matching according to claim 1, characterized in that, The stability of each key point is calculated using the following formula: ； In the formula, Indicates the first The stability of each key point; For the first Harris corner response values ​​at key points; The maximum response value of all keypoints in the current image, used for normalization; Indicates the first Local curvature at key points; Indicates the first The average curvature of contour points within the neighborhood of a key point, for example, a neighborhood of size . ; Indicates the first Gradient strength at each key point along the normal direction; It is a very small positive number to prevent the denominator from being zero.

3. The workpiece visual positioning method based on contour key point matching according to claim 1, characterized in that, The formula for calculating the strength of topological association is: ； In the formula, Indicates the first The key point and the first The topological correlation strength of each key point Indicates the first The stability of each key point Indicates the first The stability of each key point Indicates the first The key point and the first Euclidean distance between key points This represents the average distance from all key points to the geometric center of the workpiece. This indicates that in the polar coordinate system with the workpiece's center of gravity as the origin, the first... The key point and the first The absolute value of the polar angle difference at each key point. This represents the adjustment coefficient.

4. The workpiece visual positioning method based on contour key point matching according to claim 1, characterized in that, The formula for calculating the matching weight is: ； In the formula: Indicates the first Matching weights for each key point; Indicates the first The distance from each key point to the geometric center of the profile; Indicates the first The mean of the topological association strength between each key point and other key points in its neighborhood; This represents the average distance from all keypoints to the center. For the first The stability of each key point.

5. The workpiece visual positioning method based on contour key point matching according to claim 1, characterized in that, Extracting the contour of the current image includes: The current image is denoised using a bilateral filter, edges are extracted using the Canny operator, and morphological closing operations are performed on the extracted edges to obtain closed contours.

6. The workpiece visual positioning method based on contour key point matching according to claim 1, characterized in that, After determining the pose of the workpiece in the current image, the following steps are also included: The pose is refined using weighted least squares on the set of interior points selected by the weighted RANSAC algorithm.

7. The workpiece visual positioning method based on contour key point matching according to claim 1, characterized in that, Matching key points of the template image with key points of the current image based on the matching weights includes: The key points of the template image and the current image are sorted in descending order according to the matching weight. The top N key points are selected as seed points and descriptors are generated for them. The descriptors of the seed points are matched, and the matching weight is used as part of the matching cost function.

8. The workpiece visual positioning method based on contour key point matching according to claim 1, characterized in that, The method further includes: Based on the pose control, the robotic arm moves above the workpiece in the current image and controls the end effector to perform rotation compensation to align with the workpiece axis and perform a gripping operation.

9. The workpiece visual positioning method based on contour key point matching according to claim 7, characterized in that, The value of N ranges from 40 to 60.

10. A workpiece visual positioning system based on contour key point matching, comprising a processor and a memory, characterized in that, The memory stores a computer program, and the processor executes the computer program to implement the workpiece visual positioning method based on contour key point matching as described in any one of claims 1-9.

Images