Workpiece positioning correction method and device, computer equipment and storage medium

Abstract

The invention relates to a workpiece positioning correction method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a workpiece depth image of a target workpiece and template feature point information of a template workpiece; according to the workpiece depth image, performing feature point identification on the target workpiece to obtain target feature point information; according to feature point matching information between the target feature point information and the template feature point information, the pose deviation between the target workpiece and the template workpiece is recognized; and performing positioning correction on the target workpiece according to the pose deviation. The method is beneficial to improving the positioning and deviation rectifying accuracy of the workpiece.

Description

Technical Field

[0001] This application relates to the field of industrial vision and intelligent technology, and in particular to a workpiece positioning and correction method, device, computer equipment, and storage medium. Background Technology

[0002] Currently, polygon localization and correction in industrial vision typically employ the following methods: extracting two-dimensional feature points based on traditional image processing methods, or obtaining three-dimensional features based on point clouds. In the matching stage, iterative nearest point algorithm, feature descriptor matching, or geometric template matching are used to calculate the correspondence between the current workpiece position and the template workpiece position.

[0003] However, in the workpiece positioning and correction process of related technologies, feature detection based on traditional image processing methods is prone to missed detections and false detections under complex lighting and surface reflection interference. At the same time, the feature matching method in the feature matching stage is also difficult to deal with matching scenarios with complex structures, partial occlusion and deformation, resulting in low accuracy of workpiece positioning and correction. Summary of the Invention

[0004] Therefore, it is necessary to provide a workpiece positioning and correction method, device, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy of workpiece positioning and correction, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a workpiece positioning correction method, including:

[0006] Acquire the workpiece depth image of the target workpiece and the template feature point information of the template workpiece;

[0007] Based on the workpiece depth image, feature points are identified in the target workpiece to obtain target feature point information;

[0008] Based on the feature point matching information between the target feature point information and the template feature point information, the pose deviation between the target workpiece and the template workpiece is identified.

[0009] The target workpiece is positioned and corrected based on the pose deviation.

[0010] Secondly, this application also provides a workpiece positioning and correction device, comprising:

[0011] The data acquisition module is used to acquire the workpiece image and depth image of the target workpiece, as well as the template feature point information of the template workpiece;

[0012] The feature point information extraction module is used to identify feature points of the target workpiece based on the workpiece image and depth image to obtain target feature point information;

[0013] The pose deviation determination module is used to identify the pose deviation between the target workpiece and the template workpiece based on the feature point matching information between the target feature point information and the template feature point information.

[0014] The positioning correction module is used to correct the positioning of the target workpiece based on the positional deviation.

[0015] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above-described embodiments of the workpiece positioning and correction method.

[0016] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the above-described embodiments of the workpiece positioning and correction method.

[0017] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in any of the above-described embodiments of the workpiece positioning and correction method.

[0018] The aforementioned workpiece positioning and correction method, apparatus, computer equipment, computer-readable storage medium, and computer program product first acquire workpiece images and depth images containing two-dimensional and three-dimensional information of the target workpiece, and acquire template feature point information of the template workpiece used for positioning and correction. Second, based on the workpiece image and depth image, feature point recognition is performed on the target workpiece to obtain target feature point information. Thus, the fusion of two-dimensional and three-dimensional information of the target workpiece for feature point information extraction is beneficial to improving feature detection accuracy under interference scenarios such as changes in illumination and surface reflection, and further, it is beneficial to improve the accuracy of workpiece positioning and correction. Third, based on the target feature point information and the template feature point information, feature point information matching is performed on the target workpiece and the template workpiece, without relying on initial position estimation. Moreover, matching based on feature point information can adapt to scenarios where the target workpiece image is partially occluded, improving the robustness of feature point matching. Finally, based on the feature point matching information, the pose deviation of the target workpiece relative to the template workpiece is determined to drive the positioning and correction of the target workpiece. This realizes automated pose deviation calculation and positioning correction, without relying on expert experience, significantly reducing the workload of parameter adjustment and improving work efficiency. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is an application environment diagram of the workpiece positioning and correction method in one embodiment;

[0021] Figure 2 This is a flowchart illustrating a workpiece positioning and correction method in one embodiment;

[0022] Figure 3 This is a flowchart illustrating the workpiece positioning and correction method in another embodiment;

[0023] Figure 4 This is a flowchart illustrating the workpiece positioning and correction method in yet another embodiment;

[0024] Figure 5 This is a flowchart illustrating the workpiece positioning and correction method in another embodiment;

[0025] Figure 6 This is a structural block diagram of a workpiece positioning and correction device in one embodiment;

[0026] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0028] The workpiece positioning and correction method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown is illustrated. In this environment, the control terminal 102 communicates with the robot controller 104. A data storage system can store the data that the control terminal 102 needs to process. The data storage system can be integrated into the control terminal 102 or placed in the cloud or on another network server.

[0029] Specifically, the operator can upload the workpiece image and depth image of the target workpiece, as well as the template feature point information of the template workpiece, to the control terminal 102 via a terminal. The control terminal 102 acquires the workpiece depth image of the target workpiece and the template feature point information of the template workpiece. Next, based on the workpiece image and depth image, it performs feature point recognition on the target workpiece to obtain target feature point information. Then, based on the feature point matching information between the target feature point information and the template feature point information, it identifies the pose deviation between the target workpiece and the template workpiece. Finally, based on the pose deviation, it performs positioning correction on the target workpiece. The control terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, and tablets.

[0030] In one exemplary embodiment, such as Figure 2 As shown, a workpiece positioning and correction method is provided, which is applied to... Figure 1 Taking control terminal 102 as an example, the explanation includes the following steps (hereinafter referred to as S): S100 to S400. Wherein:

[0031] S100: Obtain the workpiece depth image of the target workpiece and the template feature point information of the template workpiece.

[0032] The target workpiece is determined based on the actual working scenario, and this application does not impose any limitations on it. For example, taking the positioning and correction scenario of automotive parts as an example, the target workpiece can be various types of automotive parts.

[0033] The template workpiece can be selected from the target workpiece and positioned by the robot driven by the teach-controlled robot controller.

[0034] The template feature point information of the template workpiece can include feature points of the template workpiece in the depth image of the template workpiece and the structural features corresponding to those feature points. Feature points include, but are not limited to, edge points and corner points of the template workpiece. For edge points of the template workpiece, edge detection operators can be used to perform edge detection on the template workpiece in the depth image of the template workpiece to obtain the edge points. For corner points, edge detection operators can be used to detect the depth image of the template workpiece to obtain an edge detection map of the depth image of the template workpiece. Subsequently, corner points are identified in the edge detection map using a corner point detection algorithm to obtain the corner points. The corner point detection algorithm can include the Harris corner detection algorithm, the Shi-Tomasi corner detection algorithm, and the scale-invariant feature transform algorithm, etc.

[0035] The structural features corresponding to the feature points may include, but are not limited to, the number of feature points and interior angles. The interior angles can be determined using a vector method, which involves determining the vectors between adjacent feature points and then determining the interior angle based on the angle between the vectors.

[0036] Among them, the template workpiece depth image and the workpiece depth image are obtained by the image acquisition device from the template workpiece and the target workpiece, respectively. The image acquisition device includes, but is not limited to, depth sensors, binocular cameras and structured light cameras.

[0037] In practical applications, the operator can pre-select a template workpiece from the target workpiece, obtain a template workpiece depth image through a depth sensor, and then transmit the template workpiece depth image to the control terminal.

[0038] After acquiring the template workpiece depth image at the control end, image preprocessing is performed on the template workpiece image to reduce noise and enhance edges. Preprocessing may include grayscale conversion, filtering, and denoising. Subsequently, for the template workpiece image, the template workpiece depth image can be detected using an edge detection operator to obtain an edge detection map of the template workpiece depth image. Then, corner points in the edge detection map are identified using a corner detection algorithm to obtain the corner points. Subsequently, the number of corner points is counted and determined as the number of feature points. The vector between adjacent feature points is determined, and the interior angle is determined based on the angle between the vectors. By integrating the corner point coordinates, the number of corner points, and the interior angle, the template feature point information of the template workpiece is obtained.

[0039] In practice, a depth sensor can acquire a workpiece depth image of the target workpiece, and then transmit the workpiece depth image to the control terminal.

[0040] S200: Based on the workpiece depth image, feature point recognition is performed on the target workpiece to obtain target feature point information.

[0041] The target feature point information may include the identified feature points and the structural features corresponding to the feature points.

[0042] In specific implementation, the method for obtaining the target feature point information of the target workpiece can refer to the method for obtaining the template feature point information of the template workpiece in the above embodiment. Specifically, image preprocessing is performed on the target workpiece image to reduce noise and enhance edges. Preprocessing may include grayscale processing, filtering, and noise reduction. Subsequently, for the target workpiece image, the workpiece depth image can be detected by an edge detection operator to obtain an edge detection map of the workpiece depth image. Then, corner points are identified in the edge detection map by a corner detection algorithm to obtain corner points. Subsequently, the number of corner points is counted and determined as the number of feature points. The vector between adjacent feature points is determined, and the interior angle is determined based on the angle between the vectors. The corner point coordinates, the number of corner points, and the interior angle are integrated to obtain the target feature point information of the target workpiece.

[0043] S300 identifies the pose deviation between the target workpiece and the template workpiece based on the feature point matching information between the target feature point information and the template feature point information.

[0044] The feature point matching information includes matching feature point pairs between the target feature point information and the template feature point information.

[0045] In specific implementation, the acquisition of feature point matching information can be achieved by: determining the distance between feature points of the target feature point information, determining the distance between feature points of the template feature point information, and initially screening feature point pairs that meet the range of interior angles based on the interior angles in the structural features of the aforementioned two types of feature points. Subsequently, based on the feature point pairs, feature point matching (such as similarity calculation) is performed based on the distance between feature points of the target feature point information and the distance between feature points of the template feature point information to screen out feature point pairs that meet the distance requirements. The feature point pairs that meet the distance requirements are then determined as matching feature point pairs.

[0046] Subsequently, the coordinate system of the target workpiece can be established by aligning the feature points of the target workpiece with the feature points of the template workpiece, and the coordinate system of the template workpiece can be established by aligning the feature points of the template workpiece with the feature points of the template workpiece. Then, the pose of the target workpiece and the pose of the template workpiece can be determined based on the transformation matrix between the two coordinate systems, and then the pose deviation can be calculated.

[0047] S400 performs positioning correction on the target workpiece based on the positional deviation.

[0048] In practice, the pose deviation is converted into robot control commands, which are then sent to the robot controller to drive the robot to position and correct the target workpiece.

[0049] In the aforementioned workpiece positioning and correction method, firstly, workpiece images and depth images containing two-dimensional and three-dimensional information of the target workpiece are acquired, and template feature point information of the template workpiece used for positioning and correction is obtained. Secondly, based on the workpiece image and depth image, feature point recognition is performed on the target workpiece to obtain target feature point information. Thus, the fusion of two-dimensional and three-dimensional information of the target workpiece for feature point information extraction is beneficial to improving the feature detection accuracy under interference scenarios such as changes in illumination and surface reflection, and further, it is beneficial to improve the accuracy of workpiece positioning and correction. Thirdly, based on the target feature point information and the template feature point information, feature point information matching is performed on the target workpiece and the template workpiece. This does not rely on initial position estimation, and matching based on feature point information can adapt to scenarios where the target workpiece image is partially occluded, improving the robustness of feature point matching. Finally, based on the feature point matching information, the pose deviation of the target workpiece relative to the template workpiece is determined to drive the positioning and correction of the target workpiece. This realizes automated pose deviation calculation and positioning correction, without relying on expert experience, significantly reducing the workload of parameter adjustment and improving work efficiency.

[0050] In one exemplary embodiment, such as Figure 3 As shown, based on the workpiece depth image, feature point recognition is performed on the target workpiece, obtaining target feature point information including S201 to S203. Wherein:

[0051] S201, Extract the contour features of the target workpiece from the workpiece depth image.

[0052] In practice, the workpiece depth image can be preprocessed first to reduce interference and improve the accuracy of feature point extraction. Image preprocessing includes grayscale conversion, filtering, and binarization. For example, the workpiece depth image is grayscale converted, then smoothed using a Gaussian filter kernel, and finally binarized using an adaptive threshold binarization algorithm. The Gaussian kernel size is [value missing]. Standard deviation This effectively eliminates noise interference. The adaptive threshold calculation formula is as follows:

[0053]

[0054] in: An adaptive threshold; The mean of a local region in the image; The standard deviation of a local region in the image; This is the adjustment factor, with a default value of 1.2.

[0055] Subsequently, for the preprocessed workpiece depth image, edge detection can be performed using an edge detection operator. Then, a contour tracking algorithm is used to extract contour features from the detected edge features. Preferably, edge detection is performed using the ED (Edge Drawing) algorithm: First, the gradient magnitude and gradient direction of each pixel in the workpiece depth image are determined using the Sobel operator. Then, anchor points are detected from the pixels based on the gradient magnitude. Next, the anchor points are connected based on the continuity of the gradient direction to obtain the edge features. Afterward, the detected edge features are filled with gaps using a morphological closing operation, and the contour tracking algorithm is used to extract the contour features of the target workpiece in the workpiece depth image.

[0056] In other implementations, edge detection of the workpiece depth image can be performed using the Canny operator, with edge continuity achieved by adjusting the threshold parameter; it can also be based on the Sobel operator combined with morphological operations, obtaining continuous edge features through gradient calculation and edge connection; or it can be based on deep learning edge detection networks (such as HED (Holistically-Nested Edge Detection) or RCF (Richer Convolutional Features)) to detect edge features of the workpiece depth image.

[0057] It is understandable that the method of obtaining the template feature point information of the template workpiece can be the same as that of the target feature point information of the target workpiece, which will not be elaborated here.

[0058] S202, Determine the target contour features of the target workpiece based on the geometric parameters of the contour features and the depth image.

[0059] The target contour feature can be a polygonal contour feature, such as a quadrilateral or hexagonal contour feature. The geometric parameters of the contour feature characterize the geometric and shape features of the polygon. For example, the geometric parameters characterizing the geometric features can include area and perimeter, and the geometric parameters characterizing the shape features can include the number of vertices, interior angles, and aspect ratio.

[0060] In practice, the first step is to obtain the point cloud coordinates corresponding to the contour features from the depth image, and then convert the two-dimensional coordinates in the contour features into three-dimensional point cloud coordinates. Subsequently, for the converted contour features, the corner points of the contour features are identified, and based on the corner points, the area, perimeter, number of vertices, interior angles, and aspect ratio of the contour features are determined.

[0061] Secondly, contour features whose vertex count does not match the preset target value (matching the number of polygon vertices) are removed from multiple contour features. Further, contour features that meet at least three of the following selection criteria are selected from multiple contour features and determined as target contour features: (1) the area of ​​the contour feature is greater than the preset area threshold; (2) the perimeter of the contour feature is greater than the preset perimeter threshold; (3) all interior angles of the contour feature are within the preset interior angle threshold range; (4) the aspect ratio is within the preset aspect ratio threshold range.

[0062] S203, extract multiple feature points from the target contour features, and determine the multiple feature points and the geometric features between the multiple feature points as target feature point information.

[0063] Among them, geometric features may include, but are not limited to, the angle between diagonals, the direction of the normal vector, the length ratio of adjacent sides, and the topological connection information between each feature point.

[0064] In practical implementation, the process can involve identifying the corner points of the target contour features and defining them as feature points of the target workpiece. Then, the order of the corner points (clockwise) is determined through the cross product of their vectors. Regarding the length ratio of adjacent sides, the length of the side corresponding to the target contour feature can be determined based on the corner point order and its point cloud coordinates, further determining the length ratio of adjacent sides. For the diagonal angle, the corner points at both ends of the diagonal can be determined based on the corner point order, and the diagonal angle can be determined using a vector method. Regarding the normal vector direction, the normal vector of each corner point can be determined, and the average of the normal vectors can be used to determine the normal vector direction. Regarding the topological connection information between feature points, topological connections can be established between the corner points based on their order to obtain the topological connection information.

[0065] It is understandable that the method of obtaining the template feature point information of the template workpiece is the same as the method of obtaining the target feature point information of the target workpiece, so it will not be elaborated here.

[0066] In this embodiment, feature points are extracted by filtering the target contour features and extracting the geometric features between the feature points. This is beneficial for feature point matching by combining the feature points and the geometric features between the feature points. Compared with traditional feature matching methods, this is beneficial for improving the feature detection accuracy under strong reflective and partial occlusion conditions.

[0067] In one exemplary embodiment, the number of contour features is multiple, such as... Figure 4 As shown, based on the geometric parameters of the contour features and the depth image, the target contour features of the target workpiece are determined, including S220 to S240, wherein:

[0068] S220: Based on the shape feature parameters of multiple contour features, candidate contour features are selected from multiple contour features.

[0069] In this embodiment, the target contour feature is a quadrilateral contour feature. Shape feature parameters may include the number of vertices, interior angles, and aspect ratio, etc.

[0070] In practice, the extracted contour features undergo multi-level filtering to obtain candidate contour features representing quadrilateral contours. Specifically, firstly, contour features whose geometric feature parameters do not meet preset requirements are eliminated from multiple contour features: the area and perimeter of each contour feature are determined, and the geometric feature parameters include area and perimeter. Contour features with an area smaller than a preset area threshold (e.g., 100 pixels) or a perimeter smaller than a preset perimeter threshold are eliminated from multiple contour features. Understandably, the area threshold and perimeter threshold are set according to the actual operating scenario.

[0071] Subsequently, filtering is performed based on the number of vertices: for each contour feature, corner points in the contour feature can be identified, and the number of corner points can be counted to obtain the number of vertices. Contour features whose number of vertices matches the target value (e.g., the target value is 4) are filtered from multiple contour features. The method for identifying corner points can refer to the implementation steps for detecting corner points in the above embodiment.

[0072] Finally, structural verification is performed on the selected contour features based on their interior angles and aspect ratio. Contour features that pass structural verification are identified as candidate contour features. Specifically, each interior angle of the contour feature can be determined using a vector method, and the aspect ratio can be determined based on the distance between the identified vertices. Then, it is verified whether the interior angles of the contour feature are within a preset interior angle threshold range, and whether the aspect ratio of the contour feature is within a preset aspect ratio threshold range. If both the interior angles and aspect ratio of the contour feature are within the preset interior angle threshold range, then the contour feature is determined to have passed structural verification, and the contour features that pass structural verification are identified as candidate contour features.

[0073] S240: Based on the depth image, determine the candidate contour point cloud corresponding to the candidate contour feature, and perform polygon point cloud fitting based on the candidate contour point cloud to obtain the target contour feature.

[0074] In this embodiment, the selected candidate contour features are mapped to three-dimensional space to obtain target contour features containing three-dimensional contour points. Specifically, firstly, point cloud extraction is performed on the workpiece depth image based on the candidate contour features to obtain the candidate contour point cloud corresponding to each contour point in the candidate contour features.

[0075] Subsequently, the minimum bounding moments of multiple candidate contour point clouds are determined to define the initial spatial range. Then, a plane fitting algorithm is used to fit the candidate contour point clouds to a plane, obtaining a fitting plane. Preferably, the PEAC (Progressive Edge-Aware Clustering) plane fitting algorithm is used to fit the candidate contour point clouds to a plane. This algorithm improves the plane fitting accuracy through an edge-aware clustering strategy. Specifically, the candidate contour point clouds are clustered based on point cloud density to obtain multiple clusters. The cluster boundaries are optimized according to preset edge constraints, and outliers are removed using the RANSAC strategy. The fitting plane is obtained based on the center point clouds of the clusters.

[0076] In other embodiments, the methods for performing plane fitting on the candidate contour point cloud include, but are not limited to, RANSCA-based plane fitting, least squares-based plane fitting, and principal component analysis-based plane fitting.

[0077] Then, the candidate contour point cloud is projected onto the fitting plane to eliminate the influence of measurement noise, thus obtaining the projected point cloud of the candidate contour point cloud on the fitting plane.

[0078] Next, quadrilateral fitting is performed on the projected point cloud to obtain the target contour features. If quadrilateral fitting fails, the convex hull of the projected point cloud is calculated, that is, the smallest convex polygon contour feature containing all projected point clouds is determined, and the positioning correction is performed based on the smallest convex polygon contour feature.

[0079] In this embodiment, two-dimensional and three-dimensional features are fused for feature detection, which improves the detection stability of features under varying lighting conditions and surface reflection.

[0080] In an exemplary embodiment, the target feature point information includes a plurality of first feature points and a first geometric feature between the plurality of first feature points, and the template feature point information includes a plurality of second feature points and a second geometric feature between the plurality of second feature points. Figure 5 As shown, the methods for obtaining feature point matching information between target feature point information and template feature point information include S301 to S303, wherein:

[0081] S301, determine the first distance between multiple first feature points and the second distance between multiple second feature points respectively.

[0082] S302, based on the first distance and the second distance, select multiple candidate feature point pairs from multiple first feature points and multiple second feature points.

[0083] S303, based on the first and second geometric features of multiple candidate feature point pairs, select the target feature point pair from the multiple candidate feature point pairs, and the feature point matching information includes the target feature point pair.

[0084] In this embodiment, to improve the adaptability of feature matching to different scenarios, an improved maximum common subgraph algorithm is used for feature point matching. Specifically, step 1: Determine the distance between feature points: For the first feature point set C, determine the Euclidean distance between each pair of first feature points, and use the Euclidean distance as the first distance. Integrate the first distances to obtain the first distance matrix D_C. For the second feature point set T, determine the Euclidean distance between each pair of second feature points, and use the Euclidean distance as the second distance. Subsequently, integrate the second distances to obtain the second distance matrix D_T.

[0085] Step 2: Initialize the target feature point pair set M as an empty set.

[0086] Step 3: Based on the first and second distance matrices, recursively execute steps 3.1 to 3.3 to search for the maximum common subgraph:

[0087] Step 3.1: For any pair of feature points (Including the first and second feature points), determine the distance difference between it and the matched point pairs in the target feature point pair set, and verify that the distance difference value is less than the preset distance tolerance threshold:

[0088]

[0089] in, Distance difference value; Absolute value operation; First feature point The first feature point that has been matched with the target feature point pair set The Euclidean distance between them; Second feature point The second feature point that has been matched with the target feature point pair set The Euclidean distance between them; Distance tolerance threshold (unit: millimeters).

[0090] If the distance difference value corresponding to the feature point pair is less than the preset distance tolerance threshold, the feature point pair is determined as a candidate feature point pair.

[0091] Step 3.2: Verify the consistency of geometric features between the candidate feature point pairs and the matched feature point pairs in the target feature point pair set. First, for the first feature point among the matched feature points in the candidate and target feature point pairs sets, determine the corresponding geometric features: the length ratio of adjacent sides, the angle between diagonals, the direction of the normal vector, and topological connectivity information. Second, for the second feature point among the matched feature points in the candidate and target feature point pairs sets, determine the corresponding geometric state: the length ratio of adjacent sides, the angle between diagonals, the direction of the normal vector, and topological connectivity information.

[0092] If the differences between the geometric features corresponding to the first feature point and the geometric features corresponding to the second feature point both meet the preset requirements, it is determined that the geometric features of the candidate feature point pair are consistent with those of the matched feature point pairs in the target feature point pair set. The candidate feature point pair is then identified as a matching target feature point pair and added to the target feature point pair set M.

[0093] Step S3.3, Pruning optimization: If the number of remaining unmatched first feature points in the current branch is less than the maximum number of matched target feature point pairs in other branches, then stop the recursive search of the current branch.

[0094] In this embodiment, feature matching is performed using an improved maximum common subgraph algorithm. On the one hand, this improves the efficiency of feature matching; on the other hand, it does not rely on initial position estimation and can adapt to scenarios where the image of the target workpiece is partially occluded, thus improving the robustness of feature point matching.

[0095] In an exemplary embodiment, the feature point matching information includes matching pairs of target feature points in the target feature point information and the template feature point information. Based on the feature point matching information between the target feature point information and the template feature point information, the pose deviation between the target workpiece and the template workpiece is identified, including steps S304 to S306, wherein:

[0096] S304, Based on the target feature point pairs, construct the first coordinate system of the target workpiece and the second coordinate system of the template workpiece respectively.

[0097] In this embodiment, the target feature point pairs are described as matching corner point pairs.

[0098] In specific implementation, a first coordinate system for the target workpiece is constructed for the first feature point in the matching target feature point pair, and a second coordinate system for the target workpiece is constructed for the second feature point in the matching target feature point pair. For example, for the first feature point in the target feature point pair, the centroid of the first feature point is determined, and this centroid is defined as the origin of the first coordinate system:

[0099]

[0100] in, These are the coordinates of the corner point. This represents the number of corner points.

[0101] Subsequently, the normal vector of each first feature point is determined, the mean vector of each normal vector is determined, and the mean vector is normalized to obtain the Z-axis of the first coordinate system:

[0102]

[0103] in, These are the normal vectors for each corner point.

[0104] Next, the mean vector of the line vectors at the left and right symmetrical corner points is determined, and the mean vector is normalized to obtain the X-axis of the first coordinate system:

[0105]

[0106] in, The left corner point, The right corner point, The number of corner points is the logarithm.

[0107] Finally, by determining the cross product of the X and Z axes, we obtain the Y axis, which satisfies the right-handed coordinate system:

[0108]

[0109] It is understandable that the implementation method of constructing the second coordinate system based on the second feature point in the target feature point pair is the same as the implementation method of constructing the first coordinate system based on the first feature point described above, and will not be repeated here.

[0110] S305, determine the transformation parameters between the first coordinate system and the second coordinate system.

[0111] In practice, let the first coordinate system be C_frame and the second coordinate system be T_frame. Based on the relative relationship between the first and second coordinate systems, calculate the transformation matrix (transformation parameters):

[0112]

[0113] The transformation matrix is ​​decomposed into: Rotation matrix and Translation vector :

[0114]

[0115] S306, Based on the transformation parameters and the target feature point pair, determine the pose deviation of the target workpiece relative to the template workpiece.

[0116] Among them, pose deviation includes translation deviation and rotation deviation.

[0117] In practice, the coordinate difference between the first and second feature points in the target feature point pair can be determined, and this coordinate difference can be defined as the translational deviation. The translational deviation can then be directly sent to the robot controller as a Cartesian space displacement.

[0118]

[0119] Subsequently, the rotational deviation was determined using the axis angle representation method:

[0120]

[0121]

[0122] In this embodiment, the pose deviation is determined by two coordinate systems, which is beneficial for driving the positioning correction of the target workpiece based on the pose deviation.

[0123] In an exemplary embodiment, before identifying the pose deviation between the target workpiece and the template workpiece based on the feature point matching information between the target feature point information and the template feature point information, the method further includes:

[0124] The reprojection error is determined based on the transformation parameters and the coordinates of the target feature point pairs.

[0125] Among them, the reprojection error characterizes the coordinate error between the feature points of the template workpiece and the feature points of the target workpiece.

[0126] In practice, the reprojection error is determined using the following formula:

[0127]

[0128] in, The coordinates of the second feature point; The coordinates of the first feature point; For rotation matrix, It is a translation vector; This represents the number of target feature point pairs to be matched.

[0129] If the reprojection error is greater than the preset reprojection error threshold, it is determined that the feature point matching information indicates that the feature points do not match, and the process returns to the step of identifying the feature points of the target workpiece based on the workpiece image and depth image to obtain the target feature point information, until the reprojection error is less than or equal to the preset reprojection error threshold.

[0130] The reprojection error threshold is set according to the positioning accuracy requirements.

[0131] In practice, the reprojection error is compared with a preset reprojection error threshold. If the reprojection error is greater than the preset reprojection error threshold (e.g., 3.0 mm), it is determined that the feature points between the target workpiece and the template workpiece do not match, and the process returns to S200 to re-perform feature detection and matching.

[0132] In this embodiment, the accuracy of feature matching is improved by evaluating the reprojection error.

[0133] To provide a clearer explanation of the workpiece positioning and correction method provided in this application, a specific embodiment is described below, which includes the following steps:

[0134] S1, acquire the workpiece depth image of the target workpiece and the template feature point information of the template workpiece.

[0135] S2, extract the contour features of the target workpiece from the workpiece depth image.

[0136] S3. Based on the shape feature parameters of multiple contour features, candidate contour features are selected from multiple contour features. Based on the depth image, the candidate contour point cloud corresponding to the candidate contour feature is determined. Based on the candidate contour point cloud, polygon point cloud fitting is performed to obtain the target contour feature.

[0137] S4 extracts multiple feature points from the target contour features and determines the multiple feature points and the geometric features between them as target feature point information.

[0138] S5, the target feature point information includes multiple first feature points and the first geometric features between the multiple first feature points, and the template feature point information includes multiple second feature points and the second geometric features between the multiple second feature points.

[0139] S6, determine the first distance between multiple first feature points and the second distance between multiple second feature points respectively, select multiple candidate feature point pairs from multiple first feature points and multiple second feature points according to the first distance and the second distance, select target feature point pairs from multiple candidate feature point pairs according to the first geometric features and the second geometric features of multiple candidate feature point pairs, and the feature point matching information includes the target feature point pairs.

[0140] S7. Based on the target feature point pair, construct the first coordinate system of the target workpiece and the second coordinate system of the template workpiece respectively, determine the transformation parameters between the first coordinate system and the second coordinate system, and determine the pose deviation of the target workpiece relative to the template workpiece based on the transformation parameters and the target feature point pair.

[0141] S8. Based on the transformation parameters and the coordinates of the target feature point pair, determine the reprojection error. If the reprojection error is greater than the preset reprojection error threshold, determine that the feature point matching information indicates that the feature points do not match, and return to S2 until the reprojection error is less than or equal to the preset reprojection error threshold.

[0142] S9, based on the positional deviation, performs positioning correction on the target workpiece.

[0143] The beneficial effects of this embodiment, as tested, include:

[0144] (1) Significantly improve the real-time performance of the algorithm: Existing ICP and RANSAC algorithms have high computational complexity and are difficult to meet the requirements of high-speed production lines. This embodiment reduces the average processing time from more than 200ms to less than 100ms through a multi-strategy adaptive matching mechanism and a dual pruning strategy in the MCS algorithm, achieving a real-time response of more than 10Hz.

[0145] (2) Significantly enhanced environmental adaptability: Traditional methods have poor stability under varying lighting conditions and surface reflectivity. This embodiment uses 2D / 3D fusion feature detection (ED edge detection + PEAC plane fitting) and topological consistency constraints, which improves the feature detection success rate from 60% to over 90% under strong reflectivity and partial occlusion conditions.

[0146] (3) Achieving sub-millimeter positioning accuracy: Existing technologies are limited in accuracy due to insufficient 2D / 3D information fusion. According to tests, this embodiment achieves a repeatability accuracy of 0.3mm and an absolute positioning accuracy of 0.5mm by using a projection optimization mechanism and topology-based MCS matching, combined with reprojection error assessment.

[0147] (4) Effectively reduce usage and maintenance costs: Traditional methods rely on expert experience for parameter tuning. This embodiment adopts adaptive threshold binarization and automatic contour screening mechanism, which reduces the workload of parameter tuning by more than 60%, and ordinary operators can complete daily maintenance, greatly reducing the reliance on professional personnel.

[0148] (5) Significantly improve system robustness: Existing technologies are sensitive to initial position and are prone to failure under large deviations. The MCS algorithm based on the distance matrix in this embodiment does not rely on initial position estimation and, with the help of multiple alternative schemes, still maintains a matching success rate of over 85% under large position deviations (rotation > 30°).

[0149] (6) Extend the service life of the system: According to the test, through online quality monitoring and automatic template update mechanism, the mean time between failures in this embodiment is extended from 3 months to more than 12 months, which significantly reduces maintenance costs and downtime.

[0150] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0151] In one exemplary embodiment, such as Figure 6 As shown, a workpiece positioning and correction device 600 is provided, including: a data acquisition module 610, a feature point information extraction module 620, a pose deviation determination module 630, and a positioning correction module 640, wherein:

[0152] The data acquisition module 610 is used to acquire the workpiece image and depth image of the target workpiece, as well as the template feature point information of the template workpiece;

[0153] The feature point information extraction module 620 is used to identify feature points of the target workpiece based on the workpiece image and depth image to obtain target feature point information;

[0154] The pose deviation determination module 630 is used to identify the pose deviation between the target workpiece and the template workpiece based on the feature point matching information between the target feature point information and the template feature point information.

[0155] The positioning correction module 640 is used to perform positioning correction on the target workpiece based on the positional deviation.

[0156] In an exemplary embodiment, the feature point information extraction module 620 is further configured to extract the contour features of the target workpiece from the workpiece depth image; determine the target contour features of the target workpiece based on the geometric parameters of the contour features and the depth image; extract multiple feature points from the target contour features; and determine the multiple feature points and the geometric features between the multiple feature points as target feature point information.

[0157] In an exemplary embodiment, the feature point information extraction module 620 is further configured to: filter candidate contour features from multiple contour features based on shape feature parameters of multiple contour features; determine candidate contour point clouds corresponding to the candidate contour features based on the depth image; and perform polygon point cloud fitting based on the candidate contour point clouds to obtain the target contour features.

[0158] In an exemplary embodiment, the workpiece positioning and correction device 600 further includes a feature matching and determination module 650, which is used to determine a first distance between a plurality of first feature points and a second distance between a plurality of second feature points; to select a plurality of candidate feature point pairs from the plurality of first feature points and the plurality of second feature points based on the first distance and the second distance; and to select a target feature point pair from the plurality of candidate feature point pairs based on the first geometric features and the second geometric features of the plurality of candidate feature point pairs, wherein the feature point matching information includes the target feature point pair.

[0159] In an exemplary embodiment, the pose deviation determination module 630 is further configured to construct a first coordinate system of the target workpiece and a second coordinate system of the template workpiece based on the target feature point pair; determine the transformation parameters between the first coordinate system and the second coordinate system; and determine the pose deviation of the target workpiece relative to the template workpiece based on the transformation parameters and the target feature point pair.

[0160] In an exemplary embodiment, the workpiece positioning correction device 600 further includes a reprojection error determination module 650, which is used to determine the reprojection error based on the transformation parameters and the coordinates of the target feature point pair; if the reprojection error is greater than a preset reprojection error threshold, it is determined that the feature point matching information indicates that the feature points do not match, and returns to the step of performing feature point recognition on the target workpiece based on the workpiece depth image to obtain the target feature point information, until the reprojection error is less than or equal to the preset reprojection error threshold.

[0161] Each module in the aforementioned workpiece positioning and correction device 600 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0162] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When executed by the processor, the computer program implements a workpiece positioning and correction method.

[0163] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0164] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above embodiments of the workpiece positioning and correction method.

[0165] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps in any of the above embodiments of the workpiece positioning and correction method.

[0166] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the workpiece positioning and correction method.

[0167] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0168] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0169] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0170] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A workpiece positioning and correction method, characterized in that, The method includes: Acquire the workpiece depth image of the target workpiece and the template feature point information of the template workpiece; Based on the workpiece depth image, feature point recognition is performed on the target workpiece to obtain target feature point information; Based on the feature point matching information between the target feature point information and the template feature point information, the pose deviation between the target workpiece and the template workpiece is identified; The target workpiece is positioned and corrected based on the positional deviation.

2. The method according to claim 1, characterized in that, The step of identifying feature points of the target workpiece based on the workpiece depth image to obtain target feature point information includes: Extract the contour features of the target workpiece from the workpiece depth image; The target contour features of the target workpiece are determined based on the geometric parameters of the contour features and the depth image. Multiple feature points are extracted from the target contour features, and the multiple feature points and the geometric features between the multiple feature points are determined as target feature point information.

3. The method according to claim 2, characterized in that, The number of contour features is multiple; determining the target contour features of the target workpiece based on the geometric parameters of the contour features and the depth image includes: Candidate contour features are selected from the multiple contour features based on the shape feature parameters of the multiple contour features; Based on the depth image, a candidate contour point cloud corresponding to the candidate contour feature is determined, and polygon point cloud fitting is performed based on the candidate contour point cloud to obtain the target contour feature.

4. The method according to claim 1, characterized in that, The target feature point information includes multiple first feature points and a first geometric feature between the multiple first feature points; the template feature point information includes multiple second feature points and a second geometric feature between the multiple second feature points. The methods for obtaining the feature point matching information between the target feature point information and the template feature point information include: Determine the first distance between multiple first feature points and the second distance between multiple second feature points respectively; Based on the first distance and the second distance, multiple candidate feature point pairs are selected from multiple first feature points and multiple second feature points; Based on the first geometric features and the second geometric features of the multiple candidate feature point pairs, a target feature point pair is selected from the multiple candidate feature point pairs; The feature point matching information includes the target feature point pair.

5. The method according to claim 1, characterized in that, The feature point matching information includes the target feature point information and the template feature point information that match the target feature point pairs; The step of identifying the pose deviation between the target workpiece and the template workpiece based on the feature point matching information between the target feature point information and the template feature point information includes: Based on the target feature point pairs, construct the first coordinate system of the target workpiece and the second coordinate system of the template workpiece respectively; Determine the transformation parameters between the first coordinate system and the second coordinate system; Based on the transformation parameters and the target feature point pair, the pose deviation of the target workpiece relative to the template workpiece is determined.

6. The method according to claim 5, characterized in that, Before identifying the pose deviation between the target workpiece and the template workpiece based on the feature point matching information between the target feature point information and the template feature point information, the method further includes: The reprojection error is determined based on the transformation parameters and the coordinates of the target feature point pair. If the reprojection error is greater than a preset reprojection error threshold, it is determined that the feature point matching information indicates that the feature points do not match, and the process returns to the step of identifying the target workpiece based on the workpiece depth image to obtain the target feature point information, until the reprojection error is less than or equal to the preset reprojection error threshold.

7. A workpiece positioning and correction device, characterized in that, The device includes: The data acquisition module is used to acquire the workpiece image and depth image of the target workpiece, as well as the template feature point information of the template workpiece; The feature point information extraction module is used to identify feature points of the target workpiece based on the workpiece image and the depth image to obtain target feature point information; The pose deviation determination module is used to identify the pose deviation between the target workpiece and the template workpiece based on the feature point matching information between the target feature point information and the template feature point information; The positioning correction module is used to perform positioning correction on the target workpiece based on the posture deviation.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images