Vision-based contact measuring robot adaptive measurement point relative guidance method

Abstract

The application discloses a visual-based contact type measuring robot adaptive measuring point relative guiding method, which comprises a 2D visual-based product image contour matching method, realizes 4-degree-of-freedom measuring point guiding of plane translation, rotation and height scaling, and is suitable for contact type six-axis cooperative measuring robot measuring point guiding measurement of products with local features; the method comprises the following steps: teaching contact type six-axis cooperative measuring robot measuring points of a product to be measured; determining a teaching 2D measuring pose, obtaining a product multi-scale contour image template under the teaching 2D measuring pose and a multi-scale image of the current product, and performing contour matching of corresponding scale levels; based on the contour matching result, calculating a first pose deviation under the teaching 2D measuring pose, transforming the first pose deviation to all teaching point positions of the teaching 2D, performing pose transformation, performing iterative updating on all measuring point positions, and obtaining all updated first measuring point poses. The application better realizes adaptive contact type measuring point measurement of products.

Description

Technical Field

[0001] This invention relates to the fields of robotic contact measurement and automated assembly, and specifically to a vision-based adaptive relative guidance method for contact measurement robots. Background Technology

[0002] Due to its characteristics of automation, high human-machine collaboration, large spatial operating range, high flexibility, high precision, strong product adaptability, and strong environmental adaptability, contact six-axis collaborative measurement robots have been researched and applied in automated docking and assembly processes. The measurement objects of contact six-axis collaborative measurement are generally products with features such as planes, edges, through holes, and screw holes. However, due to the large variations in planar or spatial pose of the incoming product and the small size of the feature to be measured, as well as the multi-degree-of-freedom variation of the product's pose, problems such as incorrect contact measurement point positioning, unmeasurable features, probe collisions, and even collisions between the contact six-axis collaborative measurement robot and the product or posture adjustment mechanism are easily caused. Existing methods either use absolute positioning guidance, but the absolute positioning accuracy of the robot is not high, resulting in low guidance accuracy; or they use machine vision planar guidance, but can only achieve guidance in three degrees of freedom in a planar plane. Compared with existing methods, contact six-axis collaborative measurement robot measurement point guidance not only requires higher measurement accuracy but also places higher demands on computational efficiency and measurement real-time performance.

[0003] In view of the above, this application is hereby submitted. Summary of the Invention

[0004] The technical problem to be solved by the present invention is that the existing measurement point guidance methods are prone to problems such as incorrect position of contact measurement points, inability to measure, and low guidance accuracy due to the large changes in the planar or spatial pose of the incoming product and the small size of the feature to be measured, as well as the many degrees of freedom of the product pose.

[0005] The purpose of this invention is to provide a vision-based adaptive relative guidance method for contact measurement robots. Based on the teaching of measurement point poses, the method acquires the pose changes of the product using 2D and 3D vision, and updates the measurement point poses according to these changes, thus achieving adaptive contact measurement of the product. This invention employs small-space relative guidance relative to the taught points, resulting in higher guidance accuracy; it utilizes multi-scale contour matching and multi-scale downsampling point cloud matching methods, providing better real-time measurement performance and enabling more effective adaptive contact measurement of the product, thus possessing broad engineering application potential.

[0006] This invention is achieved through the following technical solution:

[0007] In a first aspect, the present invention provides a vision-based adaptive measurement point relative guidance method for a contact measurement robot. This method includes: achieving four degrees of freedom (DOF) measurement point guidance (planar translation, rotation, and height scaling) through a 2D vision-based product image contour matching method. Planar translation includes translation along the X-axis and along the Y-axis; rotation refers to rotation around the Z-axis; and height scaling refers to scaling in the Z-axis direction. This method is applicable to measurement point guidance measurement of contact six-axis collaborative measurement robots for products with local features.

[0008] The contact six-axis collaborative measurement robot is used to teach the measurement points of the product under test, and the position and pose of each contact measurement point and the point-to-point path of the robot are determined.

[0009] Determine the teaching 2D measurement pose, obtain the product multi-scale contour image template under the teaching 2D measurement pose and the current product multi-scale image, and perform contour matching at the corresponding scale level;

[0010] Based on the contour matching results, the first pose deviation under the teaching 2D measurement pose is calculated. The first pose deviation is then transformed to all teaching points in the teaching 2D for pose transformation. All measurement point positions are traversed and updated to obtain the pose of all updated first measurement points.

[0011] In planar measurement of products, the product has four degrees of freedom: translation, rotation, and height scaling. To address issues such as mis-touching and unmeasurable measurement points caused by large translation and rotation angles, this invention proposes an adaptive planar measurement point guidance method based on a 2D vision-based six-axis collaborative measurement robot. First, the product to be measured is taught to its measurement points using a six-axis collaborative measurement robot. Second, the taught 2D measurement pose is determined, and a multi-scale contour image template of the product under this taught 2D measurement pose and the current multi-scale image of the product are obtained, followed by contour matching at the corresponding scale levels. Finally, the first pose deviation under the taught 2D measurement pose is calculated, and this first pose deviation is transformed to all taught points in the 2D model for pose transformation. All measurement point positions are traversed and updated to obtain the updated first measurement point pose. This method is applicable to point-guided measurement of products with local features using a six-axis collaborative measurement robot.

[0012] In one possible implementation, the product has six degrees of freedom (translation, rotation, and attitude) in spatial measurement. To address issues such as mis-touching and unmeasurable measurement points caused by large translation, rotation, and attitude angles, this invention also proposes an adaptive spatial measurement point guidance method based on a 3D vision-based six-axis collaborative measurement robot. This method further includes: using a 3D vision-based product feature point cloud matching method to achieve six-degree-of-freedom measurement point guidance for spatial translation, rotation, and attitude. Spatial translation includes translation along the X-axis, Y-axis, and Z-axis; rotation refers to spatial rotation around the Z-axis; and attitude includes spatial rotation around the X-axis and Y-axis.

[0013] The contact six-axis collaborative measurement robot is used to teach the measurement points of the product under test, and the position and pose of each contact measurement point and the point-to-point path of the robot are determined.

[0014] Determine the teaching 3D measurement pose, obtain the multi-scale downsampled point cloud template under the teaching 3D measurement pose and the current product multi-scale downsampled point cloud, and perform point cloud matching at the corresponding scale level;

[0015] Based on the point cloud matching results, the second pose deviation under the teaching 3D measurement pose is calculated. Based on the second pose deviation, the pose transformation is performed on all teaching points in the teaching 3D to obtain the updated second measurement point pose.

[0016] The above technical solution and method are applicable to point-guided measurement of contact six-axis collaborative measurement robots for products without features or with local features.

[0017] In one possible implementation, a 2D teaching measurement pose is determined, and a multi-scale contour image template of the product under the 2D teaching measurement pose and a multi-scale image of the current product are obtained. Contour matching at the corresponding scale level is then performed, including:

[0018] Once the 2D measurement pose is determined, the contact six-axis collaborative measurement robot acquires a clear image of the product's local features under this 2D measurement pose.

[0019] Multi-scale contour matching is performed on the local feature image of the product to obtain a multi-scale image contour template.

[0020] After the current product pose changes, the contact six-axis collaborative measurement robot is moved to the teaching 2D measurement pose to obtain the product image;

[0021] Perform multi-scale contour matching processing on product images to obtain multi-scale image contours;

[0022] Contour matching is performed between multi-scale image contour templates and multi-scale image contours at corresponding scale levels, with the contour matching scale levels proceeding from high to low.

[0023] If the contour matching degree is less than the image contour matching threshold, the scale level is reduced and the contour matching step is repeated until the image contour matching threshold is met; if the image contour matching threshold cannot be reached after traversing all scale levels, the process exits.

[0024] In one possible implementation, multi-scale contour matching processing is performed on the local feature image of the product to obtain a multi-scale image contour template, including:

[0025] The first region of interest (ROI) is obtained by extracting the local feature image of the product.

[0026] Based on the first region of interest, connected component calculation and filtering are performed using dynamic threshold segmentation, and noise points are removed to obtain feature contours and form multi-scale image contour templates.

[0027] In one possible implementation, based on the contour matching result, the first pose deviation under the taught 2D measurement pose is calculated. This first pose deviation is then transformed to all taught points in the taught 2D model for pose transformation. All measurement point positions are iterated and updated to obtain the poses of all updated first measurement points, including:

[0028] Based on the contour matching results that meet the image contour matching threshold, calculate the translation, rotation, and scaling transformation from the multi-scale image contour template to the current image, i.e., the affine transformation.

[0029] Based on affine transformation, the flange pose deviation of the measuring robot under the taught 2D measurement pose is calculated as the first pose deviation.

[0030] Based on the first pose deviation and the taught 2D measurement pose, pose transformation is performed on all taught measurement points to obtain the updated first measurement point pose.

[0031] The contact-type six-axis collaborative measurement robot moves along the taught path to the updated pose of the first measurement point and performs planar measurement point acquisition, thereby realizing planar measurement point guided measurement.

[0032] In one possible implementation, the teaching 3D measurement pose is determined, and a multi-scale downsampled point cloud template and the current product multi-scale downsampled point cloud are obtained under the teaching 3D measurement pose. Point cloud matching at the corresponding scale level is then performed, including:

[0033] The 3D measurement pose is determined, and the contact six-axis collaborative measurement robot acquires the local feature point cloud of the product under the 3D measurement pose.

[0034] Perform multi-scale point cloud matching processing on local feature point clouds to obtain multi-scale downsampled point cloud templates;

[0035] After the current product pose changes, the contact six-axis collaborative measurement robot is moved to the teaching 3D measurement pose to obtain the local feature point cloud of the product;

[0036] Perform multi-scale point cloud matching processing on the local feature point cloud of the product to obtain a multi-scale downsampled point cloud;

[0037] Perform coarse and fine matching at the corresponding scale levels on the multi-scale downsampled point cloud template and the multi-scale downsampled point cloud of the current product, with the matching scale level proceeding from high to low.

[0038] If the point cloud matching degree is less than the point cloud matching threshold, the matching scale level is reduced and the coarse matching and fine matching steps are repeated until the point cloud matching threshold is met; if the point cloud matching threshold cannot be reached after traversing all scale levels, the process exits.

[0039] In one possible implementation, multi-scale point cloud matching processing is performed on the local feature point cloud to obtain a multi-scale downsampled point cloud template, including:

[0040] The second region of interest is obtained by extracting the ROI from the local feature point cloud.

[0041] Point cloud filtering is applied to the second region of interest to obtain the filtered second region of interest.

[0042] Based on the filtered second region of interest, a local feature point cloud template is obtained and a multi-scale downsampled point cloud template is formed.

[0043] In one possible implementation, based on the point cloud matching results, a second pose deviation is calculated under the taught 3D measurement pose. The pose transformation is then performed on all taught points in the taught 3D model according to the second pose deviation to obtain the updated second measurement point pose, including:

[0044] Based on the point cloud matching results that meet the point cloud matching threshold, calculate the 6-DOF space transformation from the multi-scale downsampled point cloud template to the current product;

[0045] Based on the obtained spatial transformation, calculate the second pose deviation under the taught 3D measurement pose.

[0046] Based on the second pose deviation and the taught 3D measurement pose, pose transformation is performed on all taught measurement points to obtain the updated corresponding second measurement point pose.

[0047] The contact-type six-axis collaborative measurement robot moves along the taught path to the updated pose of the second measurement point and performs spatial measurement point acquisition, thereby realizing spatial measurement point guided measurement.

[0048] Secondly, the present invention provides a vision-based adaptive measurement point relative guidance device for contact measurement robots. This device uses the aforementioned vision-based adaptive measurement point relative guidance method for contact measurement robots. The device includes a 2D vision-based planar measurement point guidance module, which is used to achieve 4-DOF measurement point guidance (planar translation, rotation, and height scaling) through a 2D vision-based product image contour matching method. Planar translation includes translation along the X-axis and translation along the Y-axis, rotation refers to rotation around the Z-axis, and height scaling refers to scaling in the Z-axis direction in space. It is suitable for measurement point guidance measurement of contact six-axis collaborative measurement robots with products having local features.

[0049] The 2D vision-based planar measurement point guidance module includes:

[0050] The 2D teaching unit is used to teach the measurement points of the contact six-axis collaborative measurement robot on the product under test, and to determine the position and pose of each contact measurement point and the point-to-point path of the robot.

[0051] The contour matching unit is used to determine the teaching 2D measurement pose, obtain the product multi-scale contour image template under the teaching 2D measurement pose and the current product multi-scale image, and perform contour matching at the corresponding scale level.

[0052] The first calculation and update unit is used to calculate the first pose deviation under the teaching 2D measurement pose based on the contour matching result, transform the first pose deviation to all teaching points in the teaching 2D for pose transformation, and traverse and update all measurement point positions to obtain the pose of all updated first measurement points.

[0053] In one possible implementation, the device further includes a 3D vision-based spatial measurement point guidance module, used to achieve 6-DOF (6 degrees of freedom) measurement point guidance in spatial translation, rotation, and posture through a 3D vision-based product feature point cloud matching method. Spatial translation includes translation along the spatial X-axis, translation along the spatial Y-axis, and translation along the spatial Z-axis; rotation refers to spatial rotation around the Z-axis; and posture includes spatial rotation around the X-axis and spatial rotation around the Y-axis. This is suitable for measurement point guidance measurement of contact six-axis collaborative measurement robots for products without features or with products having local features.

[0054] The 3D vision-based spatial measurement point guided measurement module includes:

[0055] The 3D teaching unit is used to teach the measurement points of the contact six-axis collaborative measurement robot on the product under test, and to determine the position and pose of each contact measurement point and the point-to-point path of the robot.

[0056] The point cloud matching unit is used to determine the teaching 3D measurement pose, obtain the multi-scale downsampled point cloud template under the teaching 3D measurement pose and the current product multi-scale downsampled point cloud, and perform point cloud matching at the corresponding scale level.

[0057] The second calculation and update unit is used to calculate the second pose deviation under the teaching 3D measurement pose based on the point cloud matching results, and to perform pose transformation on all teaching points of the teaching 3D according to the second pose deviation to obtain the updated second measurement point pose.

[0058] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0059] 1. This invention relates to a vision-based adaptive relative guidance method for contact measurement robots. Based on teaching the pose of the measurement points, it utilizes 2D and 3D vision to acquire changes in the product's pose and updates the measurement point pose accordingly, achieving adaptive contact measurement of the product. This invention employs small-space relative guidance relative to the taught points, resulting in higher guidance accuracy; it also utilizes multi-scale contour matching and multi-scale downsampling point cloud matching methods, providing better real-time measurement performance and enabling more effective adaptive contact measurement of the product, thus possessing broad engineering application potential.

[0060] 2. The present invention provides a vision-based adaptive relative guidance method for contact measurement robots. Through a 2D vision-based product image contour matching method, it can achieve 4-DOF (degrees of freedom) measurement point guidance including planar translation, rotation, and height scaling, suitable for measurement point guidance of contact six-axis collaborative measurement robots with products featuring local characteristics. Through a 3D vision-based product feature point cloud matching method, it can achieve 6-DOF (degrees of freedom) measurement point guidance including spatial translation, rotation, and posture, suitable for measurement point guidance of contact six-axis collaborative measurement robots with or without features, or with products featuring local characteristics.

[0061] 3. The present invention is a vision-based adaptive relative guidance method for contact measurement robots. The present invention has the advantages of high computational efficiency, high measurement accuracy, and good real-time performance. With the help of this method, contact measurement robots have greater flexibility and can be used not only for guiding measurement points of products with local features, but also for guiding measurement points of products without features.

[0062] This invention is applicable to the automatic measurement of important assembly parameters such as spatial orientation during product docking and assembly. It is suitable not only for relative guidance of measurement points on products with local features, but also for relative guidance of measurement points on products without features. It can be applied to spatial docking and assembly orientation measurement, such as aerospace precision docking assembly, automotive assembly, and automated assembly. Attached Figure Description

[0063] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0064] Figure 1 The flowchart of the vision-based adaptive measurement point relative guidance method for contact measurement robots of this invention is as follows. Figure 1 ;

[0065] Figure 2 The flowchart of the vision-based adaptive measurement point relative guidance method for contact measurement robots of this invention is as follows. Figure 2 ;

[0066] Figure 3 This is a detailed flowchart of the vision-based adaptive measurement point relative guidance method for contact measurement robots according to the present invention;

[0067] Figure 4 This is the structural framework of the vision-based adaptive measurement point relative guidance device for a contact measurement robot according to the present invention. Figure 1 ;

[0068] Figure 5 This is the structural framework of the vision-based adaptive measurement point relative guidance device for a contact measurement robot according to the present invention. Figure 2 . Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0070] Existing measurement point guidance methods are prone to problems such as incorrect or unmeasurable contact measurement point positions and low guidance accuracy due to the large changes in the planar or spatial pose of the incoming product, the small size of the measured feature, and the multiple degrees of freedom of the product's pose.

[0071] This invention designs a vision-based adaptive relative guidance method for contact measurement robots. Based on teaching the pose of the measurement points, it acquires the pose changes of the product using 2D and 3D vision, and updates the pose of the measurement points accordingly, achieving adaptive contact measurement of the product. This invention employs small-space relative guidance relative to the taught points, resulting in higher guidance accuracy; it utilizes multi-scale contour matching and multi-scale downsampling point cloud matching methods, providing better real-time measurement performance and enabling more effective adaptive contact measurement of the product, thus possessing broad engineering application potential.

[0072] Specifically, the product image contour matching method based on 2D vision can realize 4-DOF measurement point guidance of planar translation, rotation, and height scaling, which is suitable for measurement point guidance measurement of contact six-axis collaborative measurement robots with products with local features; the product feature point cloud matching method based on 3D vision can realize 6-DOF measurement point guidance of spatial translation, rotation, and posture, which is suitable for measurement point guidance measurement of contact six-axis collaborative measurement robots with no features or with products with local features.

[0073] Example 1

[0074] like Figure 1 and Figure 3 As shown, this invention presents a vision-based adaptive measurement point relative guidance method for contact measurement robots. This method includes: achieving four degrees of freedom (DOF) measurement point guidance—planar translation (X-axis and Y-axis directions), rotation (around the Z-axis), and height scaling (Z-axis direction)—through a 2D vision-based product image contour matching method. This method is suitable for measurement point guidance measurement of contact six-axis collaborative measurement robots with products featuring local characteristics; including:

[0075] Step 11: Teach the measurement points of the contact six-axis collaborative measurement robot on the product to be tested, and determine the position and pose of each contact measurement point and the point-to-point path of the robot.

[0076] Step 12: Determine the teaching 2D measurement pose, and obtain the product multi-scale contour image template and the current product multi-scale image under the teaching 2D measurement pose, and perform contour matching at the corresponding scale level.

[0077] Step 13: Based on the contour matching results, calculate the first pose deviation under the teaching 2D measurement pose, transform the first pose deviation to all teaching points in the teaching 2D for pose transformation, traverse and update all measurement point positions to obtain the pose of all updated first measurement points.

[0078] In this embodiment, step 12 specifically includes:

[0079] Step 120: Determine the teaching 2D measurement pose, that is, the pose of the contact six-axis collaborative measurement robot, and a clear image of the local features of the product can be obtained under the teaching 2D measurement pose.

[0080] Step 121: Perform multi-scale contour matching processing on the local feature image of the product to obtain a multi-scale image contour template;

[0081] Step 122: After the current product pose changes, move the contact six-axis collaborative measurement robot to the teaching 2D measurement pose to obtain the product image;

[0082] Step 123: Perform multi-scale contour matching processing on the obtained product image to obtain multi-scale image contours;

[0083] Step 124: Perform contour matching at the corresponding scale level on the obtained multi-scale image contour template and the multi-scale image contour, and perform contour matching at the scale level from high to low; if the contour matching degree is less than the image contour matching threshold, reduce the scale level and repeat step 124 until the image contour matching threshold is met; if the image contour matching threshold cannot be reached after traversing all scale levels, exit.

[0084] Specifically, step 121 includes:

[0085] The first region of interest (ROI) is obtained by extracting the local feature image of the product.

[0086] Based on the first region of interest, connected component calculation and filtering are performed using dynamic threshold segmentation, and noise points are removed to obtain feature contours and form multi-scale image contour templates.

[0087] Similarly, multi-scale contour matching processing is performed on the product image to obtain multi-scale image contours.

[0088] In this embodiment, step 13 specifically includes:

[0089] Step 130: Based on the contour matching results that meet the image contour matching threshold, calculate the translation, rotation, and scaling transformation from the multi-scale image contour template to the current image, i.e., affine transformation.

[0090] Step 131: Calculate the flange pose deviation of the measuring robot under the taught 2D measurement pose according to the affine transformation as the first pose deviation.

[0091] Step 132: Based on the first pose deviation and the taught 2D measurement pose, perform pose transformation on all taught measurement points to obtain the updated first measurement point pose.

[0092] Step 133: The contact six-axis collaborative measurement robot moves along the taught path to the updated pose of the first measurement point and performs planar measurement point acquisition, thereby realizing planar measurement point guided measurement.

[0093] In planar measurement of products, the product has four degrees of freedom: translation, rotation, and height scaling. To address issues such as mis-touching and unmeasurable measurement points caused by large translation and rotation angles, this invention proposes an adaptive planar measurement point guidance method based on a 2D vision-based six-axis collaborative measurement robot. First, the product to be measured is taught to its measurement points using a six-axis collaborative measurement robot. Second, the taught 2D measurement pose is determined, and a multi-scale contour image template of the product under this taught 2D measurement pose and the current multi-scale image of the product are obtained, followed by contour matching at the corresponding scale levels. Finally, the first pose deviation under the taught 2D measurement pose is calculated, and this first pose deviation is transformed to all taught points in the 2D model for pose transformation. All measurement point positions are traversed and updated to obtain the updated first measurement point pose. This method is applicable to point-guided measurement of products with local features using a six-axis collaborative measurement robot.

[0094] Example 2

[0095] like Figure 2 and Figure 3 As shown, the difference between this embodiment and Embodiment 1 is that the product has six degrees of freedom (translation, rotation, and attitude) in the spatial measurement of product points. To address issues such as mis-touching and unmeasurable measurement points caused by large translation, rotation, and attitude angles, this embodiment also proposes an adaptive spatial measurement point relative guidance method based on 3D vision for a contact-type six-axis collaborative measurement robot. This method further includes: using a 3D vision-based product feature point cloud matching method to achieve six degrees of freedom (spatial translation (spatial X-axis direction, spatial Y-axis direction, spatial Z-axis direction), rotation (spatial rotation around the Z-axis), and attitude (spatial rotation around the X-axis, spatial rotation around the Y-axis) measurement point guidance, including:

[0096] Step 21: Teach the contact six-axis collaborative measurement robot to the measurement points of the product under test, and determine the position and pose of each contact measurement point and the point-to-point path of the robot.

[0097] Step 22: Determine the teaching 3D measurement pose, and obtain the multi-scale downsampled point cloud template and the current product multi-scale downsampled point cloud under the teaching 3D measurement pose, and perform point cloud matching at the corresponding scale level.

[0098] Step 23: Based on the point cloud matching results, calculate the second pose deviation under the teaching 3D measurement pose, and perform pose transformation on all teaching points of the teaching 3D according to the second pose deviation to obtain the updated second measurement point pose.

[0099] In this embodiment, step 22 specifically includes:

[0100] Step 220: Determine the teaching 3D measurement pose, i.e. the pose of the contact six-axis collaborative measurement robot, and acquire the local feature point cloud of the product under the teaching 3D measurement pose.

[0101] Step 221: Perform multi-scale point cloud matching processing on the local feature point cloud to obtain a multi-scale downsampled point cloud template;

[0102] Step 222: After the current product pose changes, move the contact six-axis collaborative measurement robot to the teaching 3D measurement pose to obtain the local feature point cloud of the product;

[0103] Step 223: Perform multi-scale point cloud matching processing on the local feature point cloud of the product to obtain a multi-scale downsampled point cloud;

[0104] Step 224: Perform coarse matching and fine matching at the corresponding scale levels on the obtained multi-scale downsampled point cloud template and the multi-scale downsampled point cloud of the current product, and perform matching scale levels from high to low; if the point cloud matching degree is less than the point cloud matching threshold, reduce the matching scale level and repeat step 224 until the point cloud matching threshold is met; if the point cloud matching threshold cannot be reached after traversing all scale levels, exit.

[0105] Specifically, step 221 includes:

[0106] The second region of interest is obtained by extracting the ROI from the local feature point cloud.

[0107] Point cloud filtering is applied to the second region of interest to obtain the filtered second region of interest.

[0108] Based on the filtered second region of interest, a local feature point cloud template is obtained and a multi-scale downsampled point cloud template is formed.

[0109] In this embodiment, step 23 specifically includes:

[0110] Step 230: Based on the point cloud matching results that meet the point cloud matching threshold, calculate the 6-DOF space transformation from the multi-scale downsampled point cloud template to the current product.

[0111] Step 231: Calculate the second pose deviation under the taught 3D measurement pose based on the obtained spatial transformation;

[0112] Step 232: Based on the second pose deviation and the taught 3D measurement pose, perform pose transformation on all taught measurement points to obtain the updated corresponding second measurement point pose.

[0113] Step 233: The contact six-axis collaborative measurement robot moves along the taught path to the updated pose of the second measurement point and performs spatial measurement point acquisition, thereby realizing spatial measurement point guided measurement.

[0114] The above technical solution and method are applicable to point-guided measurement of contact six-axis collaborative measurement robots for products without features or with local features.

[0115] Example 3

[0116] like Figure 4 and Figure 5 As shown, the difference between this embodiment and embodiment 1 is that this embodiment provides a vision-based adaptive measurement point relative guidance device for a contact measurement robot, which uses the vision-based adaptive measurement point relative guidance method for a contact measurement robot in embodiment 1.

[0117] like Figure 4 As shown, the device includes a 2D vision-based planar measurement point guidance module, which is used to achieve 4-DOF (degree of freedom) measurement point guidance in planar translation, rotation, and height scaling through a 2D vision-based product image contour matching method. It is suitable for measurement point guidance measurement of contact six-axis collaborative measurement robots with products having local features.

[0118] The 2D vision-based planar measurement point guidance module includes:

[0119] The 2D teaching unit is used to teach the measurement points of the contact six-axis collaborative measurement robot on the product under test, and to determine the position and pose of each contact measurement point and the point-to-point path of the robot.

[0120] The contour matching unit is used to determine the teaching 2D measurement pose, obtain the product multi-scale contour image template under the teaching 2D measurement pose and the current product multi-scale image, and perform contour matching at the corresponding scale level.

[0121] The first calculation and update unit is used to calculate the first pose deviation under the teaching 2D measurement pose based on the contour matching result, transform the first pose deviation to all teaching points in the teaching 2D for pose transformation, and traverse and update all measurement point positions to obtain the pose of all updated first measurement points.

[0122] In one possible implementation, such as Figure 5 As shown, the device also includes a 3D vision-based spatial measurement point guidance module, which is used to achieve spatial translation, rotation and attitude 6-DOF measurement point guidance through a 3D vision-based product feature point cloud matching method. It is suitable for measurement point guidance measurement of contact six-axis collaborative measurement robots with no features or with local features.

[0123] The 3D vision-based spatial measurement point guided measurement module includes:

[0124] The 3D teaching unit is used to teach the measurement points of the contact six-axis collaborative measurement robot on the product under test, and to determine the position and pose of each contact measurement point and the point-to-point path of the robot.

[0125] The point cloud matching unit is used to determine the teaching 3D measurement pose, obtain the multi-scale downsampled point cloud template under the teaching 3D measurement pose and the current product multi-scale downsampled point cloud, and perform point cloud matching at the corresponding scale level.

[0126] The second calculation and update unit is used to calculate the second pose deviation under the teaching 3D measurement pose based on the point cloud matching results, and to perform pose transformation on all teaching points of the teaching 3D according to the second pose deviation to obtain the updated second measurement point pose.

[0127] The execution process of each unit can be carried out according to the steps of the vision-based contact measurement robot adaptive measurement point relative guidance method in Example 1, and will not be described in detail in this example.

[0128] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0130] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0131] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0132] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A vision-based adaptive measurement point relative guidance method for a contact measurement robot, characterized in that, This method includes: using a 2D vision-based product image contour matching method to achieve 4-DOF (degree of freedom) measurement point guidance, encompassing planar translation, rotation, and height scaling. Planar translation includes translation along the X-axis and Y-axis, rotation refers to rotation around the Z-axis, and height scaling refers to scaling along the Z-axis in space. This method is suitable for measurement point guidance measurement of contact-type six-axis collaborative measurement robots for products with local features; including: The contact six-axis collaborative measurement robot is used to teach the measurement points of the product under test, and the position and pose of each contact measurement point and the point-to-point path of the robot are determined. Determine the teaching 2D measurement pose, obtain the product multi-scale contour image template under the teaching 2D measurement pose and the current product multi-scale image, and perform contour matching at the corresponding scale level; Based on the contour matching results, the first pose deviation under the teaching 2D measurement pose is calculated, and the first pose deviation is transformed to all teaching points in the teaching 2D to perform pose transformation. All measurement point positions are traversed and updated to obtain the pose of the first measurement point after all updates. Determine the teaching 2D measurement pose, obtain the product multi-scale contour image template under the teaching 2D measurement pose and the current product multi-scale image, and perform contour matching at the corresponding scale levels, including: Determine the 2D teaching measurement pose, and the contact six-axis collaborative measurement robot acquires local feature images of the product under the 2D teaching measurement pose; Perform multi-scale contour matching processing on the local feature image of the product to obtain a multi-scale image contour template; After the current product pose changes, the contact six-axis collaborative measurement robot is moved to the taught 2D measurement pose to obtain the product image; The product image is subjected to multi-scale contour matching processing to obtain multi-scale image contours. Contour matching is performed between the multi-scale image contour template and the multi-scale image contour at corresponding scale levels, with the contour matching scale levels proceeding from high to low. If the contour matching degree is less than the image contour matching threshold, the scale level is reduced and the contour matching step is repeated until the image contour matching threshold is met; if the image contour matching threshold cannot be reached after traversing all scale levels, the process is terminated. Based on the contour matching results, the first pose deviation under the taught 2D measurement pose is calculated. This first pose deviation is then transformed to all taught points in the taught 2D model for pose transformation. All measurement point positions are iterated and updated to obtain the updated poses of all the first measurement points, including: Based on the contour matching results that meet the image contour matching threshold, calculate the translation, rotation, and scaling transformation from the multi-scale image contour template to the current image, i.e., the affine transformation. Based on the affine transformation, the flange pose deviation of the measurement robot under the taught 2D measurement pose is calculated as the first pose deviation. Based on the first pose deviation and the taught 2D measurement pose, pose transformation is performed on all taught measurement points to obtain the updated first measurement point pose. The contact-type six-axis collaborative measurement robot moves along the taught path to the updated pose of the first measurement point and performs planar measurement point acquisition, thereby realizing planar measurement point guided measurement.

2. The vision-based adaptive measurement point relative guidance method for a contact measurement robot according to claim 1, characterized in that, The method also includes: using a 3D vision-based product feature point cloud matching method to achieve 6-DOF (6 degrees of freedom) measurement point guidance in spatial translation, rotation, and posture. Spatial translation includes translation along the X-axis, Y-axis, and Z-axis; rotation refers to spatial rotation around the Z-axis; and posture includes spatial rotation around the X-axis and Y-axis. This method is applicable to point-guided measurement of contact-type six-axis collaborative measurement robots for products without features or with local features. The contact six-axis collaborative measurement robot is used to teach the measurement points of the product under test, and the position and pose of each contact measurement point and the point-to-point path of the robot are determined. Determine the teaching 3D measurement pose, obtain the multi-scale downsampled point cloud template under the teaching 3D measurement pose and the current product multi-scale downsampled point cloud, and perform point cloud matching at the corresponding scale level; Based on the point cloud matching results, the second pose deviation under the teaching 3D measurement pose is calculated. Based on the second pose deviation, the pose transformation is performed on all teaching points in the teaching 3D to obtain the updated second measurement point pose.

3. The vision-based adaptive measurement point relative guidance method for a contact measurement robot according to claim 1, characterized in that, The local feature image of the product is subjected to multi-scale contour matching processing to obtain a multi-scale image contour template, including: The first region of interest (ROI) is obtained by extracting the local feature image of the product. Based on the first region of interest, connected component calculation and filtering are performed using dynamic threshold segmentation, and noise points are removed to obtain feature contours and form multi-scale image contour templates.

4. The vision-based adaptive measurement point relative guidance method for a contact measurement robot according to claim 2, characterized in that, Determine the teaching 3D measurement pose, obtain the multi-scale downsampled point cloud template under the teaching 3D measurement pose and the current product's multi-scale downsampled point cloud, and perform point cloud matching at the corresponding scale levels, including: The 3D measurement pose is determined, and the contact six-axis collaborative measurement robot acquires the local feature point cloud of the product under the 3D measurement pose. Perform multi-scale point cloud matching processing on the local feature point cloud to obtain a multi-scale downsampled point cloud template; After the current product pose changes, the contact six-axis collaborative measurement robot is moved to the teaching 3D measurement pose to obtain the local feature point cloud of the product; Perform multi-scale point cloud matching processing on the local feature point cloud of the product to obtain a multi-scale downsampled point cloud; The multi-scale downsampled point cloud template and the multi-scale downsampled point cloud of the current product are subjected to coarse matching and fine matching at the corresponding scale levels, and the matching scale levels are performed from high to low. If the point cloud matching degree is less than the point cloud matching threshold, the matching scale level is reduced and the coarse matching and fine matching steps are repeated until the point cloud matching threshold is met; if the point cloud matching threshold cannot be reached after traversing all scale levels, the process exits.

5. The vision-based adaptive measurement point relative guidance method for a contact measurement robot according to claim 4, characterized in that, Perform multi-scale point cloud matching processing on the local feature point cloud to obtain a multi-scale downsampled point cloud template, including: The local feature point cloud is subjected to point cloud ROI extraction to obtain the second region of interest; Point cloud filtering is applied to the second region of interest to obtain the filtered second region of interest. Based on the filtered second region of interest, a local feature point cloud template is obtained and a multi-scale downsampled point cloud template is formed.

6. The vision-based adaptive measurement point relative guidance method for a contact measurement robot according to claim 2, characterized in that, Based on the point cloud matching results, the second pose deviation under the teaching 3D measurement pose is calculated. The pose transformation is then performed on all teaching points in the teaching 3D based on the second pose deviation to obtain the updated second measurement point pose, including: Based on the point cloud matching results that meet the point cloud matching threshold, calculate the 6-DOF space transformation from the multi-scale downsampled point cloud template to the current product; Based on the obtained spatial transformation, calculate the second pose deviation under the taught 3D measurement pose. Based on the second pose deviation and the taught 3D measurement pose, pose transformation is performed on all taught measurement points to obtain the updated corresponding second measurement point pose. The contact-type six-axis collaborative measurement robot moves along the taught path to the updated pose of the second measurement point and performs spatial measurement point acquisition, thereby realizing spatial measurement point guided measurement.

7. A vision-based adaptive measurement point relative guidance device for a contact measurement robot, characterized in that, The device uses the vision-based adaptive measurement point relative guidance method for contact measurement robots as described in any one of claims 1 to 6; the device includes a 2D vision-based planar measurement point guidance module, which is used to achieve 4-DOF measurement point guidance of planar translation, rotation, and height scaling through a 2D vision-based product image contour matching method. Planar translation includes translation along the X-axis and translation along the Y-axis, rotation refers to rotation around the Z-axis, and height scaling refers to scaling in the Z-axis direction in space. Suitable for point-guided measurement of contact-type six-axis collaborative measurement robots for products with local features; The 2D vision-based planar measurement point guided measurement module includes: The 2D teaching unit is used to teach the measurement points of the contact six-axis collaborative measurement robot on the product under test, and to determine the position and pose of each contact measurement point and the point-to-point path of the robot. The contour matching unit is used to determine the teaching 2D measurement pose, obtain the product multi-scale contour image template under the teaching 2D measurement pose and the current product multi-scale image, and perform contour matching at the corresponding scale level. The first calculation and update unit is used to calculate the first pose deviation under the teaching 2D measurement pose based on the contour matching result, transform the first pose deviation to all teaching points in the teaching 2D for pose transformation, and traverse and update all measurement point positions to obtain the pose of all updated first measurement points.

8. The vision-based adaptive measurement point relative guidance device for a contact measurement robot according to claim 7, characterized in that, The device also includes a 3D vision-based spatial measurement point guidance module, which is used to achieve spatial translation, rotation and attitude 6-DOF measurement point guidance through a 3D vision-based product feature point cloud matching method. Spatial translation includes translation along the spatial X-axis, translation along the spatial Y-axis and translation along the spatial Z-axis. Rotation refers to spatial rotation around the Z-axis. Attitude includes spatial rotation around the X-axis and spatial rotation around the Y-axis. Suitable for point-guided measurement of contact-type six-axis collaborative measurement robots for products with no features or with local features; The 3D vision-based spatial measurement point guided measurement module includes: The 3D teaching unit is used to teach the measurement points of the contact six-axis collaborative measurement robot on the product under test, and to determine the pose of each contact measurement point and the point-to-point path of the robot. The point cloud matching unit is used to determine the teaching 3D measurement pose, obtain the multi-scale downsampled point cloud template under the teaching 3D measurement pose and the current product multi-scale downsampled point cloud, and perform point cloud matching at the corresponding scale level. The second calculation and update unit is used to calculate the second pose deviation under the teaching 3D measurement pose based on the point cloud matching result, and to perform pose transformation on all teaching points of the teaching 3D according to the second pose deviation to obtain the updated second measurement point pose.

Images