A method for measuring the pose of curved surface components without clear positioning features based on multi-vision

Abstract

A method for locating curved surface components without explicit positioning features based on multi-view vision includes constructing a multi-view vision measurement system for the curved surface component's pose, and attaching four circular reflective markers as pose control points to the upper surface of the curved surface component. A Gaussian curve is fitted using the grayscale gradient values ​​of sampling points along the gradient direction of edge pixels to obtain sub-pixel edges, and ellipse fitting is performed using an improved random sampling consistency algorithm. Extrinsic parameter calibration between two non-overlapping binocular vision subsystems is achieved using a laser tracker. Based on the coordinates of the four pose control points in the curved surface component coordinate system and the vision measurement coordinate system, the pose of the curved surface component without explicit positioning features is solved through SVD decomposition. This invention enables pose measurement of components without explicit positioning features, whose shapes are mostly curved surfaces and curves, and solves the problem of occlusion of the field of view by the robot's end effector during the measurement process, providing pose information for automated assembly by robots.

Description

Technical Field

[0001] This invention relates to a measurement method, and more specifically to a method for measuring the pose of curved surface components without clearly defined positioning features based on multi-view vision, belonging to the field of automated assembly measurement technology. Background Technology

[0002] In the aerospace field, there are curved surface components without clearly defined positioning features. These components lack locating features such as reference holes and corner points, and their shapes are mostly curved surfaces and curves. Currently, their assembly is mostly done manually, resulting in low efficiency and assembly quality limited by operator experience. With the development of robotics and digital measurement technologies, automated assembly is increasingly being used in the aerospace field. Robots, with their high degree of automation and flexibility, have become an important vehicle for achieving automated and intelligent assembly in the aerospace industry. Using industrial robots and flexible end effectors as carriers in the assembly of curved surface components without clearly defined positioning features, combined with machine vision measurement methods, can meet the high-precision assembly requirements.

[0003] For curved surface components without clearly defined positioning features, it is difficult to estimate their pose by extracting their own features, and the accuracy is hard to guarantee. Obtaining their point cloud data through stereo vision or structured light technology is greatly affected by ambient light and surface material, and the processing time is long. At the same time, curved surface components have the requirement of flexible and random adsorption, and the camera's field of view is easily obstructed by the end effector. Therefore, a pose measurement method for curved surface components without clearly defined positioning features based on multi-view vision is proposed to achieve accurate pose measurement in automated assembly processes. Summary of the Invention

[0004] The purpose of this invention is to address the need for accurate pose measurement in the automated assembly of curved surface components without clear positioning features. It solves the problem that the objects to be assembled have no clear positioning features and their shapes are mostly curved surfaces and curves, making it difficult to directly extract the object features to achieve pose calculation during the assembly process. At the same time, there is a problem that the robot's end effector occludes the camera's field of view during the measurement process.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for measuring the pose of surface components without explicit localization features based on multi-view vision includes the following steps:

[0007] S1: Construction of a multi-view vision measurement system for the pose of curved components. Four circular reflective markers are pasted on the upper surface of the curved components as pose control points.

[0008] S2: A Gaussian curve is fitted with the gray-level gradient values ​​of the sampling points along the gradient direction of the edge pixels to obtain the sub-pixel edges, and the center of the pose control point is accurately extracted through the improved Random Sample Consensus (RANSAC) algorithm.

[0009] S3: External parameter calibration between two binocular vision subsystems is achieved using a laser tracker;

[0010] S4: Based on the coordinates of four pose control points in the surface component coordinate system and the visual measurement coordinate system, the pose of a surface component without explicit positioning features is solved through SVD decomposition.

[0011] As a preferred embodiment of the present invention, step S1 includes the following:

[0012] S1.1: The multi-view vision measurement system consists of two sets of binocular vision subsystems with non-overlapping fields of view, located on both sides of the robot's end effector. Each subsystem measures the three-dimensional coordinates of two pose control points on that side and unifies them into the vision measurement coordinate system.

[0013] S1.2: Four reflective circular markers are pasted on the upper surface of the curved component as pose control points. Before the assembly operation, the coordinate system of the curved component and the coordinates of the pose control points in the coordinate system are obtained by scanning with a high-precision measuring device and then reconstructed in reverse. This information is then input into the subsequent assembly and pose measurement process as known information.

[0014] As a preferred embodiment of the present invention, step S2 includes the following:

[0015] S2.1: The visual measurement system acquires images, performs binarization processing, and extracts contours. It sets roundness and area thresholds for the contours to filter the ROIs (Regions of Interest) corresponding to the pose control points, thereby improving processing speed. To ensure extraction stability under different lighting conditions, the OTSU algorithm is used to implement adaptive thresholding for Canny edge detection, and then the ROI image of the pose control points is processed to obtain integer-pixel edges.

[0016] S2.2: Calculate the grayscale gradient G in the x-direction of integer pixel edge points of the pose control points using the Sobel operator. x y-axis grayscale gradient G y Then calculate the gradient magnitude G. i and gradient direction θ i :

[0017]

[0018] S2.3: Sample 3 points on each side of the gradient direction along the edge of the whole pixel with a step size of 1, and calculate the gradient magnitude using bilinear interpolation. The gradient magnitude is expressed as the sampling point index x. j and the gradient magnitude G at that pointj Construct a point set (x) j G j To fit a Gaussian curve, where μ is the mean and σ is the standard deviation, substitute the constructed point set into the Gaussian function expression and take the logarithm of both sides of the equation:

[0019]

[0020] It conforms to the form of a conic section, so that The three coefficients of the quadratic curve can be obtained by solving the least squares method using the constructed point set. The center position of the Gaussian curve can be expressed as... Then the sub-pixel coordinates of the pose control point ( s x i , s y i )for:

[0021]

[0022] Processing all pixel edge points yields the sub-pixel edges of the pose control points;

[0023] S2.4: The general equation of an ellipse is denoted as: Ax 2 +Bxy+Cy 2 +Dx+Ey+F=0. To effectively remove outliers caused by image noise or interfering contours, an improved RANSAC (Random Sample Consensus) algorithm is used for ellipse fitting. Specifically, during the iterative fitting of the ellipse and selection of interior points, to ensure fitting accuracy when the proportion of sampled points to the ellipse contour is small, a super least squares method is used to fit the ellipse. The center (x0, y0) of the final pose control point can be represented as:

[0024]

[0025] As a preferred embodiment of the present invention, step S3 includes the following:

[0026] S3.1: The parameters of the two sets of binocular vision subsystems were calibrated using Zhang Zhengyou's calibration method. A set of common points in the workspace of the laser tracker and the single-sided binocular vision subsystem were measured respectively. The coordinates of the points in the laser tracker coordinate system obtained from the final measurements were then set as follows: l p i (i = 1, 2, ..., n), the coordinates of the point in the coordinate system of the single-sided binocular vision subsystem are: c1 p i (i = 1, 2, ..., n), the two sets of point sets satisfy the following relationship:

[0027] c1 p i =Rl p i +t (13)

[0028] R and t represent the relative pose between the coordinate system of the visual subsystem on one side and the coordinate system of the laser tracker;

[0029] S3.2: Calculate the point set c1 P and l The center of mass of P: make Constructing the covariance matrix

[0030] S3.3: Performing SVD decomposition on H yields: H = UDV T ,but:

[0031]

[0032]

[0033] This indicates the pose of the laser tracker coordinate system in the coordinate system of the binocular vision subsystem on this side;

[0034] S3.4: The pose of the laser tracker coordinate system in the coordinate system of the other binocular vision subsystem can also be calculated using the same method described above. Let the relative poses of the laser tracker and the two sets of binocular vision subsystems be respectively and The global extrinsic parameters of the multi-view vision measurement system and It can be represented as:

[0035]

[0036] As a preferred embodiment of the present invention, step S4 includes the following:

[0037] S4.1: Filter the true pose control points. Constrain the spatial distance provided by the coordinate values ​​of the pose control points in the coordinate system of the curved component. Traverse the set of measurement points, take four points each time and calculate the spatial distance of each point. Find four pose control points whose spatial distance meets the requirements, and use the order of the pose control points with the smallest error after coordinate transformation as the final corresponding point order.

[0038] S4.2: Let the coordinates of the pose control points in the camera coordinate system be... c p i (i = 1, 2, 3, 4), the coordinates of the pose control points corresponding to the measured model coordinate system of the curved component are: t p i(i=1,2,3,4), calculate the pose of the curved component relative to the visual measurement system using equations (5)-(7).

[0039] The beneficial effects of this invention are as follows:

[0040] 1. It realizes precise pose measurement in the automated assembly process of curved surface components without clear positioning features in the aerospace field using robots as actuators, and enables high-precision assembly of curved surface components without clear positioning features.

[0041] 2. By attaching reflective circular markers to the upper surface of curved components as pose control points, the problem of difficulty in directly extracting features to calculate pose during assembly is solved, as these components lack clear positioning features and are mostly curved surfaces and curves.

[0042] 3. A multi-view vision measurement system consisting of two sets of non-overlapping binocular vision subsystems was constructed, which avoids the problem of the robot end effector occluding the camera field of view during the measurement process. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the multi-view vision measurement system for curved components according to the present invention.

[0044] Figure 2 This is a schematic diagram of the overall process of the present invention.

[0045] Figure 3 This is a schematic diagram of pose control point (ROI) image extraction and pixel edge acquisition.

[0046] Figure 4 This is a schematic diagram of subpixel edge processing and ellipse fitting.

[0047] Figure 5 This is a schematic diagram of the global extrinsic parameter calibration principle for a multi-view vision measurement system.

[0048] Figure 6 This is a schematic diagram illustrating the principle of pose measurement for curved surface components without clearly defined positioning features. Detailed Implementation

[0049] The technical solutions in the implementation process of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0050] Please see Figure 1-6 .

[0051] A technical solution for a method of measuring the pose of curved surface components without explicit localization features based on multi-view vision includes the following steps:

[0052] S1: Construction of a multi-view vision measurement system for the pose of curved components;

[0053] S2: Obtain sub-pixel edges of pose control points and extract the center by fitting an ellipse;

[0054] S3: Extrinsic parameter calibration between the two binocular vision subsystems;

[0055] S4: Pose calculation of surface components without explicit positioning features.

[0056] in:

[0057] Step S1: Construction of a multi-view vision measurement system for the pose of curved components:

[0058] The multi-view vision measurement system consists of two sets of non-overlapping binocular vision subsystems, located on both sides of the robot's end effector. The camera and lens are fixed to the robot's flexible end effector by tooling. Each subsystem measures the three-dimensional coordinates of two pose control points on its side and unifies them into the vision measurement coordinate system.

[0059] Four circular reflective markers were affixed to the upper surface of the curved component as pose control points. Before assembly, the coordinate system of the curved component and the three-dimensional coordinates of the pose control points in that coordinate system were obtained by scanning with a blue light scanner and then reconstructed in reverse. This information was then input into the subsequent assembly and pose measurement processes. To ensure good imaging results, a red ring LED light source was used for supplementary lighting, and the camera was arranged coaxially with the light source.

[0060] Step S2: Obtain sub-pixel edges of pose control points and extract the center by fitting an ellipse:

[0061] Pose control points account for a small proportion of the acquired image and are clearly distinguishable from the background grayscale values. To improve processing efficiency and accuracy, the image is first binarized, and the contour of the binarized image is extracted and then determined based on roundness. The area S feature is used to set filtering conditions, and the ROI is divided by expanding the largest bounding rectangle of the filtered contour by 4 pixels.

[0062] The Canny operator is used for integer-pixel edge acquisition. Since the Canny operator requires two thresholds to distinguish between strong and weak edges, it is improved to ensure stability under different measurement environments and lighting conditions. Adaptive threshold acquisition is achieved based on the Otsu algorithm.

[0063] For an original grayscale image of size M×N, its integer gray levels are {1,2,...,n}, and the probability of gray level i is p. iLet the segmentation threshold be k (k∈(0,n)), and the image be divided into two classes, I1 and I2. The probabilities of a pixel belonging to class I1 and I2 are P1(k) and P2(k), respectively. The gray mean values ​​of classes I1 and I2 are g1(k) and g2(k), respectively. Then, the inter-class variance σ of the image is... 2 =P1(k)(g1(k)-g) 2 +P2(k)(g2(k)-g) 2 .

[0064] The high threshold is set when the inter-class variance of the image is maximized, and the low threshold is set when 0.5k is set, thus obtaining the integer pixel edges of the pose control points.

[0065] The gray-level gradient G in the x-direction of integer-pixel edge points of the pose control points is calculated using the Sobel operator. x y-axis grayscale gradient G y Then calculate the gradient magnitude G. i and gradient direction θ i :

[0066]

[0067] Along the gradient direction of the integer pixel edge point, sample 3 points on each side with a step size of 1, and calculate the gradient magnitude using bilinear interpolation. The sample point index x is then used as the denoted value. j and the gradient magnitude G at that point j Construct a point set (x) j G j To fit a Gaussian curve, where μ is the mean and σ is the standard deviation, substitute the constructed point set into the Gaussian function expression and take the logarithm of both sides of the equation:

[0068]

[0069] It conforms to the form of a conic section, so that The three coefficients of the quadratic curve can be obtained by solving the least squares method using the constructed point set. The center position of the Gaussian curve can be expressed as... Then the sub-pixel coordinates of the pose control point ( s x i , s y i )for:

[0070]

[0071] Processing all pixel edge points yields the sub-pixel edges of the pose control points;

[0072] The general equation of an ellipse is denoted as: Ax 2 +Bxy+Cy 2+Dx+Ey+F=0. To effectively remove outliers caused by image noise or interfering contours, an improved RANSAC (Random Sample Consensus) algorithm is used for ellipse fitting. Specifically, during the iterative fitting of the ellipse and selection of interior points, to ensure fitting accuracy when the proportion of sampled points to the ellipse contour is small, a super least squares method is used to fit the ellipse. The center (x0, y0) of the final pose control point can be represented as:

[0073] Step S3: Extrinsic parameter calibration between the two binocular vision subsystems:

[0074] The intrinsic and extrinsic parameters of the two sets of binocular vision subsystems were calibrated using Zhang Zhengyou's calibration method.

[0075] Target ball holders are set up on both sides of the calibration fixture, which can hold visual target balls or laser target balls. The orientation of the calibration fixture is changed multiple times in the measurement space, and a series of spatial points P = [p1, p2, ..., p] can be obtained using a laser tracker. n Simultaneously, images of the visual target sphere within their respective fields of view are acquired through two sets of binocular vision subsystems, resulting in two sets of spatial points C = [c1, c2, ..., c...]. m ]、C′=[c′ m+1 ,c′ m+2 ,...,c n Each of these points corresponds one-to-one with the measurement points of the laser tracker.

[0076] Taking the binocular vision subsystem on one side as an example, it satisfies:

[0077] c i =Rp i +t (21)

[0078] R and t represent the relative pose between the coordinate system of the visual subsystem on one side and the coordinate system of the laser tracker;

[0079] Calculate the centroids of point sets P and C: make Constructing the covariance matrix

[0080] Performing SVD decomposition on H yields: H = UDV T ,but:

[0081]

[0082]

[0083] This indicates the pose of the laser tracker coordinate system in the coordinate system of the binocular vision subsystem on this side;

[0084] The pose of the laser tracker coordinate system in the coordinate system of the other binocular vision subsystem can also be calculated using the same method. Let the relative poses of the laser tracker and the two sets of binocular vision subsystems be respectively and The global extrinsic parameters of the multi-view vision measurement system and It can be represented as:

[0085]

[0086] Step S4: Pose calculation for surface components without explicit positioning features:

[0087] Constraints are provided by the spatial distances provided by the coordinate values ​​of the pose control points in the coordinate system of the curved component. The measurement point set is traversed, and four points are selected each time to calculate the spatial distance between each point. Four pose control points with spatial distances that meet the requirements are found, and the order of the pose control points with the smallest error after coordinate transformation is taken as the final corresponding point order.

[0088] Let the coordinates of the pose control point in the camera coordinate system be... c p i (i = 1, 2, 3, 4), the coordinates of the pose control points corresponding to the measured model coordinate system of the curved component are: t p i (i=1,2,3,4), calculate the pose of the curved component relative to the visual measurement system using equations (5)-(7). c t T.

[0089] In the description of this invention, it should be understood that the indicated orientation or positional relationship is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing this invention and simplifying the description, and is not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0090] In this invention, unless otherwise explicitly specified and limited, for example, it can be a fixed connection, a detachable connection, or an integral part; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two elements or an interaction between two elements. Unless otherwise explicitly limited, those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0091] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

[0092] The parts not covered in this invention are the same as or can be implemented using existing technologies.

Claims

1. A method for measuring the pose of a surface component without explicit localization features based on multi-view vision, characterized in that, Includes the following steps: S1: Construction of a multi-view vision measurement system for the pose of curved components. Four circular reflective markers are pasted on the upper surface of the curved components as pose control points. S2: By fitting a Gaussian curve with the gray-level gradient values ​​of the sampling points along the gradient direction of the edge pixels, sub-pixel edges are obtained, and ellipse fitting is performed using the improved Random Sample Consensus (RANSAC) algorithm, thus achieving accurate extraction of the pose control point center. S3: External parameter calibration between two binocular vision subsystems is achieved using a laser tracker; S4: Based on the coordinates of four pose control points in the surface component coordinate system and the visual measurement coordinate system, the pose of a surface component without clear positioning features is solved by SVD decomposition. Step S2 includes the following: S2.1: The visual measurement system acquires images, performs binarization processing and extracts contours, and sets roundness and area thresholds for the contours to filter the ROI (region of interest) corresponding to the pose control points in order to improve processing speed. To ensure extraction stability under different lighting conditions, the OTSU algorithm is used to implement an adaptive threshold for Canny edge detection, and then the ROI image of the pose control point is processed to obtain integer pixel edges. S2.2: Calculate the grayscale gradient G in the x-direction of integer pixel edge points of the pose control points using the Sobel operator. x y-axis grayscale gradient G y Then calculate the gradient magnitude G. i and gradient direction θ i : S2.3: Sample 3 points on each side of the gradient direction along the edge of the whole pixel with a step size of 1, and calculate the gradient magnitude using bilinear interpolation. The gradient magnitude is expressed as the sampling point index x. j and the gradient magnitude G at that point j Construct a point set (x) j G j To fit a Gaussian curve, where μ is the mean and σ is the standard deviation; substitute the constructed point set into the Gaussian function expression and take the logarithm of both sides of the equation: It conforms to the form of a conic section, so that The three coefficients of the quadratic curve can be obtained by solving the least squares method using the constructed point set. The center position of the Gaussian curve can be expressed as... Then the sub-pixel coordinates of the pose control point ( s x i , s y i )for: Processing all pixel edge points yields the sub-pixel edges of the pose control points; S2.4: The general equation of an ellipse is denoted as: Ax 2 +Bxy+Cy 2 +Dx+Ey+F=0, to effectively remove outliers caused by image noise or interfering contours, an improved RANSAC (Random Sample Consensus) algorithm is used for ellipse fitting. Specifically, during the iterative fitting of the ellipse and selection of interior points, to ensure fitting accuracy when the proportion of sampled points to the ellipse contour is small, a super least squares method is employed to fit the ellipse. The final center coordinates (x0, y0) of the pose control point can be expressed as:

2. The method for measuring the pose of curved surface components without explicit localization features based on multi-view vision according to claim 1, characterized in that: Step S1 includes the following: S1.1: The multi-view vision measurement system consists of two sets of binocular vision subsystems with non-overlapping fields of view, located on both sides of the robot's end effector. Each subsystem measures the three-dimensional coordinates of two pose control points on that side and unifies them into the vision measurement coordinate system. S1.2: Four reflective circular markers are pasted on the upper surface of the curved component as pose control points. Before the assembly operation, the coordinate system of the curved component and the coordinates of the pose control points in the coordinate system are obtained by scanning with a high-precision measuring device and then reconstructed in reverse. This information is then input into the subsequent assembly and pose measurement process as known information.

3. The method for measuring the pose of curved surface components without explicit positioning features based on multi-view vision according to claim 1, characterized in that: Step S3 includes the following: S3.1: The parameters of the two sets of binocular vision subsystems were calibrated using Zhang Zhengyou's calibration method; Using a laser tracker and a single-sided binocular vision subsystem, a set of common points in their workspaces are measured respectively. Let the coordinates of the points in the laser tracker coordinate system obtained from the final measurements be... l p i (i = 1, 2, ..., n), the coordinates of the point in the coordinate system of the single-sided binocular vision subsystem are: c1 p i (i = 1, 2, ..., n), the two sets of point sets satisfy the following relationship: c1 p i =R l p i +t (5) R and t represent the relative pose between the coordinate system of the visual subsystem on one side and the coordinate system of the laser tracker; S3.2: Calculate the point set c1 P and l The center of mass of P: make Constructing the covariance matrix S3.3: Performing SVD decomposition on H yields: H = UDV T ,but: This indicates the pose of the laser tracker coordinate system in the coordinate system of the binocular vision subsystem on this side; S3.4: The pose of the laser tracker coordinate system in the coordinate system of the other binocular vision subsystem can also be calculated using the same method described above. Let the relative poses of the laser tracker and the two sets of binocular vision subsystems be respectively and The global extrinsic parameters of the multi-view vision measurement system and It can be represented as: 。 4. The method for measuring the pose of curved surface components without explicit localization features based on multi-view vision according to claim 1, characterized in that: Step S4 includes the following: S4.1: Filter the true pose control points. Constrain the spatial distance provided by the coordinate values ​​of the pose control points in the coordinate system of the curved component. Traverse the set of measurement points, take four points each time to calculate the spatial distance of each point, find four pose control points whose spatial distance meets the requirements, and take the order of the pose control points with the smallest error after coordinate transformation as the final corresponding point order. S4.2: Let the coordinates of the pose control points in the camera coordinate system be... c p i (i = 1, 2, 3, 4), the coordinates of the pose control points corresponding to the measured model coordinate system of the curved component are: t p i (i=1,2,3,4), calculate the pose of the curved component relative to the visual measurement system using equations (5)-(7).

Images