Method for measuring pose of curved surface component without clear positioning characteristic based on multi-view vision

Abstract

A multi-view-vision-based method for measuring the pose of a curved surface component without clear positioning characteristics comprises the steps that a multi-view-vision measuring system for the pose of the curved surface component is constructed, and meanwhile, four circular reflective mark points are pasted on the upper surface of the curved surface component to serve as pose control points; fitting a Gaussian curve by using a sampling point gray scale gradient value in an edge pixel point gradient direction to obtain a sub-pixel edge, and performing ellipse fitting through an improved random sampling consistency algorithm; external parameter calibration between the two binocular vision subsystems without overlapping view fields is realized through a laser tracker; based on coordinates of four pose control points under a curved surface component coordinate system and a visual measurement coordinate system, pose solving of the curved surface component without clear positioning features is achieved through SVD decomposition. According to the invention, the pose measurement of the component which has no clear positioning characteristic and is mostly a curved surface and a curve in appearance is realized, the problem that a view field is shielded by a robot end execution mechanism in the measurement process is solved, and pose information is provided for automatic assembly of the robot.

Description

Technical Field

[0001] The present invention relates to a measuring method, and more specifically to a method for measuring the posture of a curved surface component without clear positioning features based on multi-eye vision, belonging to the technical field of automated assembly measurement. Background Art

[0002] In the field of aerospace, there are curved surface components without clear positioning features. They are characterized by the lack of positioning features such as reference holes and corner points, and their shapes are mostly curved surfaces and curves. At present, their assembly is mostly done manually, which has problems such as low efficiency and assembly quality limited by operator experience. With the development of robotics technology and digital measurement technology, automated assembly is increasingly used in the field of aerospace. Robots have the advantages of high automation and flexibility, and have become an important carrier for automated and intelligent assembly in the field of aerospace. In the assembly of curved surface components without clear positioning features, industrial robots and flexible end effectors are used as carriers, combined with machine vision measurement methods, to meet their high-precision assembly needs.

[0003] In view of the characteristics of curved surface components without clear positioning features, it is difficult to estimate the pose by extracting their own features and the accuracy is difficult to guarantee. The point cloud data obtained by stereo vision or structured light technology is greatly affected by ambient light and surface material and takes a long time to process. At the same time, curved surface components require flexible and random adsorption, and the camera field of view is easily blocked by the end effector. Therefore, a pose measurement method for curved surface components without clear positioning features based on multi-eye vision is proposed to achieve accurate pose measurement in the automated assembly process. Summary of the invention

[0004] The purpose of the present invention is to meet the demand for accurate posture measurement in the automated assembly of curved surface components without clear positioning features, to solve the problem that the objects to be assembled have no clear positioning features and their shapes are mostly curved surfaces and curves. It is difficult to directly solve the posture of the assembly process by extracting object features. At the same time, there is a problem that the robot's end effector blocks the camera's field of view during the measurement process.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A method for measuring the position and posture of a curved surface component without clear positioning features based on multi-eye vision comprises the following steps:

[0007] S1: Construction of multi-view visual measurement system for curved surface component pose, with four circular reflective marking points pasted on the upper surface of the curved surface component as pose control points;

[0008] S2: The Gaussian curve is fitted with the grayscale gradient value of the sampling point in the gradient direction of the edge pixel point to obtain the sub-pixel edge, and the center of the pose control point is accurately extracted through the improved random sampling consensus (RANSAC) algorithm;

[0009] S3: realize external parameter calibration between two binocular vision subsystems through laser tracker;

[0010] S4: Based on the coordinates of the four pose control points in the surface component coordinate system and the visual measurement coordinate system, the pose solution of the surface component without clear positioning features is realized through SVD decomposition.

[0011] As a preferred solution of the present invention, step S1 includes the following contents:

[0012] S1.1: The multi-eye vision measurement system consists of two sets of binocular vision subsystems with non-overlapping fields of view, located on both sides of the robot's end effector. Each subsystem measures the three-dimensional coordinates of the two pose control points on that side and unifies them into the visual measurement coordinate system.

[0013] S1.2: Four reflective circular marking points are pasted on the upper surface of the curved component as posture control points. Before assembly, the coordinate system of the curved component and the coordinates of the posture control points in this coordinate system are scanned by high-precision measuring equipment and then reversely reconstructed and input as known information into the subsequent assembly and posture measurement process.

[0014] As a preferred solution of the present invention, step S2 includes the following contents:

[0015] S2.1: The visual measurement system collects images, performs binarization processing and extracts contours, sets the contour roundness and area thresholds to filter the ROI (region of interest) corresponding to the pose control points to improve the processing speed. To ensure the extraction stability under different lighting environments, the OTSU algorithm is used to implement the adaptive threshold of Canny edge detection, and then the ROI image of the pose control points is processed to obtain the whole pixel edge;

[0016] S2.2: Calculate the x-direction grayscale gradient G of the integer pixel edge point of the pose control point through the Sobel operator x , gray gradient G in the y direction y , and then calculate the gradient amplitude G i and the gradient direction θ i :

[0017]

[0018] S2.3: Sample 3 points on both sides of the gradient direction of the integer pixel edge point with a step size of 1, and calculate its gradient amplitude by bilinear interpolation, with the sampling point subscript x j and the gradient amplitude G at that pointj Construct a point set (x j ,G j ) to fit a Gaussian curve, where μ is the mean and σ is the standard deviation. Substitute the constructed point set into the Gaussian function expression and take the logarithm on both sides of the equation:

[0019]

[0020] It conforms to the form of a quadratic curve, The three coefficients of the quadratic curve can be solved by the least squares method through the constructed point set, and the center position of the Gaussian curve can be expressed as Then the sub-pixel coordinates of the pose control point ( s x i , s y i )for:

[0021]

[0022] By processing all pixel edge points, the sub-pixel edge of the pose control point can be obtained;

[0023] S2.4: The general equation of an ellipse is: Ax 2 +Bxy+Cy 2 +Dx+Ey+F=0, in order to effectively remove the external points caused by image noise or interference contour, the improved RANSAC (Random Sample Consensus) algorithm is used for ellipse fitting, that is, in the process of iteratively fitting the ellipse to select the inner points, in order to ensure the fitting accuracy when the sampling point distribution accounts for a small proportion of the ellipse contour, the super least squares method is used to fit the ellipse. The center (x0, y0) of the final pose control point can be expressed as:

[0024]

[0025] As a preferred solution of the present invention, step S3 includes the following contents:

[0026] S3.1: Complete the calibration of two sets of binocular vision subsystem parameters using Zhang Zhengyou calibration method. Use the laser tracker and the unilateral binocular vision subsystem to measure a set of common points in their workspaces respectively, and let the coordinates of the points in the laser tracker coordinate system be l p i (i=1,2,...,n), the coordinates of the point in the unilateral binocular vision subsystem coordinate system are c1 p i (i=1,2,...,n), the two sets of points satisfy the following relationship:

[0027] c1 p i =Rl p i +t (13)

[0028] R and t represent the relative position between the visual subsystem coordinate system and the laser tracker coordinate system on one side;

[0029] S3.2: Calculation point set c1 P and l The centroid of P: make Constructing the covariance matrix

[0030] S3.3: Perform SVD decomposition on H and we can get: H = UDV T ,but:

[0031]

[0032]

[0033] That is, it represents the position and posture of the laser tracker coordinate system in the binocular vision subsystem coordinate system of that side;

[0034] S3.4: The above method can also be used to calculate the position and posture of the laser tracker coordinate system in the coordinate system of the other side binocular vision subsystem. The relative positions of the laser tracker and the two sets of binocular vision subsystems are and The global external parameter of the multi-view vision measurement system is and It can be expressed as:

[0035]

[0036] As a preferred solution of the present invention, step S4 includes the following contents:

[0037] S4.1: Filter the true pose control points, constrain them through the spatial distance provided by the coordinate values ​​of the pose control points in the curved component coordinate system, traverse the measurement point set, take four points each time to calculate the spatial distance of each point, find four pose control points whose spatial distances meet the requirements, and use the pose control point sequence with the smallest error after the coordinates of the four points are converted as the final corresponding point sequence.

[0038] S4.2: The coordinates of the pose control points in the camera coordinate system are c p i (i=1,2,3,4), the corresponding position control point coordinates in the measured model coordinate system of the curved component are: t p i(i=1,2,3,4), the position and orientation of the curved component relative to the visual measurement system are calculated by equations (5)-(7)

[0039] The beneficial effects of the present invention are:

[0040] 1. It realizes the precise position measurement of the surface components without clear positioning features in the aerospace field with robots as actuators during the automated assembly process, and can realize high-precision assembly of surface components without clear positioning features.

[0041] 2. By sticking reflective circular marking points on the surface of curved components as posture control points, the problem of no clear positioning features, mostly curved surfaces and curves, and difficulty in directly solving the posture of the assembly process by extracting features is solved.

[0042] 3. A multi-eye vision measurement system consisting of two sets of binocular vision subsystems with non-overlapping fields of view was constructed, avoiding the problem of the robot's end effector blocking the camera's field of view during the measurement process. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the structure of the multi-eye vision measurement system for curved surface components of the present invention.

[0044] Figure 2 It is a schematic diagram of the overall process of the present invention.

[0045] Figure 3 It is a schematic diagram of posture control point ROI image extraction and pixel edge acquisition.

[0046] Figure 4 It is a schematic diagram of sub-pixel edge processing and ellipse fitting.

[0047] Figure 5 This is the schematic diagram of the global external parameter calibration principle of the multi-eye vision measurement system.

[0048] Figure 6 It is a schematic diagram of the posture measurement of curved surface components without clear positioning features. DETAILED DESCRIPTION

[0049] The following is a clear and complete description of the technical solutions in the implementation process of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] See also Figure 1-6 .

[0051] A technical solution for a method for measuring the position and posture of a curved surface component without clear positioning features based on multi-eye vision includes the following steps:

[0052] S1: Construction of multi-view vision measurement system for curved component position;

[0053] S2: Get the sub-pixel edge of the pose control point and fit the ellipse to extract the center;

[0054] S3: External parameter calibration between two binocular vision subsystems;

[0055] S4: There is no clear positioning feature surface component pose solution.

[0056] in:

[0057] Step S1: Construction of multi-eye vision measurement system for curved component pose:

[0058] The multi-eye vision measurement system consists of two sets of binocular vision subsystems with non-overlapping fields of view, which are located on both sides of the robot's end effector. The camera and lens are fixed to the robot's flexible end effector through tooling. Each subsystem measures the three-dimensional coordinates of the two pose control points on that side and unifies them into the visual measurement coordinate system.

[0059] Four circular reflective marking points are pasted on the upper surface of the curved component as posture control points. Before assembly, the curved component coordinate system and the three-dimensional coordinates of the posture control points in the coordinate system are scanned by a blue light scanner and then reversely reconstructed, which is input as known information into the subsequent assembly and posture measurement process. To ensure good imaging effects, a red ring LED light source is used for fill light, and the camera is coaxially arranged with the light source.

[0060] Step S2: Get the sub-pixel edge of the pose control point and fit the ellipse to extract the center:

[0061] The proportion of pose control points in the collected image is small and the gray value is clearly distinguished from the background. In order to improve the processing efficiency and accuracy, the image is first binarized and the contour of the binary image is extracted. The area S feature is used to set the filtering condition, and the ROI is divided by expanding the maximum circumscribed rectangle of the filtered contour by 4 pixels.

[0062] The Canny operator is used to obtain integer pixel edges. Since the Canny operator needs to give a double threshold to divide strong and weak edges, in order to meet the stability under different measurement environments and lighting conditions, it is improved and adaptive threshold acquisition is realized based on the Otsu algorithm.

[0063] For an original grayscale image of size M×N, its integer grayscale levels are {1,2,...,n}, and the probability of grayscale level i is p i. Take the segmentation threshold as k (k∈(0,n)), divide the image into two categories I1 and I2, the probability of a pixel belonging to I1 and I2 are P1(k) and P2(k) respectively, the grayscale means of I1 and I2 are g1(k) and g2(k), then the inter-class variance σ of the image is 2 =P1(k)(g1(k)-g) 2 +P2(k)(g2(k)-g) 2 .

[0064] Take k when the variance between image classes is the largest as the high threshold and 0.5k as the low threshold to obtain the integer pixel edge of the pose control point.

[0065] The Sobel operator is used to calculate the x-direction grayscale gradient G of the integer pixel edge point of the pose control point x , gray gradient G in the y direction y , and then calculate the gradient amplitude G i and the gradient direction θ i :

[0066]

[0067] Along the gradient direction of the integer pixel edge point, three points are sampled on both sides with a step size of 1, and the gradient amplitude is calculated by bilinear interpolation. j and the gradient amplitude G at that point j Construct a point set (x j ,G j ) to fit a Gaussian curve, where μ is the mean and σ is the standard deviation. Substitute the constructed point set into the Gaussian function expression and take the logarithm on both sides of the equation:

[0068]

[0069] It conforms to the form of a quadratic curve, The three coefficients of the quadratic curve can be solved by the least squares method through the constructed point set, and the center position of the Gaussian curve can be expressed as Then the sub-pixel coordinates of the pose control point ( s x i , s y i )for:

[0070]

[0071] By processing all pixel edge points, the sub-pixel edge of the pose control point can be obtained;

[0072] The general equation of an ellipse is: Ax 2 +Bxy+Cy 2+Dx+Ey+F=0, in order to effectively remove the external points caused by image noise or interference contour, the improved RANSAC (Random Sample Consensus) algorithm is used for ellipse fitting, that is, in the process of iteratively fitting the ellipse to select the inner points, in order to ensure the fitting accuracy when the sampling point distribution accounts for a small proportion of the ellipse contour, the super least squares method is used to fit the ellipse. The center (x0, y0) of the final pose control point can be expressed as:

[0073] Step S3: External parameter calibration between two binocular vision subsystems:

[0074] The calibration of internal and external parameters of two sets of binocular vision subsystems was completed using Zhang Zhengyou calibration method.

[0075] Target ball seats are arranged on both sides of the calibration fixture to place visual target balls or laser target balls. The posture of the calibration fixture is changed multiple times in the measurement space, and a series of space points P = [p1, p2, ..., p n ], and at the same time, two sets of binocular vision subsystems collect images of the visual target ball in their respective fields of view, and obtain two sets of spatial points C = [c1, c2, ..., c m ]、C′=[c′ m+1 ,c′ m+2 ,...,c n ′], which correspond one-to-one to the measurement points of the laser tracker.

[0076] Taking the binocular vision subsystem on one side as an example, it satisfies:

[0077] c i =Rp i +t (21)

[0078] R and t represent the relative position between the visual subsystem coordinate system and the laser tracker coordinate system on one side;

[0079] Calculate the centroid of point sets P and C: make Constructing the covariance matrix

[0080] Performing SVD decomposition on H, we can get: H = UDV T ,but:

[0081]

[0082]

[0083] That is, it represents the position and posture of the laser tracker coordinate system in the binocular vision subsystem coordinate system of that side;

[0084] According to the above method, the position and orientation of the laser tracker coordinate system in the coordinate system of the binocular vision subsystem on the other side can also be calculated. The relative positions of the laser tracker and the two sets of binocular vision subsystems are and The global external parameter of the multi-view vision measurement system is and It can be expressed as:

[0085]

[0086] Step S4: Position and posture calculation of surface components without clear positioning features:

[0087] The spatial distance provided by the coordinate values ​​of the pose control points in the curved component coordinate system is used to constrain the measurement point set. Four points are taken each time to calculate the spatial distance of each point, and four pose control points whose spatial distances meet the requirements are found. The order of pose control points with the smallest error after the coordinates of the four points are converted is used as the final corresponding point order.

[0088] The coordinates of the pose control point in the camera coordinate system are c p i (i=1,2,3,4), the corresponding position control point coordinates in the measured model coordinate system of the curved component are: t p i (i=1,2,3,4), the position and orientation of the curved component relative to the visual measurement system are calculated by equations (5)-(7) c t T.

[0089] In the description of the present invention, it is necessary to understand that the indicated orientation or positional relationship is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0090] In the present invention, unless otherwise clearly stipulated and limited, for example, it can be a fixed connection, a detachable connection, or an integrated one; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, for ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0091] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0092] The present invention does not involve parts that are the same as the prior art or can be implemented by using the prior art.

Claims

1. A method for measuring the position and posture of a curved surface component without clear positioning features based on multi-eye vision, characterized in that: The following steps are involved: S1: Construction of multi-view visual measurement system for curved surface component pose, with four circular reflective marking points pasted on the upper surface of the curved surface component as pose control points; S2: Fit the Gaussian curve with the grayscale gradient value of the sampling point in the gradient direction of the edge pixel point to obtain the sub-pixel edge, and perform ellipse fitting through the improved random sampling consensus (RANSAC) algorithm to achieve accurate extraction of the center of the pose control point; S3: realize external parameter calibration between two binocular vision subsystems through laser tracker; S4: Based on the coordinates of the four pose control points in the surface component coordinate system and the visual measurement coordinate system, the pose solution of the surface component without clear positioning features is realized through SVD decomposition.

2. The method for measuring the position and posture of a curved surface component without clear positioning features based on multi-eye vision according to claim 1 is characterized in that: The step S1 includes the following contents: S1.1: The multi-eye vision measurement system consists of two sets of binocular vision subsystems with non-overlapping fields of view, located on both sides of the robot's end effector. Each subsystem measures the three-dimensional coordinates of the two pose control points on that side and unifies them into the visual measurement coordinate system. S1.2: Four reflective circular marking points are pasted on the upper surface of the curved component as posture control points. Before assembly, the coordinate system of the curved component and the coordinates of the posture control points in this coordinate system are scanned by high-precision measuring equipment and then reversely reconstructed and input as known information into the subsequent assembly and posture measurement process.

3. The method for measuring the position and posture of a curved surface component without clear positioning features based on multi-eye vision according to claim 1 is characterized in that: The step S2 includes the following contents: S2.1: The visual measurement system collects images, performs binarization processing and extracts contours, sets the contour roundness and area thresholds to filter the ROI (region of interest) corresponding to the pose control points to improve the processing speed. To ensure the extraction stability under different lighting environments, the OTSU algorithm is used to implement the adaptive threshold of Canny edge detection, and then the ROI image of the pose control points is processed to obtain the whole pixel edge; S2.2: Calculate the x-direction grayscale gradient G of the integer pixel edge point of the pose control point through the Sobel operator x , gray gradient G in the y direction y , and then calculate the gradient amplitude G i and the gradient direction θ i : S2.3: Sample 3 points on both sides of the gradient direction of the integer pixel edge point with a step size of 1, and calculate its gradient amplitude by bilinear interpolation, with the sampling point subscript x j and the gradient amplitude G at that point j Construct a point set (x j ,G j ) to fit a Gaussian curve, where μ is the mean and σ is the standard deviation. Substitute the constructed point set into the Gaussian function expression and take the logarithm on both sides of the equation: It conforms to the form of a quadratic curve, The three coefficients of the quadratic curve can be solved by the least squares method through the constructed point set, and the center position of the Gaussian curve can be expressed as Then the sub-pixel coordinates of the pose control point ( s x i , s y i )for: By processing all pixel edge points, the sub-pixel edge of the pose control point can be obtained; S2.4: The general equation of an ellipse is: Ax 2 +Bxy+Cy 2 +Dx+Ey+F=0, in order to effectively remove the external points caused by image noise or interference contour, the improved RANSAC (Random Sample Consensus) algorithm is used for ellipse fitting, that is, in the process of iteratively fitting the ellipse to select the inner points, in order to ensure the fitting accuracy when the sampling point distribution accounts for a small proportion of the ellipse contour, the super least squares method is used to fit the ellipse. The center coordinates (x0, y0) of the final pose control point can be expressed as:

4. The method for measuring the position and posture of a curved surface component without clear positioning features based on multi-eye vision according to claim 1 is characterized in that: The step S3 includes the following contents: S3.1: Complete the calibration of two sets of binocular vision subsystem parameters using Zhang Zhengyou calibration method. Use the laser tracker and the unilateral binocular vision subsystem to measure a set of common points in their workspaces respectively, and let the coordinates of the points in the laser tracker coordinate system be l p i (i=1,2,…,n), the coordinates of the point in the unilateral binocular vision subsystem coordinate system are c1 p i (i=1,2,...,n), the two sets of points satisfy the following relationship: c1 p i =R l p i +t (5) R and t represent the relative position between the visual subsystem coordinate system and the laser tracker coordinate system on one side; S3.2: Calculation point set c1 P and l The centroid of P: make Constructing the covariance matrix S3.3: Perform SVD decomposition on H and we can get: H = UDV T ,but: That is, it represents the position and posture of the laser tracker coordinate system in the binocular vision subsystem coordinate system of that side; S3.4: The above method can also be used to calculate the position and posture of the laser tracker coordinate system in the coordinate system of the other side binocular vision subsystem. The relative positions of the laser tracker and the two sets of binocular vision subsystems are and The global external parameter of the multi-view vision measurement system is and It can be expressed as:

5. The method for measuring the position and posture of a curved surface component without clear positioning features based on multi-eye vision according to claim 1, characterized in that: The step S4 includes the following contents: S4.1: Filter the true pose control points, constrain them through the spatial distance provided by the coordinate values ​​of the pose control points in the curved component coordinate system, traverse the measurement point set, take four points each time to calculate the spatial distance of each point, find four pose control points whose spatial distances meet the requirements, and use the pose control point sequence with the smallest error after the coordinates of the four points are converted as the final corresponding point sequence. S4.2: The coordinates of the pose control points in the camera coordinate system are c p i (i=1,2,3,4), the corresponding position control point coordinates in the measured model coordinate system of the curved component are: t p i (i=1,2,3,4), the position and orientation of the curved component relative to the visual measurement system are calculated by equations (5)-(7)

Images