A relative posture measurement method for assembly surface positioning based on waist-shaped hole features

Abstract

The present invention provides a method for measuring the relative pose of an assembly surface based on waist-shaped hole features, comprising the following steps: Step 1: extracting a region of interest (ROI); Step 2: edge extraction; Step 3: screening the edge point set to obtain a point set of two arc segments; Step 4: performing ellipse fitting to obtain an ellipse equation on the image plane; Step 5: solving the pose of the corresponding spatial circle to obtain the center of the corresponding spatial circle and the normal vector of the spatial plane in which it is located; Step 6: obtaining the normal vector of the plane in which the waist-shaped hole is located and the coordinates of the geometric center point of the waist-shaped hole; Step 7: obtaining the pose information of the waist-shaped hole. The present invention solves the problem of measuring the relative pose of waist-shaped hole features on the assembly surface, provides effective pose parameters for assembly and docking, and achieves a pose accuracy of 0.1 mm, effectively improving assembly quality and efficiency.

Description

Technical Field

[0001] The invention belongs to the field of docking assembly, and in particular relates to a method for measuring the relative posture of an assembly surface based on waist-shaped hole features. Background Art

[0002] Docking is a critical part of the equipment process. Traditionally, docking assembly involves hoisting or mobile floor tooling to dock components, followed by manual assistance. For example, in the automotive assembly field, to ensure the relative positions of the chassis and body meet docking requirements, the body is typically aligned manually or mechanically positioned. This results in low docking accuracy, efficiency, and automation. With the advancement of modern technology, the use of visual measurement technology to accurately measure the relative positions of mating surfaces during docking can effectively improve the quality and efficiency of assembly and docking.

[0003] Visual pose measurement technology first identifies and extracts target features, then calculates the target's position by applying constraints such as reprojection of the extracted feature points, slope constraints on feature lines, and radius constraints on feature circles. Patent [CN108090931A] proposes a marker recognition and pose measurement method based on circle and cross features that is resistant to occlusion and interference. Patent [CN104517291] proposes a pose measurement method based on target coaxial circle features. Summary of the Invention

[0004] Purpose of the invention: The methods proposed in the prior art are not applicable to the waist-shaped hole features on the assembly surface. Therefore, the present invention proposes a method for measuring the relative posture of the assembly surface based on the waist-shaped hole features, so as to realize high-precision relative posture measurement of the waist-shaped hole features on the assembly surface during docking assembly.

[0005] The method comprises the following steps:

[0006] Step 1: Extract the region of interest (ROI): Obtain the original image, then use the YOLO neural network to detect the waist-shaped hole features from the original image, and select the largest rectangular image area where the waist-shaped hole is located as the region of interest (ROI);

[0007] Step 2, edge extraction: perform filtering, histogram equalization, morphological operations on the region of interest (ROI), and then perform edge extraction based on polynomial interpolation on the preprocessed image to obtain the edge point set of the waist-shaped hole;

[0008] Step 3: Filter the edge point set obtained in step 2 to obtain the point set of two arc segments;

[0009] Step 4, using the point set obtained in step 3 to perform ellipse fitting to obtain the ellipse equation on the image plane;

[0010] Step 5: Use the two ellipse equations obtained in step 4 to solve the position of the corresponding space circle to obtain the center of the corresponding space circle and the normal vector of the space plane;

[0011] Step 6, using the information calculated in step 5, obtain the normal vector of the plane where the waist-shaped hole is located and the coordinates of the geometric center point of the waist-shaped hole;

[0012] Step 7, using the information calculated in step 6 to obtain the posture information of the waist-shaped hole.

[0013] Step 3 includes:

[0014] Step 3-1, use Hough transformation to fit the two straight lines of the waist-shaped hole straight segment to obtain the slopes k1 and k2 and the intercepts b1 and b2 of the two straight lines;

[0015] Step 3-2, use the point-to-line formula to obtain the distance L from all points to the two lines 1i , L 2i ;

[0016] Step 3-3, when point i satisfies L 1i or L 2i If the value is less than or equal to the threshold value 0.5, the point i is retained, and the point set of the upper and lower arcs is obtained;

[0017] In steps 3-4, the remaining arc segment point set is divided into an upper arc segment point set U1 and a lower arc segment point set U2 based on the arc segment's position on the image. Step 3-3 yields a single point set containing all arc segment points, so the subsequent steps 3-4 require this division. This division is for the subsequent ellipse fitting. Using two upper and lower point sets in step 4 reduces the computational effort compared to using a single point set.

[0018] Step 4 includes:

[0019] Step 4-1, randomly select five points from the upper arc segment point set U1 and the lower arc segment point set U2;

[0020] Step 4-2, calculate the equation of the ellipse formed by the five points;

[0021] Step 4-3: Substitute all points in the upper arc segment point set U1 and the lower arc segment point set U2 into the ellipse equation calculated in step 4-2. If the calculated result is less than or equal to 0.1, it means that the points satisfy the ellipse equation. Count the number of points that satisfy the ellipse equation.

[0022] Step 4-4, repeat steps 4-1 to 4-3 until the number of points that satisfy the ellipse equation is greater than or equal to the threshold, and take the obtained ellipse as the final result to obtain the following arc point set U i The corresponding ellipse equation E i :

[0023] a i u 2 +b i v 2 +c i uv+d i u+e i v+f i =0

[0024] where a i 、b i 、c i d i 、e i 、f i To calculate the equation parameters of the i-th ellipse, i=1, 2, (u, v) is the image coordinate.

[0025] Step 5 includes:

[0026] Step 5-1, Camera Model f0 is the focal length of the camera, (x, y, z) is the coordinate of the point with image coordinates (u, v) in camera coordinates;

[0027] Obtained from the camera model in the camera coordinate system O c -X c Y c Z c The equation of the elliptical cone is:

[0028] A i x 2 +B i y 2 +C i xy+D i xz+E i yz+F i Z 2 =0

[0029] Among them, the intermediate parameter A i =a i f0 2 , intermediate parameter B i =b i f0 2 , intermediate parameter C i =c i f0 2 , intermediate parameter D i =d i f0, intermediate parameter E i =e i f0, intermediate parameter F i =f i ;

[0030] The elliptical cone equation is rewritten into the following matrix form:

[0031] [xyz]Q i [xyz] T =0

[0032]

[0033] Among them, Q i is the parameter matrix of the elliptical cone equation, T represents the matrix transpose;

[0034] Step 5-2, transform the elliptical cone equation in the camera coordinate system in step 5-1 to a new coordinate system O c -X ci 'Y ci 'Z ci ', so that the equation of the elliptical cone is transformed into the standard form:

[0035] New coordinate system O c -X ci 'Y ci 'Z ci 'With the camera coordinate system O c -X c Y c Z c There is only a rotation relationship between them, expressed as [xy z] T =P i [x i ′y i ′z i ′] T , P i is a 3×3 matrix, x′ i 、y i ′、z i ′ is the coordinates x, y, z in the new coordinate system O c -X ci 'Y ci 'Z ci 'Coordinates in the new coordinate system O c -X ci 'Y ci 'Z ci The matrix form of the elliptical cone equation is:

[0036] [x i ′y i ′z i ′]P i -1 Q i P i [x i ′y i ′z i ′]T =0;

[0037] Step 5-3, by matrix Q i The eigenvalue decomposition of Q is obtained i The eigenvalue of Coefficients of the standard elliptical cone equation and matrix is the eigenvalue The corresponding eigenvector is determined by the following method, where is the eigenvalue The corresponding eigenvector, is the eigenvalue The corresponding eigenvector, is the eigenvalue The corresponding eigenvectors are:

[0038] Adjustment order, so that and Same number and and Different signs, find the corresponding normalized eigenvector

[0039]

[0040] if but otherwise No change;

[0041] Step 5-4, using the coefficients of the standard elliptical cone equation and the actual radius R of the semicircular arc at both ends of the waist-shaped hole, determine the space circle corresponding to the arc in the new coordinate system O c -X ci 'Y ci 'Z ci 'Normal vector n of the plane below i ′ and the center coordinate c i ′:

[0042]

[0043]

[0044] in is the normal vector n i ' in O c X ci 'Projection in direction, ′ is the normal vector n i ' in O c Y ci'Projection in direction, is the normal vector n i ' in O c Z ci 'Projection in direction; is the coordinate of the center of the circle c i ' in O c X ci 'Projection in direction, is the coordinate of the center of the circle c i ' in O c Y ci 'Projection in direction, is the coordinate of the center of the circle c i ' in O c Z ci 'Projection in direction;

[0045] Step 5-5, using the rotation matrix P i Get the camera coordinate system O c -X c Y c Normal vector n under Z i and the center coordinates c i :

[0046]

[0047]

[0048] in is the normal vector n i In O c X c Projection in direction, is the normal vector n i In O c Y c Projection in direction, is the normal vector n i In O c Z c Projection in direction. is the coordinate of the center of the circle c i In O c X c Projection in direction, is the coordinate of the center of the circle c i In O c Y ci 'Projection in direction, is the coordinate of the center of the circle c i In O c Z ci 'Projection in direction.

[0049] Step 6 includes:

[0050] Step 6-1, the two sets of solutions obtained from the space circle C1 are c 11 and n 11 、c 12 and n 12 , the two sets of solutions obtained by the space circle C2 are c 21 and n 21 、c 22 and n 22 , calculate the angle α between the two space circle normal vectors:

[0051]

[0052] Where i = 1, 2;

[0053] Step 6-2, if α>90°, let n 2i =-n 2i , and then calculate n 1i With n 2i The angle between

[0054] Step 6-3: Since the arcs at both ends of the waist-shaped hole are on the same plane, the normal vectors of the plane where the arcs at both ends correspond to the space circles must be parallel; when the angle between one set of vectors satisfies 0°≤α≤0.5°, the n in the solution of the set is 1i and the corresponding center coordinates c 1i is the correct solution of the normal vector n1 and the center coordinate c1 of the space circle C1. 2i and the corresponding center coordinates c 2i is the correct solution for the normal vector n2 and center coordinate c2 of the space circle C2;

[0055] Step 6-4, the normal vector n of the plane where the waist-shaped hole is located is (n1+n2) / 2, the centers of the arcs at both ends are c1 and c2 respectively, and the geometric center of the waist-shaped hole is c=(c1+c2) / 2.

[0056] Step 7 includes:

[0057] Step 7-1, the position information of the waist-shaped hole is represented by the coordinates of the geometric center c, and the posture information of the waist-shaped hole is represented by the yaw angle ψ, pitch angle θ and roll angle φ; in the image, the projection of c2 is always below the projection of c1, and the yaw angle ψ is defined as the vector c2c1 in X c O c Y c The angle between the vertical projection on the plane and the positive direction of the y-axis is in the range of [-π / 2, π / 2]. The pitch angle ψ is positive when the angle is θ, which is defined as the angle between the plane where the waist hole is located and the plane X. c O c Y cThe angle range is [-π / 2, π / 2]. When c The pitch angle θ is positive when the vertical angle is 0.001, and the roll angle φ is defined as the angle between the normal vector n and the vertical plane c1c2c1′c2′ where the straight line c1c2 is located. c1′ and c2′ are the angles of c1 in X direction. c O c Y c The vertical projection on the plane and c2 in X c O c Y c The vertical projection on the plane has a value range of [-π / 2, π / 2]. The roll angle φ is positive when the normal vector is to the right of the plane c1c2c1′c2′. The calculation formulas for the yaw angle ψ, pitch angle θ, and roll angle φ are:

[0058]

[0059] in are the center c i In O c X c Projection in direction and O c Y c The projection in the direction, n is the normal vector of the plane where the waist hole is located in the camera coordinate system, n x 、n y 、n z They are the normal vector n in O c X c Projection in direction, O c Y c Projection in direction and O c Z c Projection in direction, is the normal vector of the vertical plane c1c2c1′c2′.

[0060] Step 7-2, the position information of the waist hole c=[c x c y c z ] T =(c1+c2) / 2, attitude information e=[ψθφ] T .

[0061] The present invention also provides a storage medium storing a computer program or instruction. When the computer program or instruction is executed, the method for measuring the relative posture of the assembly surface positioning based on the waist-shaped hole feature is implemented.

[0062] Beneficial effects: Compared with traditional auxiliary assembly technologies, such as manual positioning and mechanical positioning, the relative posture measurement method for assembly surface positioning based on waist-shaped hole features proposed in the present invention solves the high-precision measurement of the posture of waist-shaped hole features on the assembly surface, and the posture accuracy reaches 0.1mm, which effectively improves the efficiency and accuracy of assembly and docking, reduces labor costs, and improves the degree of automation. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more apparent.

[0064] Figure 1 This is the waist-shaped hole feature on the automobile assembly surface of the present invention.

[0065] Figure 2 This is a flow chart of the relative posture measurement method for assembly surface positioning based on waist-shaped hole features of the present invention.

[0066] Figure 3 This is a schematic diagram of the waist-shaped hole projection imaging model of the present invention.

[0067] Figure 4 It is a schematic diagram describing the waist-shaped hole position of the present invention. DETAILED DESCRIPTION

[0068] like Figure 2 As shown, the present invention proposes a method for measuring the relative position and posture of an assembly surface based on waist-shaped hole features, comprising the following steps:

[0069] Step 1: Obtain the original image, and then use the YOLO deep neural network to complete the target detection of the waist-shaped hole feature, and use the rectangular image area where the waist-shaped hole is detected as the region of interest (ROI); Figure 1 The figure shows the waist-shaped hole feature on the automobile assembly surface of the present invention;

[0070] Step 2: Perform grayscale, filtering, histogram equalization, morphological and other operations on the region of interest (ROI) of the image in step 1, and then use polynomial interpolation to extract the target edge point set;

[0071] Step 3: Filter the edge point set obtained in step 2 to obtain the point set of two arc segments;

[0072] Step 4: Use the point set obtained in step 3 to perform ellipse fitting to obtain the ellipse equation on the image plane;

[0073] Step 5: Use the ellipse equation obtained in step 4 to solve the position of the corresponding space circle and obtain the center of the corresponding space circle and the normal vector of the space plane;

[0074] Step 6: Using the information calculated in step 5, the normal vector of the plane where the waist-shaped hole is located and the coordinates of the geometric center point of the waist-shaped hole are obtained;

[0075] Step 7: Use the information calculated in step 6 to obtain the posture information of the waist-shaped hole.

[0076] Furthermore, the ellipse equations E1 and E2 obtained in step 4 are:

[0077] a1u 2 +b1v 2 +c1uv+d1u+e1v+f1=0

[0078] a2u 2 +b2v 2 +c2uv+d2u+e2v+f2=0

[0079] Among them, a i 、b i 、c i d i 、e i 、f i are the parameters of the ellipse equation, and u and v are the pixel coordinates of the points on the ellipse.

[0080] Furthermore, the specific process of calculating the center of the space circle and the normal vector of the space plane in which it is located using the ellipse equation in step 5 is as follows:

[0081] (1) Based on the ellipse equation fitted in the image coordinate system, the camera projection model can reversely project the ellipse equation into the camera coordinate system to form an oblique elliptical cone with the camera coordinate origin as the vertex and the ellipse on the image plane as the base:

[0082] A i x 2 +B i y 2 +C i xy+D i xz+E i yz+F i Z 2 =0

[0083] It can be written in matrix form:

[0084] [xyz]Q i [xyz] T =0 where

[0085]

[0086] (2) Set the camera coordinate system O c -Xc Y c Z c The oblique elliptical cone is transformed into the new coordinate system O c -X c 'Y c 'Z c ', so that the expression of the elliptical cone in this coordinate system is O c -X c 'Y c 'Z c 'The origin of the coordinate system is the same as that of the camera coordinate system, so there is only a rotation relationship between the two coordinate systems. The rotation relationship is represented by a 3x3 matrix P. Define [xyz] T =P i [x i ′y i ′z i ′] T , we can get:

[0087] [x i ′y i ′z i ′]P i -1 Q i P i [x i ′y i ′z i ′] T =0

[0088] (3) Perform eigenvalue decomposition on the matrix Q to obtain the eigenvalue and the corresponding eigenvector Adjust the order of λ1, λ2, and λ3 so that λ1 and λ2 have different signs from λ3 and |λ1| ≥ |λ2|. The equation of the transformed standard elliptical cone is:

[0089]

[0090] (4) Solve the position and attitude information of the space circle in the standard coordinate system. Assuming that the radius of the space circle is R, solving its position and attitude is to solve the space circle in the standard coordinate system O c -X c 'Y c 'Z c 'Find a plane that intersects the standard elliptical cone to form a circle with a radius of R. The coordinates of the center of the circle and the normal vector of the plane in which the circle lies are the information required. In the standard coordinate system, based on the equation of the elliptical cone, the coordinates of the center of the circle and the normal vector of the plane in which the circle lies are:

[0091]

[0092]

[0093] In the new coordinate system O c -X c 'Y c 'Z c 'The result is converted to the camera coordinate system O c -X c Y c Z c ,get:

[0094]

[0095]

[0096] Furthermore, in step 6, the specific process of calculating the geometric center of the waist-shaped hole and the normal vector of the plane on which it is located using the four groups of solutions calculated in step 5 is as follows:

[0097] (1) Figure 3 As shown, the solution of the space circle C1 is c 11 and n 11 、c 12 and n 12 , the solution obtained by the space circle C2 is c 21 and n 21 、c 22 and n 22 . Calculate the angle α between the normal vectors of two space circles:

[0098]

[0099] (2) If α>90°, let n 2i =-n 2i , and then calculate n 1i With n 2i The angle between

[0100] (3) When the angle between a set of vectors is 0°≤α≤0.5°, the normal vector n in the solution of this set of vectors is 1i and n 2i And the corresponding center coordinates c 1i and c 2i This is the correct solution, the normal vector n1 of the space circle C1 = n 1i , the center coordinates of the space circle C1 are c1=c 1i , the normal vector n2 of the space circle C2 = n 2i , the center coordinates of the space circle C2 are c2=c 2i ;

[0101] (4) The normal vector of the plane where the waist-shaped hole is located is n = (n1 + n2) / 2, the centers of the arcs at both ends are c1 and c2 respectively, and the geometric center of the waist-shaped hole is c = (c1 + c2) / 2.

[0102] Furthermore, the specific process of determining the posture information of the waist-shaped hole in step 7 is as follows: the position information of the waist-shaped hole is represented by its geometric center c, and the posture information e of the waist-shaped hole is represented by the yaw angle ψ, the pitch angle θ and the roll angle φ. The yaw angle ψ is defined as the vector c2c1 in X c O c Y c The angle between the projection on the plane and the positive direction of the y-axis is in the range of [-π / 2, π / 2]. The pitch angle ψ is positive when θ is 0. c O c Y c The angle range is [-π / 2, π / 2]. The pitch angle θ is positive when . The roll angle φ is defined as the angle between the normal vector n and the vertical plane c1c2c1′c2' where the line c1c2 is located. The value range is [-π / 2, π / 2]. When the normal vector is on the right side of the plane c1c2c1′c2', the roll angle φ is positive, as shown in the following example: Figure 4 shown.

[0103] Example

[0104] This embodiment provides a method for measuring relative position and posture of an assembly surface based on waist-shaped hole features, which specifically includes:

[0105] Step 1: Extract ROI: Obtain the original image, then use the YOLO neural network to identify the waist-shaped hole features from the original image, and select the largest rectangular image area where the waist-shaped hole is located as the region of interest (ROI);

[0106] Step 2, image processing: through grayscale, filtering, histogram equalization, morphology and other operations

[0107] Step 3: Edge extraction: Use polynomial interpolation to extract the edge point set of the waist-shaped hole from the processed image;

[0108] Step 4, arc segment point set screening: The edge point set is screened based on Hough transform to obtain the arc segment point set;

[0109] Step 5, ellipse fitting: Use the RANSAC (Random Sample Consensus) algorithm to perform ellipse fitting on the two point sets obtained in step 4 to obtain ellipse equations E1 and E2;

[0110] Step 6, space circle solution: Calculate the center of the space circle corresponding to the ellipse in the image and the normal vector of the plane on which it is located by using the known radius R of the two arc segments of the waist-shaped hole and the ellipse fitting parameters obtained in step 5.

[0111] Step 7, waist hole posture solution: The geometric center of the waist hole is calculated by the two circle center coordinates, and the posture of the waist hole is calculated by the obtained normal vector and the two circle center coordinates.

[0112] Step 3 includes:

[0113] Step 3-1, use Hough transformation to fit the two straight lines of the waist-shaped hole straight segment to obtain the slopes k1 and k2 and the intercepts b1 and b2 of the straight lines;

[0114] Step 3-2, use the point-to-line formula to obtain the distance L from all points to the two lines 1i , L 2i ;

[0115] Step 3-3, when point i satisfies L 1i or L 2i If it is less than or equal to the threshold, this point i is retained, and the point set of the upper and lower arcs is obtained;

[0116] Step 3-4: Divide the retained arc segment point set into an upper arc segment point set U1 and a lower arc segment point set U2 according to the position of the arc segment on the image.

[0117] Step 4 includes:

[0118] Step 4-1: Randomly select five points from the obtained arc segment point set;

[0119] Step 4-2, using the five points in step 4-1 to calculate the equation of the ellipse;

[0120] In step 4-3, all points in the point set are substituted into the ellipse equation calculated in step 4-2. If the calculated result is less than or equal to 0.1, it means that the point satisfies the ellipse equation. The number of points that satisfy the ellipse equation is counted.

[0121] Step 4-4: Repeat steps 4-1 to 4-3 until the number of points that satisfy the ellipse equation is greater than or equal to 90% of the total number of points in the point set. This ellipse is taken as the final result and the ellipse equation is obtained:

[0122] a i u 2 +b i v 2 +c i uv+d i u+e i v+f i =0

[0123] Where i=1, 2.

[0124] Step 5 includes:

[0125] Step 5-1, Camera Model f0 is the focal length of the camera, and (x, y, z) is the coordinate of the point (u, v) in the image coordinates corresponding to the camera coordinates. The corresponding elliptical cone equation can be obtained from the camera model:

[0126] A i x 2 +B i y 2 +C i xy+D i xz+E i yz+F i Z 2 =0(i=1,2)

[0127] Among them: A i =a i f0 2 , B i =b i f0 2 , C i =c i f0 2 , D i =d i f0,E i =e i f0,F i =f i .

[0128] The elliptical cone equation is rewritten into the following matrix form:

[0129] x=[xyz]Q i [xyz] T =0

[0130]

[0131] Step 5-2, transform the elliptical cone equation in the camera coordinate system in step 5-1 to a new coordinate system O c -X ci 'Y ci 'Z ci ', so that the equation of the elliptical cone is transformed into a standard form, as New coordinate system O c -X ci 'Y ci 'Z ci 'To ensure that the camera coordinate system O c -X c Yc Z c There is only a rotation relationship between them, which can be expressed as [x yz] T =P i [x i ′y i ′z i ′] T , P i Is a 3×3 matrix. In the new coordinate system O c -X ci 'Y ci 'Z ci The matrix form of the elliptical cone equation is:

[0132] [x i ′y i ′z i ′]P i -1 Q i P i [x i ′y i ′z i ′] T =0.

[0133] Step 5-3, by matrix Q i The eigenvalue decomposition of Q is obtained i The eigenvalue of and the corresponding normalized eigenvector Coefficients of the standard elliptical cone equation and matrix Determine by the following methods: ① Adjust order, so that and Same number and Different name;② ③ if So on the contrary No change.

[0134] Step 5-4, the eigenvalues ​​obtained in step 5-3 and the actual radius R of the semicircular arc at both ends of the waist-shaped hole, determine the space circle corresponding to the arc in the new coordinate system O c -X ci 'Y ci 'Z ci 'Normal vector n of the plane below i ′ and the center coordinate c′ i :

[0135]

[0136]

[0137] Step 5-5, using the rotation matrix P i Get the camera coordinate system O c -X c Y c Normal vector n under Z i and the center coordinates c i :

[0138]

[0139]

[0140] Step 6 includes:

[0141] Step 6-1, the two sets of solutions obtained from the space circle C1 are c 11 and n 11 、c 12 and n 12 , the two sets of solutions obtained by the space circle C2 are c 21 and n 21 、c 22 and n 22 , calculate the angle α between the two space circle normal vectors:

[0142]

[0143] Where i = 1, 2;

[0144] Step 6-2, if α>90°, let n 2i =-n 2i , and then calculate n 1i With n 2i The angle between

[0145] Step 6-3: Since the arcs at both ends of the waist-shaped hole are on the same plane, the normal vectors of the plane where the arcs at both ends correspond to the space circles must be parallel; when the angle between one set of vectors satisfies 0°≤α≤0.5°, the n in the solution of the set is 1i and the corresponding center coordinates c 1i is the correct solution of the normal vector n1 and the center coordinate c1 of the space circle C1. 2i and the corresponding center coordinates c 2i is the correct solution for the normal vector n2 and center coordinate c2 of the space circle C2;

[0146] Step 6-4, the normal vector n of the plane where the waist-shaped hole is located is (n1+n2) / 2, the centers of the arcs at both ends are c1 and c2 respectively, and the geometric center of the waist-shaped hole is c=(c1+c2) / 2.

[0147] Step 7 includes:

[0148] Step 7-1, the position information of the waist-shaped hole is represented by the coordinates of the geometric center c, and the posture information of the waist-shaped hole is represented by the yaw angle ψ, pitch angle θ and roll angle φ; in the image, the projection of c2 is always below the projection of c1, and the yaw angle ψ is defined as the vector c2c1 in X c O c Y c The angle between the vertical projection on the plane and the positive direction of the y-axis is in the range of [-π / 2, π / 2]. The pitch angle ψ is positive when the angle is θ, which is defined as the angle between the plane where the waist hole is located and the plane X. c O c Y c The angle range is [-π / 2, π / 2]. The pitch angle θ is positive when the vertical angle is 0.001, and the roll angle φ is defined as the angle between the normal vector n and the vertical plane c1c2c1′c2′ where the straight line c1c2 is located. c′1 and c2′ are the angles of c1 in X c O c Y c The vertical projection on the plane and c2 in X c O c Y c The vertical projection on the plane has a value range of [-π / 2, π / 2]. The roll angle φ is positive when the normal vector is to the right of the plane c1c2c1′c2′. The calculation formulas for the yaw angle ψ, pitch angle θ, and roll angle φ are:

[0149]

[0150] in are the center c i In O c X c Projection in direction and O c Y c The projection in the direction, n is the normal vector of the plane where the waist hole is located in the camera coordinate system, n x 、n y 、n z They are the normal vector n in O c X c Projection in direction, O c Y c Projection in direction and O c Z c Projection in direction, is the normal vector of the vertical plane c1c2c1′c2′.

[0151] Step 7-2, the position of the waist hole is as follows Figure 4 As shown, the position information of the waist hole c=[cx c y c z ] T =(c1+c2) / 2, attitude information e=[ψ θ φ] T .

[0152] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium is capable of storing a computer program. When the computer program is executed by the data processing unit, it can run the invention content of the method for measuring the relative posture of the assembly surface positioning based on the waist-shaped hole feature provided by the present invention and some or all of the steps in each embodiment. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0153] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a computer program, i.e., a software product. The computer program software product can be stored in a storage medium and includes several instructions for enabling a device including a data processing unit (which can be a personal computer, a server, a single-chip microcomputer, a MUU, or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0154] The present invention provides a method for measuring the relative position and orientation of an assembly surface based on waist-shaped hole features. There are numerous methods and approaches for implementing this technical solution. The above is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the present invention. Any components not specified in this embodiment may be implemented using existing technologies.

Claims

1. A method for measuring relative position and posture of an assembly surface based on waist-shaped hole features, characterized in that: The following steps are involved: Step 1: Extract the region of interest (ROI): Obtain the original image, then use the YOLO neural network to detect the waist-shaped hole features from the original image, and select the largest rectangular image area where the waist-shaped hole is located as the region of interest (ROI); Step 2, edge extraction: Filter, histogram equalization, and morphological operations are performed on the region of interest (ROI), and then edge extraction based on polynomial interpolation is performed on the image to obtain the edge point set of the waist-shaped hole; Step 3: Filter the edge point set obtained in step 2 to obtain the point set of two arc segments; Step 4, using the point set obtained in step 3 to perform ellipse fitting to obtain the ellipse equation on the image plane; Step 4 includes: Step 4-1, randomly select five points from the upper arc segment point set U1 and the lower arc segment point set U2; Step 4-2, calculate the equation of the ellipse formed by the five points; Step 4-3: Substitute all points in the upper arc segment point set U1 and the lower arc segment point set U2 into the ellipse equation calculated in step 4-2. If the calculated result is less than or equal to 0.1, it means that the points satisfy the ellipse equation. Count the number of points that satisfy the ellipse equation. Step 4-4, repeat steps 4-1 to 4-3 until the number of points that satisfy the ellipse equation is greater than or equal to the threshold, and take the obtained ellipse as the final result to obtain the following arc point set U i The corresponding ellipse equation E i : a i u 2 +b i v 2 +c i uv+d i u+e i v+f i =0 where a i 、b i 、c i d i 、e i 、f i To calculate the equation parameters of the i-th ellipse, i = 1, 2, (u, v) is the image coordinate; Step 5: Use the two ellipse equations obtained in step 4 to solve the position of the corresponding space circle to obtain the center of the corresponding space circle and the normal vector of the space plane; Step 6, using the information calculated in step 5, obtain the normal vector of the plane where the waist-shaped hole is located and the coordinates of the geometric center point of the waist-shaped hole; Step 7, using the information calculated in step 6 to obtain the posture information of the waist-shaped hole.

2. The method according to claim 1, characterized in that Step 3 includes: Step 3-1, use Hough transformation to fit the two straight lines of the waist-shaped hole straight segment to obtain the slopes k1 and k2 and the intercepts b1 and b2 of the two straight lines; Step 3-2, use the point-to-line formula to obtain the distance L from all points to the two lines 1i , L 2i ; Step 3-3, when point i satisfies L 1i or L 2i If the value is less than or equal to the threshold value 0.5, the point i is retained, and the point set of the upper and lower arcs is obtained; Step 3-4: Divide the retained arc segment point set into an upper arc segment point set U1 and a lower arc segment point set U2 according to the position of the arc segment on the image.

3. The method according to claim 2, characterized in that Step 5 includes: Step 5-1, Camera Model f0 is the focal length of the camera, (x, y, z) is the coordinate of the point with image coordinates (u, v) in camera coordinates; Obtained from the camera model in the camera coordinate system O c -X c Y c Z c The equation of the elliptical cone is: A i x 2 +B i y 2 +C i xy+D i xz+E i yz+F i z 2 =0 Among them, the intermediate parameter A i =a i f0 2 , intermediate parameter B i =b i f0 2 , intermediate parameter C i =c i f0 2 , intermediate parameter D i =d i f0, intermediate parameter E i =e i f0, intermediate parameter F i =f i ; The elliptical cone equation is rewritten into the following matrix form: [xyz]Q i [xyz] T =0 Among them, Q i is the parameter matrix of the elliptical cone equation, T represents the matrix transpose; Step 5-2, transform the elliptical cone equation in the camera coordinate system in step 5-1 to a new coordinate system O c -X ci 'Y ci 'Z ci ', so that the equation of the elliptical cone is transformed into the standard form: New coordinate system O c -X ci 'Y ci 'Z ci 'With the camera coordinate system O c -X c Y c Z c There is only a rotation relationship between them, expressed as [xyz] T =P i [x i ′y i ′z i ′] T , P i is a 3×3 matrix, x i ′、y i ′、z i ′ is the coordinates x, y, z in the new coordinate system O c -X ci 'Y ci 'Z ci 'Coordinates in the new coordinate system O c -X ci 'Y ci 'Z ci The matrix form of the elliptical cone equation is: [x i ′ y i ′ z i ′]P i -1 Q i P i [x i ′ y i ′ z i ′] T =0; Step 5-3, by matrix Q i The eigenvalue decomposition of Q is obtained i The eigenvalue of Coefficients of the standard elliptical cone equation and matrix Determined by the following method, is the eigenvalue The corresponding eigenvector, is the eigenvalue The corresponding eigenvector, is the eigenvalue The corresponding eigenvectors are: Adjustment order, so that and Same number and and Different signs, find the corresponding normalized eigenvector if but otherwise No change; Step 5-4, using the coefficients of the standard elliptical cone equation and the actual radius R of the semicircular arc at both ends of the waist-shaped hole, determine the space circle corresponding to the arc in the new coordinate system O c -X ci 'Y ci 'Z ci 'Normal vector n of the plane below i ′ and the center coordinate c i ′: in is the normal vector n i ' in O c X ci 'Projection in direction, is the normal vector n i ' in O c Y ci 'Projection in direction, is the normal vector n i ' in O c Z ci 'Projection in direction; is the coordinate of the center of the circle c i ' in O c X ci 'Projection in direction, is the coordinate of the center of the circle c i ' in O c Y ci 'Projection in direction, is the coordinate of the center of the circle c i ' in O c Z ci 'Projection in direction; Step 5-5, using the rotation matrix P i Get the camera coordinate system O c -X c Y c Normal vector n under Z i and the center coordinates c i : in is the normal vector n i In O c X c Projection in direction, is the normal vector n i In O c Y c Projection in direction, is the normal vector n i In O c Z c Projection in direction; is the coordinate of the center of the circle c i In O c X c Projection in direction, is the coordinate of the center of the circle c i In O c Y ci 'Projection in direction, is the coordinate of the center of the circle c i In O c Z ci 'Projection in direction.

4. The method according to claim 3, characterized in that Step 6 includes: Step 6-1, the two sets of solutions obtained from the space circle C1 are c 11 and n 11 、c 12 and n 12 , the two sets of solutions obtained by the space circle C2 are c 21 and n 21 、c 22 and n 22 , calculate the angle α between the two space circle normal vectors: Where i = 1, 2; Step 6-2, if α>90°, let n 2i =-n 2i , and then calculate n 1i With n 2i The angle between Step 6-3: Since the arcs at both ends of the waist-shaped hole are on the same plane, the normal vectors of the plane where the arcs at both ends correspond to the space circles must be parallel. When the angle between one set of vectors satisfies 0°≤α≤0.5°, the n in the solution of the set is 1i and the corresponding center coordinates c 1i is the correct solution of the normal vector n1 and the center coordinate c1 of the space circle C1. 2i and the corresponding center coordinates c 2i is the correct solution for the normal vector n2 and center coordinate c2 of the space circle C2; Step 6-4, the normal vector n of the plane where the waist-shaped hole is located is (n1+n2) / 2, the centers of the arcs at both ends are c1 and c2 respectively, and the geometric center of the waist-shaped hole is c=(c1+c2) / 2.

5. The method according to claim 4, characterized in that Step 7 includes: Step 7-1, the position information of the waist-shaped hole is represented by the coordinates of the geometric center c, and the posture information of the waist-shaped hole is represented by the yaw angle ψ, pitch angle θ and roll angle φ; in the image, the projection of c2 is always below the projection of c1, and the yaw angle ψ is defined as the vector c2c1 in X c O c Y c The angle between the vertical projection on the plane and the positive direction of the y-axis is in the range of [-π / 2, π / 2]. The pitch angle ψ is positive when the angle is θ, which is defined as the angle between the plane where the waist hole is located and the plane X. c O c Y c The angle range is [-π / 2, π / 2]. The pitch angle θ is positive when the vertical angle is 0.001, and the roll angle φ is defined as the angle between the normal vector n and the vertical plane c1c2c1′c2′ where the straight line c1c2 is located. c1′ and c2′ are the angles of c1 in X direction. c O c Y c The vertical projection on the plane and c2 in X c O c Y c The vertical projection on the plane has a value range of [-π / 2, π / 2]. The roll angle φ is positive when the normal vector is to the right of the plane c1c2c1′c2′. The calculation formulas for the yaw angle ψ, pitch angle θ, and roll angle φ are: in are the center c i In O c X c Projection in direction and O c Y c The projection in the direction, n is the normal vector of the plane where the waist hole is located in the camera coordinate system, n x 、n y 、n z They are the normal vector n in O c X c Projection in direction, O c Y c Projection in direction and O c Z c Projection in direction, is the normal vector of the vertical plane c1c2c1′c2′; Step 7-2, the position information of the waist hole c=[c x c y c z ] T =(c1+c2) / 2, attitude information e=[ψθφ] T .

6. A storage medium, characterized in that A computer program or instruction is stored, and when the computer program or instruction is executed, the method according to any one of claims 1 to 5 is implemented.

Images