Polygonal target visual detection alignment method based on geometric feature similarity

Abstract

The present invention relates to a polygonal target visual detection and alignment method based on geometric feature similarity, including target contour straight line detection, target polygon contour extraction and target center posture alignment process. First, an edge point set is obtained through image preprocessing, and the improved RLD straight line detection algorithm is used to detect the straight lines that may be included, and the straight line model is fitted by the least squares method to obtain an initial straight line set. Secondly, based on the geometric feature similarity method, the straight lines are simplified and spliced ​​to extract the target polygonal contour. Thirdly, the minimum circumscribed rectangle of the target polygonal contour is extracted by the rotating caliper method, the relative posture deviation between the alignment device and the polygonal target is calculated, and the posture feedback controller is designed to eliminate the relative posture deviation, so as to realize the alignment control of the alignment device and the polygonal target.

Description

Technical Field

[0001] The invention relates to the field of visual alignment, and in particular to a polygonal target visual detection alignment method based on geometric feature similarity. Background Art

[0002] Visual alignment technology has important applications in the fields of automation and robotics, but common problems in the prior art include insufficient accuracy and poor adaptability in complex environments. The present invention solves the problems of poor straight line detection accuracy and low contour extraction efficiency in traditional methods by introducing the concept of geometric feature similarity, thereby improving the accuracy and robustness of target positioning.

[0003] Compared with the prior art, the present invention has the following advantages: 1) high-precision target contour detection and extraction to obtain accurate target geometric information; 2) precise target center pose calculation to effectively determine the spatial position and orientation of the target, providing an accurate reference for subsequent alignment operations; 3) effective processing of redundant straight lines and broken straight lines to improve the accuracy and stability of subsequent polygon contour extraction; 4) an improved RLD straight line detection algorithm and a straight line simplification and splicing method based on spatial line feature similarity are adopted to maintain efficient performance and stable results, which is suitable for automated visual inspection tasks in actual industrial scenarios.

[0004] At present, common visual detection alignment methods include feature point matching, template matching and deep learning-based methods. The feature point matching method detects key points in the image and uses descriptors to compare and match target features. The template matching method relies on the positioning and matching process of pre-defined target templates in the image. Although these methods perform well in some cases, they are limited in dealing with targets with complex shapes, deformations or multi-angle perspective changes. For example, the Chinese invention patent with publication number CN117876346A requires multiple perspectives for shooting and the camera rotation angle needs to be controlled. Summary of the invention

[0005] The purpose of the present invention is to propose a polygonal target visual detection alignment method based on geometric feature similarity for the visual alignment function of the robot, aiming to overcome the problems of low alignment accuracy and efficiency existing in the prior art.

[0006] The present invention is achieved through the following technical solutions:

[0007] The present invention provides a polygonal target visual detection and alignment method based on geometric feature similarity, which specifically includes target contour straight line detection, target polygon contour extraction and target posture detection and alignment steps:

[0008] The target contour straight line detection process comprises the following steps:

[0009] Step 1: Target image preprocessing: Perform image preprocessing on the target image acquired by the depth camera, including median filtering, Canny operator edge detection and image expansion, to obtain a set of edge points in the target image;

[0010] Step 2: Target image line detection: For the edge point set after image preprocessing, the improved RLD line detection algorithm is used to detect the lines that may be contained in the target image, and the line interior point set of each line is obtained;

[0011] Step 3: Target image straight line fitting: For each set of points within the straight line, the least square method is used to perform straight line fitting to obtain the initial straight line set, including the slope, intercept parameter, and geometric properties of the starting and end point coordinates of each initial straight line;

[0012] The target polygon outline extraction process comprises the following steps:

[0013] Step 4: Target contour straight line simplification and splicing: To solve the problem of straight line redundancy and breakage in the initial straight line set, a straight line simplification and splicing method based on spatial line feature similarity is adopted. Through the steps of 2D straight line slope similarity classification, 3D straight line non-planarity judgment, 3D straight line collinearity judgment, 3D collinear straight line splicing, and 3D straight line side length screening, redundant straight lines are removed, broken straight lines are spliced, and straight lines with unreasonable side lengths are deleted to obtain a refined straight line set.

[0014] Step 5: Contour line closure and polygon extraction: For the above refined straight line set, a polygon contour extraction method based on geometric feature similarity is used to extract the target polygon contour through multiple steps of straight line end connection judgment, polygon contour closure screening and polygon contour perimeter screening;

[0015] The target posture detection and alignment process comprises the following steps:

[0016] Step 6: Extract the minimum bounding rectangle of the target: For the target polygon outline, use the rotating caliper method to fit the minimum bounding rectangle of the target polygon outline, and use the image center coordinates (x c ,y c ) is the target center;

[0017] Step 7: Calculation of polygonal target posture deviation: For the minimum circumscribed rectangle of the polygonal target, the polygonal target posture deviation calculation method is adopted. Through the steps of polygonal target space plane creation, target center space position deviation solution, polygonal target posture angle deviation solution, the relative posture deviation between the alignment device and the polygonal target is calculated;

[0018] Step 8: Polygonal target pose alignment control: Based on the relative pose deviation between the alignment device and the polygonal target, a pose feedback controller is designed to generate the alignment device speed control value that eliminates the relative pose deviation.

[0019] Preferably, the improved RLD straight line detection algorithm in step 2 specifically includes the following steps:

[0020] Step 2.1: Parameter initialization: Assume that the set of edge points in the target image is O = {p1, p2, p3, ..., p n}, where p i ={x i ,y i}; Set the angle threshold to θ s , the straight line point threshold is T s , the distance threshold for determining the inner point of the straight line is T d1 , the failure threshold is T time ; Initialize failure counter f time , initialize the line number a to 1;

[0021] Step 2.2: Random sampling of edge points: Randomly select three points p from the edge point set O i 、p j 、p k As seed points, the seed points are taken from the edge point set O;

[0022] Step 2.3: Calculate the angle of the straight line: Every two points in the above seed points form a straight line, and point p j 、p k The length of the straight line segment is L i , click p i 、p k The length of the straight line segment is L j , click p i 、p j The length of the straight line segment is L k , straight line L i , L j , L k The corresponding angle θ i ,θ j ,θ k for

[0023]

[0024]

[0025] Step 2.4: Angle threshold judgment: If the angle θ i ,θ j ,θ k There is an angle less than the angle threshold θs , then point p i 、p j 、p k A straight line can be formed, denoted as straight line l a , go to step 2.5, otherwise go to step 2.7;

[0026] Step 2.5: Determine the inner point of the line: Initialize the number of line points to S, traverse all edge points in the edge point set O except the seed point, and determine the edge point to the line l a Is the distance less than the inner point distance threshold T? d1 : If the distance is less than the threshold, add this edge point to the line l a The interior point set O a , add 1 to S; after the traversal is completed, go to step 2.6;

[0027] Step 2.6: Store points in the line: Determine whether the number of line points S is greater than the line point threshold T s :If it is established, record the constructed straight line l a The interior point set O a , increase the line number a by 1 and return to step 2.2; otherwise, go to step 2.7;

[0028] Step 2.7: Line detection completion judgment: set the failure counter f time Add 1 and judge f time Is it greater than T time :If it is established, the improved RLD straight line detection process is ended to obtain a plurality of straight lines l a The interior point set O a Otherwise, return to step 2.2.

[0029] Preferably, the straight line simplification and splicing method based on spatial line feature similarity in step 4 specifically comprises the following steps:

[0030] Step 4.1: Parameter initialization: Set the number of polygon edges to M and the slope similarity threshold to T S , the direction vector similarity threshold is T V , the distance threshold for determining collinearity of straight lines is T d2 , the length of each side of the polygon outline is c k , k = 1, 2 ... M, the edge length threshold is T ck ; Initialize the main line number i of the line classification to 1, and the line number j to be classified to 1;

[0031] Step 4.2: Main line selection: Take a line from the initial line set as the main line L for line classification i , and the remaining straight lines are taken as the set of straight lines to be classified;

[0032] Step 4.3: Select the line to be classified: Select an unselected line from the set of lines to be classified as the line to be classified L j ;

[0033] Step 4.4: 2D line slope similarity classification: Assume that the main line L is the line classification i The slope is denoted as K Li , the line to be classified L j The slope is denoted as K Lj , then L i With L j The slope similarity S ij for

[0034] S ij =K Li / K Lj (4)

[0035] If the slope similarity S ij Satisfies the following formula

[0036] |1-S ij | <T S (5)

[0037] Then the slopes of the two straight lines are judged to be similar, and the straight line to be classified L j Add the main straight line L for straight line classification i The slope similarity set n i ; Slope similarity S ij After judgment, the serial number j of the line to be classified is increased by 1;

[0038] Step 4.5: Determine whether the traversal of the lines to be classified is complete: Determine whether the traversal of all the lines in the set of lines to be classified is complete: If the traversal is complete, classify the line as the main line L i Add slope similarity set n i , and the slope similarity set n i All the lines in are deleted from the initial line set, the sequence number j of the line to be classified is initialized to 1, and the process goes to step 4.6; if the traversal is not completed, the process goes back to step 4.3;

[0039] Step 4.6: Main line traversal completion judgment: Determine whether the initial line set is empty: If it is empty, combine multiple slope similarity sets n i Construct slope similarity set N P , initialize the main line number i of the straight line classification to 1, and go to step 4.7; otherwise, add 1 to the main line number i of the straight line classification, and return to step 4.2;

[0040] Step 4.7: Slope similarity set selection: In the slope similarity set group N P Traverse and select the main straight line Li The slope similarity set n i ;

[0041] Step 4.8: 2D-3D Line Transformation: In the Slope Similarity Set n i Select two straight lines L in the traversal a With L b ; According to the intrinsic parameter matrix of the depth camera and the depth value of the detected pixel point, the pixel coordinates of the starting point and the end point of the straight line are transformed from the image coordinate system to the three-dimensional coordinates in the camera coordinate system. The pixel coordinates of the detection point are recorded as The depth camera intrinsic parameter matrix is ​​M col , the pixel depth value is Z C , then the three-dimensional coordinates of the pixel point in the camera coordinate system are for

[0042]

[0043] Step 4.9: Determine whether two lines are in different planes based on the direction vector of the three-dimensional line. Assume that X a , Y a , Z a The straight line L a The three-dimensional coordinate difference between the starting point coordinate and the end point coordinate, X b , Y b , Z b The straight line L b The three-dimensional coordinate difference between the starting point coordinates and the end point coordinates, and the formula for judging the line non-coplanarity are as follows:

[0044]

[0045] If equations (7) to (9) are all valid, then the straight line L a and L b If the two lines are coplanar, go to step 4.10; otherwise, the two lines are skew lines, return to step 4.8;

[0046] Step 4.10: Determine whether two straight lines are collinear based on the distance between them. a With L b Select a series of key points at a certain interval and calculate L a The starting point, end point and key point of L b The distance between the starting point, end point and key point; if there is a distance less than the straight line collinearity determination distance threshold T d2 , then the above two straight lines are collinear, and go to step 4.11; otherwise, the above two straight lines are not collinear, and return to step 4.8;

[0047] Step 4.11: Three-dimensional collinear line stitching: Connect the above two collinear lines L a With L b Splice; calculate L a The starting point, end point and L b The distance between any two points in the starting point and the end point of the splicing line L is the two points with the farthest distance between them. c The starting point and end point of the splicing result straight line L c Keep and update its geometric properties, and convert the original line L a With L b From the slope similarity set n i Delete from;

[0048] Step 4.12: Line combination traversal is completed: If the slope is similar to the set n i If all the straight line combinations in are traversed, go to step 4.13; otherwise, return to step 4.8;

[0049] Step 4.13: Slope similarity set traversal completed judgment: If the slope similarity set group N P All slope similarity sets n in i After the traversal is completed, the simplified slope similarity set N is obtained R , go to step 4.14; otherwise, add 1 to the main line number i of the line classification, and return to step 4.7;

[0050] Step 4.14: 3D straight line side length screening: traverse and simplify the slope similarity set group N R For each straight line in the 3D coordinates of the starting and ending points of the straight line, the straight line length d is calculated. If the straight line length d is equal to the polygon outline side length c k Satisfy the following formula:

[0051] |dc k | <T ck ,k=1,2…M (10)

[0052] If the line is not in the correct order, the line is retained; otherwise, the line is deleted;

[0053] Step 4.15: Line traversal completion judgment: judge the simplified slope similarity set group N R Are all the straight lines in the traversal completed? If completed, then all the slope similarity sets n are sorted from small to large slopes. i Construct a precise set of simple lines N L , end the straight line simplification and splicing process; otherwise, return to step 4.14.

[0054] Preferably, the polygon outline extraction method based on geometric feature similarity in step 5 specifically includes the following steps:

[0055] Step 5.1: Parameter initialization: Set the number of target polygon edges to M and the line end connection determination distance threshold to T d3 , set the target polygon perimeter and perimeter threshold to A and T respectively A ;

[0056] Step 5.2: Method feasibility judgment: Determine whether the number of polygon edges M is greater than the set of precise simple lines N L Medium slope similarity set n i Number N: If true, the polygon outline cannot be extracted, and return to step 1 to collect the target image and reprocess it; otherwise, go to step 5.3;

[0057] Step 5.3: Slope similarity set group selection: for the precise simple line set N L All slope similarity sets n in i , i = 1, 2 ... N, according to M slope similarity sets n i The rule of non-repetitive union is to traverse and select M sets of similar slopes n i , denoted as β1,β2…β M ;

[0058] Step 5.4: Contour line combination selection: According to the rule that one straight line is selected from each slope similarity set and M straight line combinations are not repeated, select M slope similarity sets β1, β2…β M In the traversal, select M straight lines, denoted as l1, l2…l M , initialize the line end connection main straight line number j to 1;

[0059] Step 5.5: Line end connection threshold judgment: Calculate the line end connection main straight line l in sequence j The starting point and end point of the other straight lines l j+1 ,l j+2 …l M The distance between the starting point and the end point; if the distance between any two points is less than the line end connection determination distance threshold T d3 If both points are not connected to other points, connect the two points; after completing the distance calculation and threshold judgment between all two points, add 1 to the main straight line number j of the line end connection;

[0060] Step 5.6: Determine whether the line end connection traversal is completed: Determine whether the line end connection main straight line number j is equal to the number of polygon edges M. If the condition is met, go to step 5.7; otherwise, return to step 5.5;

[0061] Step 5.7: Polygonal contour closure screening: determine l1, l2, l3...l MWhether the starting point and the end point of can be connected to other points: If the condition is met, the M straight lines are combined into a polygon, and the polygon model including the straight line set and the corner point set is stored in a clockwise direction, and the process goes to step 5.8; otherwise, the process goes to step 5.9;

[0062] Step 5.8: Polygon perimeter threshold judgment: Use the three-dimensional coordinates of the starting point and end point of each edge of the polygon to calculate the polygon perimeter B, and compare it with the target polygon perimeter A. If the following formula is satisfied

[0063] |BA| <T A (11) Then the above polygon is stored in the potential polygon outline set P; otherwise, the polygon model is deleted;

[0064] Step 5.9: Line combination traversal is completed: judge in the slope similarity set β1, β2…β M Is the combined traversal of the contour lines in completed? If completed, proceed to step 5.10; otherwise, return to step 5.4;

[0065] Step 5.10: Slope similarity set group traversal completion judgment: judge in the refined simple line set group N L There are M sets of similar slopes β1, β2…β M Is the traversal completed: If completed, go to step 5.11; otherwise, return to step 5.3;

[0066] Step 5.11: Target polygon outline selection: In the potential polygon outline set P, calculate the perimeter of each polygon in turn, select the polygon outline closest to the target polygon perimeter A as the target polygon outline, and end the polygon outline extraction process.

[0067] Preferably, the rotating caliper method in step 6 specifically comprises the following steps:

[0068] Step 6.1: Parameter initialization: Assume that the line set X of the target polygonal model is {L1, L2, L3, ..., L M}; the corner point set P is {P1,P2,P3,……,P M}; Initialize the main parallel line number m to 1;

[0069] Step 6.2: Main parallel line selection: Select each line in the target polygon line set X in order as the main parallel line L m ;

[0070] Step 6.3: Secondary parallel line selection: Calculate the distance from all corner points in the target polygon corner point set P to the primary parallel line L m The projection distance of the corner point with the largest projection distance is selected as P M , past point PM Main parallel line L m The parallel line of is called secondary parallel line;

[0071] Step 6.4: Construct a circumscribed rectangle: Draw a straight line perpendicular to the main parallel line through all corner points in the target polygon corner point set P, and record it as the main parallel line perpendicular line set; calculate the distance between any two perpendicular lines in the main parallel line perpendicular line set, select the two perpendicular lines with the largest distance as the sides of the circumscribed rectangle, and construct a circumscribed rectangle with the main parallel line and the secondary parallel line, and store it in the target contour circumscribed rectangle set H R ;

[0072] Step 6.5: Determine if the rotation and construction of the rectangle is completed: add 1 to the main parallel line number m, and determine whether the main parallel line number m is greater than the number of sides of the target polygon M: if the condition is met, proceed to step 6.6, otherwise proceed to step 6.2;

[0073] Step 6.6: Minimum bounding rectangle selection: For the target contour bounding rectangle set H R For each bounding rectangle in the image, calculate the pixel area of ​​the rectangular image, select the bounding rectangle with the smallest pixel area as the minimum bounding rectangle of the target contour, and select the image center coordinates (x c ,y c ) is the target center.

[0074] Preferably, the target polygonal cross-section posture deviation calculation in step 7 specifically includes the following steps:

[0075] Step 7.1: Create polygonal target space plane: For the target polygon corner point set P, calculate the three-dimensional coordinates of each corner point in the camera coordinate system, and record the pixel coordinates of the corner point as The depth camera intrinsic parameter matrix is ​​M col , the corner depth value is Z i , then the three-dimensional coordinates of the corner point in the camera coordinate system are for

[0076]

[0077] According to the three-dimensional coordinates of all corner points in the target polygon corner point set P, the spatial plane model of the target polygon outline in the camera coordinate system is calculated using the least squares method.

[0078] AX+BY+CZ+D=0 (13)

[0079] Among them, (X, Y, Z) is the three-dimensional coordinates of the corner point in the camera coordinate system, A, B and C are the model parameters of the space plane;

[0080] Step 7.2: Calculate the target center spatial position deviation; the pixel coordinates of the target center are known to be (x c,y c ), the three-dimensional coordinates of the target center in the camera coordinate system are marked as (X c ,Y c ,Z c ); Since the camera coordinate system is fixed to the docking device coordinate system, the X-axis position deviation between the docking device and the target center is e dX , Y-axis position deviation e dY and Z-axis position deviation e dZ They are

[0081]

[0082] Where f x 、f y , u0, v0 are the internal parameters of the depth camera;

[0083] Step 7.3: Calculation of polygonal target attitude angle deviation: Assuming the camera plane normal vector is (0,0,1), for the polygonal target space plane normal vector (A,B,C), set the normal vector Z-axis direction greater than 0, and the attitude angle deviation is positive in the counterclockwise direction around the coordinate axis. Then the X-axis attitude angle deviation between the docking device and the polygonal target is e θX 、Y-axis attitude angle deviation e θY They are

[0084]

[0085] Beneficial effects of the present invention:

[0086] The polygonal target visual detection and alignment method based on geometric feature similarity improves the stability and accuracy of the visual system by optimizing the target contour straight line detection, target polygon contour extraction and polygonal target pose alignment processes. It has broad market application prospects and technical promotion value. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] The present invention will be further described in detail below with reference to the accompanying drawings:

[0088] Figure 1 It is a system flow chart of the polygonal target visual detection and alignment method based on geometric feature similarity of the present invention;

[0089] Figure 2 It is a schematic diagram of the straight line splicing process of the present invention;

[0090] Figure 3 It is a schematic diagram of solving the target center spatial position deviation of the present invention;

[0091] Figure 4 It is a schematic diagram of solving the polygonal target attitude angle deviation of the present invention. DETAILED DESCRIPTION

[0092] The present invention is further described in detail below in conjunction with the embodiments and drawings.

[0093] Reference Figure 1 As shown, the polygonal target visual detection and alignment method based on geometric feature similarity in the present invention mainly includes three processes, namely, target contour straight line detection process, target polygon contour extraction process and target posture detection and alignment process. The target contour straight line detection process includes target image preprocessing, target image straight line detection and target image straight line fitting; the target polygon contour extraction process includes target contour straight line simplification and splicing, contour straight line closure and polygon extraction; the target posture detection and alignment process includes target minimum circumscribed rectangle extraction, polygonal target posture deviation calculation and polygonal target posture alignment control.

[0094] The present invention provides a polygonal target visual detection and alignment method based on geometric feature similarity, which specifically includes:

[0095] The target contour straight line detection process comprises the following steps:

[0096] Step 1: Target image preprocessing: Perform image preprocessing on the target image acquired by the depth camera, including median filtering, Canny operator edge detection and image expansion, to obtain a set of edge points in the target image;

[0097] Step 2: Target image line detection: For the edge point set after image preprocessing, the improved RLD line detection algorithm is used to detect the lines that may be contained in the target image, and the line interior point set of each line is obtained;

[0098] Step 3: Target image straight line fitting: For each set of points within the straight line, the least square method is used to perform straight line fitting to obtain the initial straight line set, including the slope, intercept parameter, and geometric properties of the starting and end point coordinates of each initial straight line;

[0099] The target polygon outline extraction process comprises the following steps:

[0100] Step 4: Target contour straight line simplification and splicing: To solve the problem of straight line redundancy and breakage in the initial straight line set, a straight line simplification and splicing method based on spatial line feature similarity is adopted. Through the steps of 2D straight line slope similarity classification, 3D straight line non-planarity judgment, 3D straight line collinearity judgment, 3D collinear straight line splicing, and 3D straight line side length screening, redundant straight lines are removed, broken straight lines are spliced, and straight lines with unreasonable side lengths are deleted to obtain a refined straight line set.

[0101] Step 5: Contour line closure and polygon extraction: For the above refined straight line set, a polygon contour extraction method based on geometric feature similarity is used to extract the target polygon contour through multiple steps of straight line end connection judgment, polygon contour closure screening and polygon contour perimeter screening;

[0102] The target posture detection and alignment process comprises the following steps:

[0103] Step 6: Extract the minimum bounding rectangle of the target: For the target polygon outline, use the rotating caliper method to fit the minimum bounding rectangle of the target polygon outline, and use the image center coordinates (x c ,y c ) is the target center;

[0104] Step 7: Calculation of polygonal target posture deviation: For the minimum circumscribed rectangle of the polygonal target, the polygonal target posture deviation calculation method is adopted. Through the steps of polygonal target space plane creation, target center space position deviation solution, polygonal target posture angle deviation solution, the relative posture deviation between the alignment device and the polygonal target is calculated;

[0105] Step 8: Polygonal target pose alignment control: Based on the relative pose deviation between the alignment device and the polygonal target, a pose feedback controller is designed to generate the alignment device speed control value that eliminates the relative pose deviation.

[0106] First, the target contour straight line detection step needs to be completed. The specific steps are as follows:

[0107] Image preprocessing is performed on the target image obtained by the depth camera, including using median filtering to reduce the noise of the image obtained by the depth camera, using the Canny edge detection operator to extract edge information in the image and performing image dilation processing on the result after edge detection, completing broken edge lines, and removing erroneously extracted interfering background points. Multiple image preprocessing steps are performed to obtain a set of edge points in the target image.

[0108] For the edge point set after image preprocessing, the improved RLD line detection algorithm is used to detect the straight lines that may be contained in the image, and the line interior point set is obtained for each straight line. The basic principle is to randomly sample in the edge point set, take three points as a group of seed points, judge whether these three points can form a straight line according to the angle threshold, and verify the rationality of the straight line through the remaining points, which can significantly improve the operation efficiency of the algorithm; the improved RLD line detection algorithm includes the following steps:

[0109] 1) Parameter initialization: Assume that the set of edge points in the target image is O = {p1, p2, p3, ..., p n}, where p i ={x i,y i}; Set the angle threshold to θ s , the straight line point threshold is T s , the distance threshold for determining the inner point of the straight line is T d1 , the failure threshold is T time ; Initialize failure counter f time , initialize the line number a to 1;

[0110] 2) Random sampling of edge points: Randomly select three points p from the edge point set O i 、p j 、p k As seed points, the seed points are taken out from the edge point set o;

[0111] 3) Line angle calculation: Every two points in the above seed points form a straight line, point p j 、p k The length of the straight line segment is L i , click p i 、p k The length of the straight line segment is L j , click p i 、p j The length of the straight line segment is L k , straight line L i , L j , L k The corresponding angle θ i ,θ j ,θ k for

[0112]

[0113]

[0114] 4) Angle threshold judgment: If the angle θ i ,θ j ,θ k There is an angle less than the angle threshold θ s , then point p i 、p j 、p k A straight line can be formed, denoted as straight line l a , go to step 5), otherwise go to step 7);

[0115] 5) Determine the inner point of the line: Initialize the number of line points to S, traverse all edge points in the edge point set O except the seed point, and determine the edge point to the line l a Is the distance less than the inner point distance threshold T? d1 : If the distance is less than the threshold, add this edge point to the line l aThe interior point set O a , add 1 to S; after the traversal is completed, go to step 6);

[0116] 6) Line internal point storage: Determine whether the line point number S is greater than the line point threshold T s :If it is established, record the constructed straight line l a The interior point set O a , increase the line number a by 1 and return to step 2); otherwise, go to step 7);

[0117] 7) Line detection completion judgment: set the failure counter f time Add 1 and judge f time Is it greater than T time :If it is established, the improved RLD straight line detection process is ended to obtain a plurality of straight lines l a The interior point set O a Otherwise, return to step 2).

[0118] For each set of inner points of the straight line obtained by image straight line detection, the least squares method is used to perform straight line fitting to obtain the initial straight line set, including the slope, intercept parameters and geometric properties of the starting and end points of each initial straight line. The slope k and intercept b are:

[0119]

[0120] Among them, (x i ,y i ) is a point inside the straight line.

[0121] After completing the target contour straight line detection step, the target polygon contour extraction step is performed. The specific steps are as follows:

[0122] In order to solve the problem of line redundancy and breakage in the initial line set, a line simplification and splicing method based on spatial line feature similarity is adopted. Through multiple steps of 2D line slope similarity classification, 3D line non-plane judgment, 3D line collinearity judgment, 3D collinear line splicing, and 3D line side length screening, redundant lines are removed, broken lines are spliced, and lines with unreasonable side lengths are deleted to obtain a refined line set. Figure 2 The straight line simplification and splicing method based on spatial line feature similarity comprises the following steps:

[0123] 1) Parameter initialization: Set the number of polygon edges to M and the slope similarity threshold to T s , the direction vector similarity threshold is T V , the distance threshold for determining collinearity of straight lines is T d2 , the length of each side of the polygon outline is c k , k = 1, 2 ... M, the edge length threshold is Tck ; Initialize the main line number i of the line classification to 1, and the line number j to be classified to 1;

[0124] 2) Main line selection: Take a line from the initial line set as the main line L for line classification i , and the remaining straight lines are taken as the set of straight lines to be classified;

[0125] 3) Selection of the line to be classified: Select an unselected line from the set of lines to be classified as the line to be classified L j ;

[0126] 4) 2D straight line slope similarity classification: Assume that the main straight line L is i The slope is denoted as K Li , the line to be classified L j The slope is denoted as K lj , then L i With L j The slope similarity S ij for

[0127] S ij =K Li / K Lj (5)

[0128] If the slope similarity S ij Satisfies the following formula

[0129] |1-S ij | <T S (6)

[0130] Then the slopes of the two straight lines are judged to be similar, and the straight line to be classified L j Add the main straight line L for straight line classification i The slope similarity set n i ; Slope similarity S ij After judgment, the serial number j of the line to be classified is increased by 1;

[0131] 5) Determine whether the traversal of the lines to be classified is complete: Determine whether the traversal of all the lines in the set of lines to be classified is complete: If the traversal is complete, classify the line as the main line L i Add slope similarity set n i , and the slope similarity set n i All the lines in are deleted from the initial line set, the sequence number j of the line to be classified is initialized to 1, and the process goes to step 6); if the traversal is not completed, the process goes back to step 3);

[0132] 6) Main line traversal completion judgment: judge whether the initial line set is empty: if it is empty, combine multiple slope similarity sets n i Construct slope similarity set N P, initialize the main line number i of the straight line classification to 1, and go to step 7) otherwise add 1 to the main line number i of the straight line classification, and return to step 2);

[0133] 7) Slope similarity set selection: In the slope similarity set group N P Traverse and select the main straight line L i The slope similarity set n i ;

[0134] 8) 2D-3D linear transformation: In the slope similarity set n i Select two straight lines L in the traversal a With L b ; According to the intrinsic parameter matrix of the depth camera and the depth value of the detected pixel point, the pixel coordinates of the starting point and the end point of the straight line are transformed from the image coordinate system to the three-dimensional coordinates in the camera coordinate system. The pixel coordinates of the detection point are recorded as The depth camera intrinsic parameter matrix is ​​M col , the pixel depth value is Z C , then the three-dimensional coordinates of the pixel point in the camera coordinate system are for

[0135]

[0136] 9) 3D straight line non-coplanarity judgment: judge whether two straight lines are non-coplanar based on the direction vector of the 3D straight line. Assume that X a , Y a , Z a The straight line L a The three-dimensional coordinate difference between the starting point coordinate and the end point coordinate, X b , Y b , Z b The straight line L b The three-dimensional coordinate difference between the starting point coordinates and the end point coordinates, and the formula for judging the line non-coplanarity are as follows:

[0137]

[0138]

[0139] If equations (8) to (10) are all valid, then the straight line L a and L b If the two straight lines are coplanar, go to step 10); otherwise, the two straight lines are skew lines, return to step 8);

[0140] 10) 3D straight line collinearity judgment: judge whether two straight lines are collinear based on the distance between them. a With L b Select a series of key points at a certain interval and calculate L a The starting point, end point and key point of Lb The distance between the starting point, end point and key point; if there is a distance less than the straight line collinearity determination distance threshold T d2 , then the above two straight lines are collinear, and the process goes to step 11); otherwise, the above two straight lines are not collinear, and the process goes back to step 8);

[0141] 11) Three-dimensional collinear line splicing: Connect the two collinear lines L a With L b Splice; calculate L a The starting point, end point and L b The distance between any two points in the starting point and the end point of the splicing line L is the two points with the farthest distance between them. c The starting point and end point of the splicing result straight line L c Keep and update its geometric properties, and convert the original line L a With L b From the slope similarity set n i Delete from;

[0142] 12) Completion of line combination traversal: If the slope is similar to the set n i If all the straight line combinations in are traversed, go to step 13); otherwise, return to step 8);

[0143] 13) Slope similarity set traversal completion judgment: If the slope similarity set group N P All slope similarity sets n in i After the traversal is completed, the simplified slope similarity set N is obtained R , go to step 14); otherwise, add 1 to the main line number i of the line classification, and return to step 7);

[0144] 14) 3D straight line side length screening: traverse and simplify the slope similarity set N R For each straight line in the 3D coordinates of the starting and ending points of the straight line, the straight line length d is calculated. If the straight line length d is equal to the polygon outline side length c k Satisfy the following formula:

[0145] |dc k | <T ck ,k=1,2…M (11)

[0146] If the line is not in the correct order, the line is retained; otherwise, the line is deleted;

[0147] 15) Line traversal completion judgment: judge the simplified slope similarity set N R Are all the straight lines in the traversal completed? If completed, then all the slope similarity sets n are sorted from small to large slopes. i Construct a precise set of simple lines N L, end the straight line simplification and splicing process; otherwise return to step 14).

[0148] For the precise set of simple lines, a polygon outline extraction method based on geometric feature similarity is adopted to extract the target polygon outline through multiple steps of straight line end connection judgment, polygon outline closure screening and polygon outline perimeter screening; the polygon outline extraction method based on geometric feature similarity specifically includes the following steps:

[0149] 1) Parameter initialization: Set the number of target polygon edges to M and the line end connection determination distance threshold to T d3 , set the target polygon perimeter and perimeter threshold to A and T respectively A ;

[0150] 2) Feasibility judgment of the method: judge whether the number of polygon edges M is greater than the precise simple line set M L Medium slope similarity set n i Number N: If true, the polygon outline cannot be extracted, and return to step 1 to collect the target image and reprocess it; otherwise, go to step 3);

[0151] 3) Slope similarity set group selection: for the precise simple line set N L All slope similarity sets n in i , i = 1, 2 ... N, according to M slope similarity sets n i The rule of non-repetitive union is to traverse and select M sets of similar slopes n i , denoted as β1,β2…β M ;

[0152] 4) Contour line combination selection: According to the rule that one straight line is selected from each slope similarity set and M straight line combinations are not repeated, select the contour line combination from M slope similarity sets β1, β2…β M In the traversal, select M straight lines, denoted as l1, l2…l M , initialize the line end connection main straight line number j to 1;

[0153] 5) Line end connection threshold judgment: Calculate the line end connection main straight line l in sequence j The starting point and end point of the other straight lines l j+1 ,l j+2 …l M The distance between the starting point and the end point; if the distance between any two points is less than the line end connection determination distance threshold T d3 If both points are not connected to other points, connect the two points; after completing the distance calculation and threshold judgment between all two points, add 1 to the main straight line number j of the line end connection;

[0154] 6) Line end connection traversal completion judgment: judge whether the line end connection main straight line number j is equal to the number of polygon edges M. If the condition is met, go to step 7); otherwise, return to step 5);

[0155] 7) Polygonal contour closure screening: determine l1, l2, l3...l M Whether the starting point and the end point of can be connected with other points: If the condition is met, the M straight lines are combined into a polygon, and the polygon model including the straight line set and the corner point set is stored in a clockwise direction, and the process goes to step 8); otherwise, the process goes to step 9);

[0156] 8) Polygon perimeter threshold judgment: Use the three-dimensional coordinates of the starting and ending points of each edge of the polygon to calculate the polygon perimeter B, and compare it with the target polygon perimeter A. If the following formula is satisfied

[0157] |BA| <T A (12) Then the above polygon is stored in the potential polygon outline set P; otherwise, the polygon model is deleted;

[0158] 9) Completion judgment of straight line combination traversal: judge in the slope similarity set β1, β2…β M Is the combined traversal of the contour lines in completed? If completed, proceed to step 10); otherwise, return to step 4);

[0159] 10) Completion of traversal of slope similarity set group: Determine whether the slope similarity set group N is L There are M sets of similar slopes β1, β2…β M Is the traversal completed: If completed, go to step 11); otherwise, return to step 3);

[0160] 11) Target polygon outline selection: In the potential polygon outline set P, calculate the perimeter of each polygon in turn, select the polygon outline closest to the target polygon perimeter A as the target polygon outline, and end the polygon outline extraction process.

[0161] Next, complete the target posture detection and alignment. The specific steps are as follows:

[0162] For the target polygonal contour, the minimum bounding rectangle of the target polygonal contour is fitted using the rotating caliper method, and the image center coordinates (x c ,y c ) is the target center; the rotating caliper method specifically comprises the following steps:

[0163] 1) Parameter initialization: Assume that the line set X of the target polygon model is {L1, L2, L3, ..., L M}; the corner point set P is {P1,P2,P3,……,PM}; Initialize the main parallel line number m to 1;

[0164] 2) Main parallel line selection: select each straight line in the target polygon straight line set X in order as the main parallel line L m ;

[0165] 3) Secondary parallel line selection: Calculate the distance from all corner points in the target polygon corner point set P to the main parallel line L m The projection distance of the corner point with the largest projection distance is selected as P M , past point P M Main parallel line L m The parallel line of is called secondary parallel line;

[0166] 4) Construction of circumscribed rectangle: Draw a straight line perpendicular to the main parallel line through all corner points in the target polygon corner point set P, and record it as the main parallel line perpendicular line set; calculate the distance between any two perpendicular lines in the main parallel line perpendicular line set, select the two perpendicular lines with the largest distance as the sides of the circumscribed rectangle, and construct a circumscribed rectangle with the main parallel line and the secondary parallel line, and store it in the target contour circumscribed rectangle set H R ;

[0167] 5) Determine whether the rotation and construction of the rectangle is completed: add 1 to the main parallel line number m, and determine whether the main parallel line number m is greater than the number of sides of the target polygon M: if the condition is met, proceed to step 6), otherwise proceed to step 2);

[0168] 6) Minimum bounding rectangle selection: For the target contour bounding rectangle set H R For each bounding rectangle in the image, calculate the pixel area of ​​the rectangular image, select the bounding rectangle with the smallest pixel area as the minimum bounding rectangle of the target contour, and select the image center coordinates (x c ,y c ) is the target center.

[0169] For the minimum circumscribed rectangle of the polygonal target, a polygonal target posture deviation calculation method is adopted, and the relative posture deviation between the alignment device and the polygonal target is calculated through multiple steps of polygonal target space plane creation, target center space position deviation solution, and polygonal target posture angle deviation solution; the polygonal target posture deviation calculation method specifically includes the following steps:

[0170] 1) Creation of polygonal target space plane: For the target polygon corner point set P, calculate the three-dimensional coordinates of each corner point in the camera coordinate system, and record the pixel coordinates of the corner point as The depth camera intrinsic parameter matrix is ​​M col , the corner depth value is Z i , then the three-dimensional coordinates of the corner point in the camera coordinate system are for

[0171]

[0172] According to the three-dimensional coordinates of all corner points in the target polygon corner point set P, the spatial plane model of the target polygon outline in the camera coordinate system is calculated using the least squares method.

[0173] AX+BY+CZ+D=0 (14)

[0174] Among them, (X, Y, Z) is the three-dimensional coordinates of the corner point in the camera coordinate system, A, B and C are the model parameters of the space plane;

[0175] 2) Calculation of target center spatial position deviation; Figure 3 As shown, the pixel coordinates of the target center are known to be (x c ,y c ), the three-dimensional coordinates of the target center in the camera coordinate system are marked as (X c ,Y c ,Z c ); Since the camera coordinate system is fixed to the docking device coordinate system, the position deviation between the docking device and the target center is the three-dimensional position of the target center in the camera coordinate system, then the X-axis position deviation between the docking device and the target center is e dX , Y-axis position deviation e dY , Z-axis position deviation e dZ They are

[0176]

[0177]

[0178] Where f x 、f y , u0, v0 are the internal parameters of the depth camera;

[0179] 3) Calculation of polygonal target attitude angle deviation: Assume that the camera plane normal vector is (0,0,1). For the polygonal target space plane normal vector (A,B,C), set the normal vector Z-axis direction greater than 0. Since the camera coordinate system is fixed to the docking device coordinate system, the attitude angle deviation between the camera plane and the polygonal space plane is the attitude angle deviation between the docking device and the polygonal target. Set the attitude angle deviation to be positive in the counterclockwise direction around the coordinate axis. Then the X-axis attitude angle deviation between the docking device and the polygonal target is e θX , Y-axis attitude angle deviation e θY They are

[0180]

[0181] Take the Y-axis attitude angle deviation solution as an example, refer to Figure 4, project the normal vector of the polygonal target space plane onto the XOZ plane, the projection vector is (A, 0, C), and the angle between the projection vector and the camera plane normal vector is the Y-axis attitude angle deviation e θY .

[0182] According to the relative posture deviation between the alignment device and the polygonal target, a posture feedback controller is designed to generate the alignment device speed control value v to eliminate the relative posture deviation. x ,ω Y , and then use the inverse kinematics model to realize the speed feedback control of the alignment device. The control quantity is as follows:

[0183]

[0184] The above describes in detail the specific implementation cases of the present invention. There are many specific application paths of the present invention. The above is only a preferred implementation mode of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principle of the present invention. These improvements should also be regarded as the protection scope of the present invention.

Claims

1. A polygonal target visual detection and alignment method based on geometric feature similarity, characterized in that It includes the target contour straight line detection, target polygon contour extraction and target posture detection alignment process; The target contour straight line detection process comprises the following steps: Step 1: Target image preprocessing: Perform image preprocessing on the target image acquired by the depth camera, including median filtering, Canny operator edge detection and image expansion, to obtain a set of edge points in the target image; Step 2: Target image line detection: For the edge point set after image preprocessing, the improved RLD line detection algorithm is used to detect the lines that may be contained in the target image, and the line interior point set of each line is obtained; Step 3: Target image straight line fitting: For each set of points within the straight line, the least square method is used to perform straight line fitting to obtain the initial straight line set, including the slope, intercept parameter, and geometric properties of the starting and end point coordinates of each initial straight line; The target polygon outline extraction process comprises the following steps: Step 4: Target contour straight line simplification and splicing: To solve the problem of straight line redundancy and breakage in the initial straight line set, a straight line simplification and splicing method based on spatial line feature similarity is adopted. Through the steps of 2D straight line slope similarity classification, 3D straight line non-planarity judgment, 3D straight line collinearity judgment, 3D collinear straight line splicing, and 3D straight line side length screening, redundant straight lines are removed, broken straight lines are spliced, and straight lines with unreasonable side lengths are deleted to obtain a refined straight line set. Step 5: Contour line closure and polygon extraction: For the above refined straight line set, a polygon contour extraction method based on geometric feature similarity is used to extract the target polygon contour through multiple steps of straight line end connection judgment, polygon contour closure screening and polygon contour perimeter screening; The target posture detection and alignment process comprises the following steps: Step 6: Extract the minimum bounding rectangle of the target: For the target polygon outline, use the rotating caliper method to fit the minimum bounding rectangle of the target polygon outline, and use the image center coordinates (x c ,y c ) is the target centre; Step 7: Calculation of polygonal target posture deviation: For the minimum circumscribed rectangle of the polygonal target, the polygonal target posture deviation calculation method is adopted. Through the steps of polygonal target space plane creation, target center space position deviation solution, polygonal target posture angle deviation solution, the relative posture deviation between the alignment device and the polygonal target is calculated; Step 8: Polygonal target pose alignment control: Based on the relative pose deviation between the alignment device and the polygonal target, a pose feedback controller is designed to generate the alignment device speed control value that eliminates the relative pose deviation.

2. The method for visual detection and alignment of polygonal targets based on geometric feature similarity according to claim 1, characterized in that The improved RLD straight line detection algorithm specifically includes the following steps: Step 2.1: Parameter initialization: Assume that the set of edge points in the target image is O = {p1, p2, p3, ..., p n }, where p i ={x i ,y i }; Set the angle threshold to θ s , the straight line point threshold is T s , the distance threshold for determining the inner point of the straight line is T d1 , the failure threshold is T time ; Initialize failure counter f time , initialize the line number a to 1; Step 2.2: Random sampling of edge points: Randomly select three points p from the edge point set O i 、p j 、p k As seed points, the seed points are taken from the edge point set O; Step 2.3: Calculate the angle of the straight line: Every two points in the above seed points form a straight line, and point p j 、p k The length of the straight line segment is denoted by L i , click p i 、p k The length of the straight line segment is denoted by L j , click p i 、p j The length of the straight line segment is denoted by L k , straight line L i , L j , L k The corresponding angle θ i ,θ j ,θ k for Step 2.4: Angle threshold judgment: If the angle θ i ,θ j ,θ k There is an angle less than the angle threshold θ s , then point p i 、p j 、p k A straight line can be formed, denoted as straight line l a , go to step 2.5, otherwise go to step 2.7; Step 2.5: Determine the inner point of the line: Initialize the number of line points to S, traverse all edge points in the edge point set O except the seed point, and determine the edge point to the line l a Is the distance less than the inner point distance threshold T? d1 : If the distance is less than the threshold, add this edge point to the line l a The interior point set O a , add 1 to S; after the traversal is completed, go to step 2.6; Step 2.6: Store points in the line: Determine whether the number of line points S is greater than the line point threshold T S :If it is established, record the constructed straight line l a The interior point set O a , increase the line number a by 1 and return to step 2.2; otherwise, go to step 2.7; Step 2.7: Line detection completion judgment: set the failure counter f time Add 1 and judge f time Is it greater than T time :If it is established, the improved RLD straight line detection process is ended to obtain a plurality of straight lines l a The interior point set O a Otherwise, return to step 2.

2.

3. The method for visual detection and alignment of polygonal targets based on geometric feature similarity according to claim 1, characterized in that The straight line simplification and splicing method based on spatial line feature similarity specifically comprises the following steps: Step 4.1: Parameter initialization: Set the number of polygon edges to M and the slope similarity threshold to T S , the direction vector similarity threshold is T V , the distance threshold for determining collinearity of straight lines is T d2 , the length of each side of the polygon outline is c k , k = 1, 2...M, the edge length threshold is T ck ; Initialize the main line number i of the line classification to 1, and the line number j to be classified to 1; Step 4.2: Main line selection: Take a line from the initial line set as the main line L for line classification i , and the remaining straight lines are taken as the set of straight lines to be classified; Step 4.3: Select the line to be classified: Select an unselected line from the set of lines to be classified as the line to be classified L j ; Step 4.4: 2D line slope similarity classification: Assume that the main line L is the line classification i The slope is denoted as K Li , the line to be classified L j The slope is denoted as K Lj , then L i With L j The slope similarity S ij For S ij =K Li / K Lj If the slope similarity S ij Satisfies the following formula |1-S ij |<T s Then the slopes of the two straight lines are judged to be similar, and the straight line to be classified L j Add the main straight line L for straight line classification i The slope similarity set n i ; Slope similarity S ij After judgment, the serial number j of the line to be classified is increased by 1; Step 4.5: Determine whether the traversal of the lines to be classified is complete: Determine whether the traversal of all the lines in the set of lines to be classified is complete: If the traversal is complete, classify the line as the main line L i Add slope similarity set n i , and the slope similarity set n i All the lines in are deleted from the initial line set, the sequence number j of the line to be classified is initialized to 1, and the process goes to step 4.6; if the traversal is not completed, the process goes back to step 4.3; Step 4.6: Main line traversal completion judgment: Determine whether the initial line set is empty: If it is empty, combine multiple slope similarity sets n i Construct slope similarity set N P , initialize the main line number i of the straight line classification to 1, and go to step 4.7; otherwise, add 1 to the main line number i of the straight line classification, and return to step 4.2; Step 4.7: Slope similarity set selection: In the slope similarity set group N P Traverse and select the main straight line L i The slope similarity set n i ; Step 4.8: 2D-3D Line Transformation: In the Slope Similarity Set n i Select two straight lines L in the traversal a With L b ; According to the intrinsic parameter matrix of the depth camera and the depth value of the detected pixel point, the pixel coordinates of the starting point and the end point of the straight line are transformed from the image coordinate system to the three-dimensional coordinates in the camera coordinate system. The pixel coordinates of the detection point are recorded as The depth camera intrinsic parameter matrix is ​​M col , the pixel depth value is Z C , then the three-dimensional coordinates of the pixel point in the camera coordinate system are for Step 4.9: Determine whether two lines are in different planes based on the direction vector of the three-dimensional line. Assume that X a , Y a , Z a The straight line L a The three-dimensional coordinate difference between the starting point coordinate and the end point coordinate, X b , Y b , Z b The straight line L b The three-dimensional coordinate difference between the starting point coordinates and the end point coordinates, and the formula for judging the line non-coplanarity are as follows: If the above equations are all true, then the straight line L a and L b If the two lines are coplanar, go to step 4.10; otherwise, the two lines are skew lines, return to step 4.8; Step 4.10: Determine whether two straight lines are collinear based on the distance between them. a With L b Select a series of key points at a certain interval and calculate L a The starting point, end point and key point of L b The distance between the starting point, end point and key point; if there is a distance less than the straight line collinearity determination distance threshold T d2 , then the above two straight lines are collinear, and go to step 4.11; otherwise, the above two straight lines are not collinear, and return to step 4.8; Step 4.11: Three-dimensional collinear line stitching: Connect the above two collinear lines L a With L b Splice; calculate L a The starting point, end point and L b The distance between any two points in the starting point and the end point of the splicing line L is the two points with the farthest distance between them. c The starting point and end point of the splicing result straight line L c Keep and update its geometric properties, and convert the original line L a With L b From the slope similarity set n i Delete from; Step 4.12: Line combination traversal is completed: If the slope is similar to the set n i If all the straight line combinations in are traversed, go to step 4.13; otherwise, return to step 4.8; Step 4.13: Slope similarity set traversal completed judgment: If the slope similarity set group N P All slope similarity sets n in i After the traversal is completed, the simplified slope similarity set N is obtained R , go to step 4.14; otherwise, add 1 to the main line number i of the line classification, and return to step 4.7; Step 4.14: 3D straight line side length screening: traverse and simplify the slope similarity set group N R For each straight line in the 3D coordinates of the starting and ending points of the straight line, the straight line length d is calculated. If the straight line length d is equal to the polygon outline side length c k Satisfy the following formula: |d-c k |<T ck ,k=1,2...M If the line is not in the correct order, the line is retained; otherwise, the line is deleted; Step 4.15: Line traversal completion judgment: judge the simplified slope similarity set group N R Are all the straight lines in the traversal completed? If so, then the similarity set n of all slopes sorted from small to large is i Construct a set of precise simple lines N L , end the straight line simplification and splicing process; otherwise, return to step 4.

14.

4. According to claim 1, a polygonal target visual detection and alignment method based on geometric feature similarity is characterized in that The polygon outline extraction method based on geometric feature similarity specifically comprises the following steps: Step 5.1: Parameter initialization: Set the number of target polygon edges to M and the line end connection determination distance threshold to T d3 , set the target polygon perimeter and perimeter threshold to A and T respectively A ; Step 5.2: Method feasibility judgment: Determine whether the number of polygon edges M is greater than the set of precise simple lines N L Medium slope similarity set n i Number N: If true, the polygon outline cannot be extracted, and return to step 1 to collect the target image and reprocess it; otherwise, go to step 5.3; Step 5.3: Slope similarity set group selection: for the precise simple line set N L All slope similarity sets n in i , i = 1, 2...N, according to M slope similarity sets n i The rule of non-repetitive union is to traverse and select M sets of similar slopes n i , denoted as β1, β2...β M ; Step 5.4: Contour line combination selection: According to the rule that one straight line is selected from each slope similarity set and M straight line combinations are not repeated, select M slope similarity sets β1, β2...β M In the traversal, select M straight lines, denoted as l1, l2...l M , initialize the line end connection main straight line number j to 1; Step 5.5: Line end connection threshold judgment: Calculate the line end connection main straight line l in sequence j The starting point and end point of the other straight lines l j+1 , l j+2 ... M The distance between the starting point and the end point; if the distance between any two points is less than the line end connection determination distance threshold T d3 If both points are not connected to other points, connect the two points; after completing the distance calculation and threshold judgment between all two points, add 1 to the main straight line number j of the line end connection; Step 5.6: Determine whether the line end connection traversal is completed: Determine whether the line end connection main straight line number j is equal to the number of polygon edges M. If the condition is met, go to step 5.7; otherwise, return to step 5.5; Step 5.7: Polygonal contour closure screening: determine l1, l2, l3...l M Whether the starting point and the end point of can be connected to other points: If the condition is met, the M straight lines are combined into a polygon, and the polygon model including the straight line set and the corner point set is stored in a clockwise direction, and the process goes to step 5.8; otherwise, the process goes to step 5.9; Step 5.8: Polygon perimeter threshold judgment: Use the three-dimensional coordinates of the starting point and end point of each edge of the polygon to calculate the polygon perimeter B, and compare it with the target polygon perimeter A. If the following formula is satisfied |B-A|<T A Then the above polygon is stored in the potential polygon outline set P; otherwise, the polygon model is deleted; Step 5.9: Line combination traversal is completed: judge in the slope similarity set β1, β2...β M Is the combined traversal of the contour lines in completed? If completed, proceed to step 5.10; otherwise, return to step 5.4; Step 5.10: Slope similarity set group traversal completion judgment: judge in the refined simple line set group N L There are M sets of similar slopes β1, β2, ... β M Is the traversal completed: If completed, go to step 5.11; otherwise, return to step 5.3; Step 5.11: Target polygon outline selection: In the potential polygon outline set P, calculate the perimeter of each polygon in turn, select the polygon outline closest to the target polygon perimeter A as the target polygon outline, and end the polygon outline extraction process.

5. According to claim 1, a polygonal target visual detection and alignment method based on geometric feature similarity is characterized in that The rotating caliper method specifically comprises the following steps: Step 6.1: Parameter initialization: Assume that the line set X of the target polygonal model is {L1, L2, L3, ..., L M }; The corner point set P is {P1, P2, P3, ....., P M }; Initialize the main parallel line number m to 1; Step 6.2: Main parallel line selection: Select each line in the target polygon line set X in order as the main parallel line L m ; Step 6.3: Secondary parallel line selection: Calculate the distance from all corner points in the target polygon corner point set P to the primary parallel line L m The projection distance of the corner point with the largest projection distance is selected as P M , past point P M Main parallel line L m The parallel line of is called secondary parallel line; Step 6.4: Construct a circumscribed rectangle: Draw a straight line perpendicular to the main parallel line through all corner points in the target polygon corner point set P, and record it as the main parallel line perpendicular line set; calculate the distance between any two perpendicular lines in the main parallel line perpendicular line set, select the two perpendicular lines with the largest distance as the sides of the circumscribed rectangle, and construct a circumscribed rectangle with the main parallel line and the secondary parallel line, and store it in the target contour circumscribed rectangle set H R ; Step 6.5: Determine if the rotation and construction of the rectangle is completed: add 1 to the main parallel line number m, and determine whether the main parallel line number m is greater than the number of sides of the target polygon M: if the condition is met, proceed to step 6.6, otherwise proceed to step 6.2; Step 6.6: Minimum bounding rectangle selection: For the target contour bounding rectangle set H R For each bounding rectangle in the image, calculate the pixel area of ​​the rectangular image, select the bounding rectangle with the smallest pixel area as the minimum bounding rectangle of the target contour, and select the image center coordinates (x c ,y c ) is the target center.

6. According to claim 1, a polygonal target visual detection and alignment method based on geometric feature similarity is characterized in that The polygonal target posture deviation calculation method specifically comprises the following steps: Step 7.1: Create polygonal target space plane: For the target polygon corner point set P, calculate the three-dimensional coordinates of each corner point in the camera coordinate system, and record the pixel coordinates of the corner point as The depth camera intrinsic parameter matrix is ​​M col , the corner depth value is Z i , then the three-dimensional coordinates of the corner point in the camera coordinate system are for According to the three-dimensional coordinates of all corner points in the target polygon corner point set P, the spatial plane model of the target polygon outline in the camera coordinate system is calculated using the least squares method. AX+BY+CZ+D=0 Among them, (X, Y, Z) is the three-dimensional coordinates of the corner point in the camera coordinate system, A, B and C are the model parameters of the space plane; Step 7.2: Calculate the target center spatial position deviation; the pixel coordinates of the target center are known to be (x c ,y c ), the three-dimensional coordinates of the target center in the camera coordinate system are marked as (X c , Y c , Z c ); Since the camera coordinate system is fixed to the docking device coordinate system, the X-axis position deviation between the docking device and the target center is e dX , Y-axis position deviation e dY and Z-axis position deviation e dZ They are Where f x 、f y , u0, v0 are the internal parameters of the depth camera; Step 7.3: Calculation of polygonal target attitude angle deviation: Assuming the camera plane normal vector is (0, 0, 1), for the polygonal target space plane normal vector (A, B, C), set the normal vector Z-axis direction greater than 0, and the attitude angle deviation is positive in the counterclockwise direction around the coordinate axis. Then the X-axis attitude angle deviation between the docking device and the polygonal target is e θX 、Y-axis attitude angle deviation e θY They are

Images