An out-of-plane point co-signaller and a method of measurement

Abstract

The present application belongs to the field of computer vision, and specifically relates to a kind of out-of-plane point cooperation marker and measuring method, and the marker includes: base surface, stand and multiple marker points;Multiple stands are arranged on the base surface;Marker points are arranged on the base surface and the stand respectively, and the marker points include: circular ring marker point and circular point marker point;Among them, the marker point arranged on the base surface is base surface marker point, and the marker point arranged on the top surface of the column is column surface marker point;The base surface marker point is circular ring marker point, and is evenly distributed on the four corners of the base surface along the center point of the base surface;The column surface marker point is circular ring marker point or circular point marker point.The present application has the characteristics of fast identification, and has strong anti-interference ability, and can still accurately complete target identification in the case of losing individual marker points.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, in particular to a non-planar point cooperation marker and a measuring method. BACKGROUND

[0002] According to different application scenarios and task requirements, visual measurement can be divided into monocular visual measurement, multi-view visual measurement, structured light visual measurement, etc. Among them, multi-view visual measurement mainly uses the texture information of the target and the background in the scene for binocular feature matching and relative pose calculation, and the algorithm is relatively complex and time-consuming. Structured light measurement is equivalent to increasing the texture information in the scene, which is beneficial to the extraction of the three-dimensional point cloud of the target, but if the point cloud is relatively dense, it will also bring the problem of large calculation amount and slow speed. In many application scenarios, through reasonable layout and configuration, the marker measurement based on monocular vision can obtain the characteristics of high measurement accuracy, fast measurement response speed and strong anti-interference ability. Marker measurement based on monocular vision has a wide range of application cases in many fields such as spacecraft rendezvous, on-orbit maintenance, submarine stop, mechanical arm target grabbing, etc.

[0003] X. Zhou and F. Zhu published a paper entitled "Discussion on the uniqueness condition of P3P problem solution" in the Computer Science Journal in the 12th issue of 2003. From the perspective of engineering application, the paper discusses the multiple solution problem of P3P problem and the unique solution condition. The invention patent with the application number 201310639611.6 "Monocular visual pose measurement method based on point features" uses the marker points in isosceles triangle distribution to construct the unique solution condition of P3P problem, and uses the bisection iterative algorithm to realize the fast solution of the relative pose. In some application scenarios, under relatively complex working conditions, the marker points on the marker may appear problems such as reflection, shielding, and a large number of light spot interference after imaging, which may cause identification failure or misidentification, seriously affecting the efficiency of target measurement. Therefore, the anti-interference ability of the marker point identification should also be fully considered in the design of the marker. SUMMARY

[0004] The purpose of the present application is to provide a non-planar structure marker composed of circular ring marker points and circular point marker points, to improve the anti-interference ability of marker identification by using the exclusivity of circular ring marker points in image processing, and to improve the relative pose measurement accuracy of the marker by using the non-planar structure. When identifying the marker, first, the circular ring feature points are searched from the relatively small number of circular ring feature points, and then the candidate minimum units that meet the model constraints are searched in combination with the circular point feature points. The P3P algorithm is used to calculate the initial value of the relative pose, and then other feature points are searched according to the model constraints. Finally, a nonlinear optimization algorithm is used to obtain the accurate solution of the relative pose.

[0005] The technical scheme adopted by the present application to achieve the above object is as follows: a non-planar point cooperation marker, comprising a base surface, a column and a plurality of marker points;

[0006] The column has a plurality of columns, which are arranged on the base surface; the marker points are arranged on the base surface and the column respectively, and the marker points comprise circular ring marker points and circular point marker points;

[0007] Among them, the marker points arranged on the base surface are base surface marker points, and the marker points arranged on the top surface of the column are column surface marker points;

[0008] The base surface marker points are circular ring marker points, which are uniformly distributed on the four corners of the base surface along the center point of the base surface; the column surface marker points are circular ring marker points or circular point marker points.

[0009] The column comprises a first column, a second column and a third column, and the center point of the base surface is provided with the first column, and the second column and the third column are symmetrically arranged on the upper and lower sides of the column along the center point of the base surface;

[0010] The first column and the second column are respectively provided with circular point marker points, and the third column is provided with circular ring marker points;

[0011] The center point of the circular point marker points on the second column is on the same horizontal line as the center point of the circular ring marker points on the two corners of the top of the base surface; the center point of the circular ring marker points on the third column is on the same horizontal line as the center point of the circular ring marker points on the two corners of the bottom of the base surface;

[0012] The center points of the circular ring marker points on the base surface are sequentially connected to form a square;

[0013] The vertical projection of the first column on the base surface is located at the center point of the square, and the vertical projections of the second column and the third column on the base surface are located at the midpoint positions of the upper and lower edges of the square respectively.

[0014] The diameter size of the circular point marker points is two-thirds of the diameter size of the circular ring marker points, and the inner diameter size of the circular ring marker points is equal to one-half of the outer diameter size of the circular ring marker points; the inner circle and the outer circle of the circular ring marker points are concentric circles.

[0015] A measurement method of a non-planar point cooperation marker, comprising the following steps:

[0016] 1) Marking the marker points on the marker, and establishing a marker coordinate system according to the marker points;

[0017] 2) According to the distribution of the marker points on the marker, a plurality of minimum recognition units are set; wherein each minimum recognition unit is composed of one column surface marker point and two base surface marker points;

[0018] 3) Place the marker in front of the camera lens, the inclination of the marker is within the set angle range, and the marker is imaged after the camera lens, and the circular ring mark point and the circular point mark point in the image are identified, and the image coordinates of the mark points are corrected for distortion;

[0019] 4) Start searching for the smallest recognition unit from the circular ring mark point, that is, select two circular ring mark points that meet the set distance constraint from all the circular ring mark points found in the image processing process, and then select a circular point mark point from all the circular point mark points to form a candidate smallest recognition unit together with the two circular ring mark points;

[0020] 5) Assuming that the candidate smallest recognition unit is one of the multiple sets of smallest recognition units set in step 2), set the model coordinates of the mark points, combine the imaging coordinates of the mark points in the smallest recognition unit, and solve the initial value of the relative pose of the marker to obtain the initial value of the pose of the marker coordinate system relative to the camera coordinate system;

[0021] 6) By obtaining the initial value of the pose of the marker coordinate system relative to the camera coordinate system, the spatial coordinates of the mark points on the marker in the three-dimensional model of the marker, and the camera model parameters obtained by calibration, the imaging coordinates of other mark points on the marker in the image are obtained. Using the imaging coordinates of the mark points, match the mark points found in the image to obtain the mark points that match successfully;

[0022] 7) According to the mark points that match successfully, taking the re-projection error as the objective function, taking the relative pose between the marker coordinate system and the camera coordinate system as the optimization variable, and taking the relative pose solved by the P3P algorithm as the initial value of the optimization variable, the relative pose is solved by nonlinear optimization, and the relative pose between the marker coordinate system and the camera coordinate system is obtained;

[0023] 8) Perform coordinate system conversion on the obtained relative pose between the marker coordinate system and the camera coordinate system: the relative pose between the passive end coordinate system fixed to the marker and the active end coordinate system fixed to the camera.

[0024] The step 1) is specifically:

[0025] The mark point on the first column is marked as 0, the mark point on the second column is marked as 1, the mark point at the right upper corner of the base surface is marked as 2, the mark point at the right lower corner of the base surface is marked as 3, the mark point on the third column is marked as 4, the mark point at the left lower corner of the base surface is marked as 5, and the mark point at the left upper corner of the base surface is marked as 6;

[0026] The projection point of the 0th marker point on the base surface is defined as the origin of the marker coordinate system, the direction from the 6th marker point to the 2nd marker point is defined as the positive direction of the X-axis of the marker coordinate system, and the direction from the 6th marker point to the 5th marker point is defined as the positive direction of the Y-axis of the marker coordinate system.

[0027] In step 2), the multiple groups of minimum recognition units are set as:

[0028] {0, 2, 3}, {0, 3, 5}, {0, 5, 6}, {0, 6, 2}, {0, 2, 5}, {0, 6, 3}, {1, 6, 2}, {4, 3, 5};

[0029] wherein the numbers 0 to 6 correspond to the 0th marker point to the 6th marker point, respectively;

[0030] wherein the layout of the three marker points in each of the first four groups {0, 2, 3}, {0, 3, 5}, {0, 5, 6}, and {0, 6, 2} on the marker is clockwise ordered;

[0031] the first marker point in each of the last four groups {0, 2, 5}, {0, 6, 3}, {1, 6, 2}, and {4, 3, 5} is located at the middle position of the projections of the second and third marker points on the base surface;

[0032] The candidate minimum recognition units are filtered by setting a constraint condition.

[0033] In step 6), the matching is performed on the marker points found in the image, and the marker points that pass the matching are obtained, specifically as follows:

[0034] If the Euclidean distance between the imaging coordinates and the coordinates of a certain marker point is less than a set threshold, that is, the marker point and the image feature point pass the matching, and if the number of matching points meets the set number constraint, the target has been recognized; otherwise, the pose initial value of the marker coordinate system relative to the camera coordinate system of the other marker points solved by the P3P algorithm is continuously determined;

[0035] If none of the poses solved by the P3P algorithm meets the matching point number constraint condition, the candidate minimum recognition units are assumed to be other cases of the multiple groups of minimum recognition units set in step 2), until all cases of the multiple groups of minimum recognition units are traversed, and until the target is correctly recognized.

[0036] The step 7) is specifically as follows:

[0037] The pose initial value of the marker coordinate system relative to the camera coordinate system is obtained, the re-projection error is taken as the objective function E, the relative pose between the marker coordinate system and the camera coordinate system is taken as the optimization variable, and the rotation and translation relationship (R t) of the marker coordinate system relative to the camera coordinate system is obtained by nonlinear optimization, that is:

[0038]

[0039]

[0040] wherein (X i ,Y i ,Z i ) is the coordinate of the i-th marker point in the marker coordinate system, (u i ,v i ) is the imaging coordinate of the corresponding i-th marker point, (R t) is the rotation and translation relationship of the marker coordinate system relative to the camera coordinate system, the corresponding position quantity of (R t) is tx, ty, tz, the Euler angle is ax, ay, az, fu, fv, u0, v0 is the internal parameter of the camera, is the imaging coordinate of the marker point recalculated through (X i ,Y i ,Z i ), the relative rotation and translation relationship (R t) and the camera internal parameter.

[0041] The step 8) is specifically:

[0042] Let the rotation and translation matrix represent the relative pose relationship of the passive end coordinate system and the marker coordinate system, let the rotation and translation matrix represent the relative pose relationship of the camera coordinate system and the active end coordinate system, let the rotation and translation matrix represent the relative pose of the marker coordinate system relative to the camera coordinate system, and let the rotation and translation matrix represent the relative pose of the passive end coordinate system relative to the active end coordinate system:

[0043]

[0044] wherein, and are fixed parameters and can be known by calibration, is (R t) calculated by nonlinear optimization.

[0045] The present application has the following beneficial effects and advantages:

[0046] 1. The marker is designed to use a non-planar marker point layout, which can effectively improve the measurement accuracy in a limited size space of the marker.

[0047] 2. The marker is designed to use a circular ring feature point with strong exclusivity, which can reduce the search range of the feature point in a complex background condition, and realize fast positioning of the feature point combination in the image.

[0048] 3. The recognition algorithm uses a search strategy based on the smallest recognition unit, which can still accurately identify the target even when some markers are lost, and has strong anti-interference ability. Attached Figure Description

[0049] Figure 1 Schematic diagram of the marker structure of this invention;

[0050] Figure 2 In this embodiment of the invention, the marker is configured with 8 sets of minimum identification units;

[0051] Figure 3 Schematic diagram of the marker point of the marker device of the present invention;

[0052] Figure 4 The two-dimensional three-view diagram of the marker structure of this invention. Detailed Implementation

[0053] like Figure 1 The diagram shown is a design schematic of the marker in this invention patent. The marker points on the marker include circular marker points and round marker points. The marker consists of a base surface and three pillars on the base surface. The marker points are distributed on the base surface and the pillars.

[0054] Based on the distribution area of ​​the marker points on the marker, the marker points are divided into base surface marker points and cylindrical surface marker points. Marker points distributed on the base surface are called base surface marker points, and marker points distributed on the cylinder are called cylindrical surface marker points.

[0055] The base surface markers are circular markers, distributed at the four corners of the base surface of the marker. The marker is divided into left, center, and right areas. The cylinder is located in the middle part of the marker. The three cylinders are distributed at the top, middle, and bottom. The bottom cylinder marker is also a circular marker, and the other two cylinder markers are round dots.

[0056] The column includes: a first column, a second column and a third column. The first column is located at the center point of the base surface, and the second and third columns are symmetrically located on the upper and lower sides of the column at the center point of the base surface.

[0057] The first and second pillars are marked with round dots, and the third pillar is marked with a circular ring.

[0058] The center point of the dot mark on the second column is on the same horizontal line as the center point of the circular mark at the top two corners of the base surface; the center point of the circular mark on the third column is on the same horizontal line as the center point of the circular mark at the bottom two corners of the base surface.

[0059] The diameter of the dot marker is two-thirds the size of the outer diameter of the ring marker, and the inner diameter of the ring marker is equal to half the outer diameter of the ring marker; the inner and outer circles of the ring marker are concentric circles.

[0060] The center line of the base surface mark points in the four corners forms a square, the vertical projection of the upper cylindrical mark point on the base surface is located at the midpoint of the upper two base surface mark points, the vertical projection of the lower cylindrical mark point on the base surface is located at the midpoint of the lower two base surface mark points, and the vertical projection of the middle cylindrical mark point on the base surface is located at the center of the upper four base surface mark points.

[0061] As shown in Figure 1 , it can be seen that four circular ring mark points are processed on the base surface of the marker, and three through holes of 6.35 mm (three positions marked as 7 / 8 / 9 in Figure 2 ) are provided, and the three through holes are mainly used for collecting data for precision test by the laser tracker.

[0062] Figure 4 Specific size information is given. The diameter of the circular point mark point on the marker is 10 mm, the outer diameter of the circular ring mark point is 15 mm, and the inner diameter is 7.5 mm; the height of the middle cylinder is 30 mm, and the diameter of the cylinder is 18 mm; the height of the upper and lower two cylinders is 20 mm, the diameter of the upper cylinder is 18 mm, and the diameter of the lower cylinder is 25 mm. The center distance of the base surface circular ring mark point is 65 mm. The surface of the marker is treated with black light-absorbing material, the area where the mark points are located is concave by 0.2 mm relative to the surface, and is coated with diffuse white paint.

[0063] The marker recognition and measurement method specifically includes the following steps:

[0064] According to the image gray distribution, a binary threshold T0 is set, the image is binarized, and pixels greater than the binary threshold are defined as target pixels, and the rest are defined as background pixels;

[0065] Region growing, finding a connected region in the image that satisfies the aspect ratio of the circumscribed rectangle not greater than T1 (for example, 1.6), according to the characteristics that the center of the circular ring feature point is a hole composed of background pixels and the circular point feature point is a solid connected region, it is judged whether there is a hole near the center of the region, if the search region does not contain a hole, it is directly classified as a circular point feature point, if there is a hole, the image coordinates of the hole center are further calculated, and compared with the center coordinates of the connected region composed of the corresponding target pixels, if the Euclidean distance between the two is less than T2 (for example, 1.0 pixel), the circular ring feature point constraint is satisfied, and the recognition region is classified as a circular ring feature point;

[0066] 1) Mark the mark points on the marker, as shown in Figure 2As shown, the middle columnar mark point is marked as No. 0 mark point, the upper columnar mark point is marked as No. 1 mark point, the right upper corner base surface mark point is marked as No. 2 mark point, the right lower corner mark point is marked as No. 3 mark point, the lower columnar mark point is marked as No. 4 mark point, the left lower corner mark point is marked as No. 5 mark point, and the left upper corner mark point is marked as No. 6 mark point; the projection point of the No. 0 mark point on the base surface is defined as the origin of the marker coordinate system, the direction from the No. 6 mark point to the No. 2 mark point is defined as the positive direction of the X-axis of the marker coordinate system, and the direction from the No. 6 mark point to the No. 5 mark point is defined as the positive direction of the Y-axis of the marker coordinate system.

[0067] 2) According to the distribution of the mark points on the marker, the following 8 groups of minimum recognition units are set, such as Figure 3 As shown: {0, 2, 3}, {0, 3, 5}, {0, 5, 6}, {0, 6, 2}, {0, 2, 5}, {0, 6, 3}, {1, 6, 2}, {4, 3, 5}.

[0068] Among them, the numbers 0 to 6 correspond to the No. 0 mark point to the No. 6 mark point. Each group of minimum recognition units is composed of one columnar mark point and two base surface mark points. Among them, the layout of the three mark points in the first four groups {0, 2, 3}, {0, 3, 5}, {0, 5, 6}, {0, 6, 2} on the marker is in clockwise order; the first mark point in the last four groups {0, 2, 5}, {0, 6, 3}, {1, 6, 2}, {4, 3, 5} is located in the middle position of the projections of the second and third mark points on the base surface, and these information can be used to set constraint conditions to screen the candidate minimum recognition units. As shown, Figures 2-3 As shown, the marks 0 to 6 are mark points, and 7 / 8 / 9 are target hole positions used for precision test.

[0069] 3) The marker is located in front of the camera lens, the inclination angle of the marker is within a certain range, and after the marker is normally imaged, the circular ring feature points and the circular point feature points in the image are first recognized, and the image coordinates of the feature points are corrected.

[0070] 4) The circular ring feature points with strong exclusivity are used to search for the minimum recognition unit, that is, all the circular ring mark points found in the image processing process are searched, two circular ring mark points satisfying a certain distance constraint are selected from them, and a circular point feature point is selected from all the circular point feature points to form a candidate minimum recognition unit together with the two circular ring feature points.

[0071] 5) Assuming that the candidate minimum recognition unit is one of the 8 groups of minimum recognition units set in step 2, the model coordinates of the mark points are set, the imaging coordinates of the minimum recognition unit are combined, and the P3P algorithm is used to solve the relative pose initial value. In theory, the P3P algorithm has 4 groups of solutions for solving the relative pose, and may contain complex solutions, only the real solutions are checked.

[0072] Since the motion range of the marker relative to the camera is limited, including the measured distance and the possible range of the attitude angle, it is assumed here that the inclination angle of the marker is not greater than T3 (for example, 30°), and the deviation of the roll angle near Theta is not greater than T4 (for example, 20°), and using this information, the initial value of the pose solved by the P3P algorithm can also be further screened.

[0073] For the initial value of the pose that meets certain constraint conditions, in combination with the model coordinates of the marker points on the marker and the camera model parameters, it can be found whether there are matching feature points in the image for other marker points, and the matching error is set to T5 (for example, 1.0 pixel), if the matching points are not less than T6 (for example, 5), it is indicated that the target has been correctly identified, and the verification of other pose solutions of P3P is not performed, and other situations in the 8 groups of minimum identification units set in step 2 are also not assumed; otherwise, the other poses solved by the P3P algorithm are continuously judged, if all the real poses solved by the P3P algorithm do not meet the matching point quantity constraint condition, the candidate minimum identification unit is continuously assumed to be other situations in the 8 groups of minimum identification units set in step 2, until all the situations in the 8 groups of minimum identification units are traversed. If the roll angle motion range of the marker is limited to within 20° near Theta, even if the No. 1 marker point and the No. 4 marker point are lost at the same time, it will not cause identification ambiguity, and in this working condition, the No. 1 marker point and the No. 4 marker point are allowed to be lost at the same time in the target identification process;

[0074] The relative pose solved by the P3P algorithm is as follows:

[0075] The first marker point in the minimum identification unit is marked with the symbol A, the second marker point is marked with the symbol B, and the third marker point is marked with the symbol C. The distance between A and B is marked with d1, the distance between A and C is marked with d2, and the distance between B and C is marked with d3. The optical center of the camera is marked with O, the distance between O and A is marked with a, the distance between O and B is marked with b, and the distance between O and C is marked with c. The angle between AOB is marked with alpha, the angle between AOC is marked with beta, and the angle between BOC is marked with gamma. Since the marker model is known, the coordinates of the marker points in the marker coordinate system can also be obtained, so d1, d2, and d3 are known quantities. The imaging point corresponding to the marker point A is marked with D, the imaging point corresponding to the marker point B is marked with E, and the imaging point corresponding to the marker point C is marked with F. There are three collinear points O, A, and D, three collinear points O, B, and E, and three collinear points O, C, and F, so there are ∠AOB = ∠DOE, ∠AOC = ∠DOF, and ∠BOC = ∠EOF. The imaging points D, E, and F have been determined in the image processing feature point search, and the corresponding imaging coordinates are also known quantities, and then the specific values of alpha, beta, and gamma can be calculated using the cosine law.

[0076] According to the cosine theorem:

[0077] In triangle AOB, we have

[0078] In triangle AOC, we have

[0079] In triangle BOC, we have

[0080] The above d1, d2 and d3, and a, b, g are known quantities, and the three equations can be solved for a, b and c. Since the direction vectors of the lines connecting points A, B and C to the optical center in the camera coordinate system are calculated by corresponding imaging coordinates and the optical center, and the distances from points A, B and C to the optical center are obtained through the above solving process, the three-dimensional coordinates of points A, B and C in the camera coordinate system can also be obtained using the product of the direction vectors and the distances. At this time, the three-dimensional coordinates of points A, B and C in the camera coordinate system and the three-dimensional coordinates in the marker coordinate system are known, and the relative pose of the marker coordinate system relative to the camera coordinate system can be obtained.

[0081] 6) According to the successfully matched marker points and feature points found in step 5, taking the re-projection error as the objective function, taking the relative pose between the marker coordinate system and the camera coordinate system as the optimization variable, and taking the pose solved by the P3P algorithm as the initial value of the optimization variable, a nonlinear optimization relative pose solving is performed. The nonlinear optimization algorithm uses the LM optimization algorithm.

[0082] The relative pose solved by the above marker recognition algorithm using the P3P algorithm can be compared with the possible pose range under certain working conditions. If the solved pose exceeds the possible pose range, it is directly excluded and does not participate in the next calculation.

[0083] In addition, if the attitude angle range of the marker relative to the camera is limited, the 1st marker point and the 4th marker point can be lost at the same time under the condition that there is no recognition ambiguity, otherwise at least one of the 1st marker point and the 4th marker point should be correctly identified.

[0084] 7) Coordinate system conversion, the relative pose between the marker coordinate system and the camera coordinate system is obtained through the above calculation process, and according to the prior calibration parameters, the relative pose is converted to the relative pose between the passive end coordinate system fixed to the marker and the active end coordinate system fixed to the camera.

[0085] Wherein, the active end is a structure fixed with the camera, and the passive end is a structure fixed with the marker. According to different actual measurement tasks, the specific objects of the active end and the passive end are also different. For example, in the process of mechanical arm operation, the camera is assembled at the end of the mechanical arm, the end of the mechanical arm is assembled with an operating tool, and the marker is assembled near the operating component. Through visual measurement, the mechanical arm carrying the operating tool is guided to complete the action on the operating component; or in the docking task of space spacecraft, the camera is installed on one of the spacecrafts, and the marker is assembled on the other spacecraft. The docking mechanism of the spacecraft assembling the camera is called the active end, and the spacecraft assembling the marker is called the passive end.

[0086] The nonlinear optimization relative pose solving is generally an iterative optimization process, which needs to determine three input conditions of optimization variables, optimization variable initial values and objective functions. The following specifically describes the three input conditions.

[0087] It is known that n markers are successfully identified, wherein the coordinates of the i-th marker in the marker coordinate system are (X i ,Y i ,Z i ), the corresponding imaging coordinates are (u i ,v i ), and the rotation and translation relationship of the marker coordinate system relative to the camera coordinate system is represented by a 3*4 matrix (R t), wherein R is a 3*3 rotation matrix, the corresponding Euler angles are ax, ay, az, t is a 3*1 column vector (tx, ty, tz)′. fu, fv are the focal ratios of the camera, u0, v0 are the principal point coordinates of the camera, and fu, fv, u0, v0 are collectively referred to as camera intrinsic parameters, which can be obtained through prior camera calibration. The image coordinates of the re-projection can be calculated by the model parameters (X i ,Y i ,Z i ), the relative rotation and translation relationship (R t) and the camera intrinsic parameters, and the specific calculation is as follows:

[0088]

[0089] Let the position variables tx, ty, tz and the Euler angles ax, ay, az be the optimization variables, and the sum of squares of the deviations of the image coordinates of the identified feature points and the image coordinates of the marker re-projection be the objective function of the nonlinear optimization relative pose solving, as shown below:

[0090]

[0091] Let the rotation and translation matrix represent the relative pose relationship between the passive end coordinate system and the marker coordinate system, and let the rotation and translation matrix The relative pose relationship between the camera coordinate system and the active end coordinate system is represented by a rotation and translation matrix The relative pose relationship between the marker coordinate system and the camera coordinate system is represented by a rotation and translation matrix The relative pose relationship between the passive end coordinate system and the active end coordinate system is represented by a rotation and translation matrix

[0092]

[0093] wherein, and are fixed parameters and can be known through calibration, represents (R t) calculated by a nonlinear optimization,

[0094] In summary, the present application sets the minimum recognition unit during marker recognition, which is composed of one cylindrical marker point and two base surface marker points, uses the strong exclusivity of the circular ring marker point to narrow the search range, quickly locates the target, uses the P3P algorithm to solve the initial value of the relative pose between the marker and the camera, uses the initial value information to further search for the feature points in the image that match the marker points, and finally uses the nonlinear optimization method to obtain the accurate solution of the relative pose.

[0095] The above only describes the embodiments of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, extension, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A cooperative marker for skew points, characterized in that, The utility model relates to a kind of cooperative marker of skew field point, its measurement method, comprising the following steps: 1) the marker point on the marker is marked, and marker coordinate system is established according to the marker point; 2) according to the distribution of marker point on the marker, set multiple groups of minimum recognition unit;Wherein, each group of minimum recognition unit is composed of one column surface marker point and two base surface marker points; 3) the marker is placed in front of camera lens, the inclination of the marker is in the setting angle range, after the marker is imaged in camera lens, the circular ring marker point and the dot marker point in image are identified, and the image coordinates of the marker point are distorted correction; 4) search minimum recognition unit starting from circular ring marker point, i.e. all circular ring marker points found in image processing process are selected from two circular ring marker points satisfying the set distance constraint, and then a dot marker point is selected from all dot marker points, to form a candidate minimum recognition unit with two circular ring marker points; 5) assuming that candidate minimum recognition unit is one of multiple groups of minimum recognition units set in step 2), set the model coordinates of the marker point, and combine the imaging coordinates of the marker point in minimum recognition unit to solve the initial value of relative pose of the marker, to obtain the initial value of the pose of the marker coordinate system relative to the camera coordinate system; 6) by obtaining the initial value of the pose of the marker coordinate system relative to the camera coordinate system, the space coordinates of the marker point on the marker in the three-dimensional model of the marker, and the camera model parameters of the calibration, the imaging coordinates of other marker points on the marker in the image are obtained, and the imaging coordinates of the marker point are used to match from the marker points found in the image, to obtain the marker points matched successfully; 7) according to the marker points matched successfully, the re-projection error is used as the objective function, the relative pose between the marker coordinate system and the camera coordinate system is used as the optimization variable, and the relative pose solved by P3P algorithm is used as the initial value of the optimization variable to solve the relative pose by nonlinear optimization, to obtain the relative pose between the marker coordinate system and the camera coordinate system; 8) the relative pose between the marker coordinate system and the camera coordinate system obtained is converted: the relative pose between the passive end coordinate system fixed with the marker and the active end coordinate system fixed with the camera. The column includes: first column, second column and third column, the center point of the base surface is provided with the first column, and the second column and the third column are symmetrically arranged on the upper and lower sides of the column along the center point of the base surface; The first column, the second column are respectively provided with dot marker point, and the third column is provided with circular ring marker point. ​ ​ ​ ​ 2. An out-of-plane point marker according to claim 1, wherein ​ ​ The center point of the dot mark point on the second column is on the same horizontal line with the center point of the ring mark point at the two corners of the top of the base surface; the center point of the ring mark point on the third column is on the same horizontal line with the center point of the ring mark point at the two corners of the bottom of the base surface.

3. An out-of-plane point marker according to claim 2, wherein, The center points of the ring mark points on the base surface are sequentially connected to form a square; The vertical projection of the first column on the base surface is located at the center point of the square, and the vertical projections of the second column and the third column on the base surface are located at the midpoint positions of the upper and lower edges of the square, respectively.

4. An out-of-plane point marker according to claim 1, wherein The diameter of the dot mark point is two-thirds of the diameter of the ring mark point, and the inner diameter of the ring mark point is equal to one-half of the outer diameter of the ring mark point; the inner circle and the outer circle of the ring mark point are concentric circles.

5. An out-of-plane point marker according to claim 1, wherein The step 1) is specifically: The mark point on the first column is marked as the 0th mark point, the mark point on the second column is marked as the 1st mark point, the mark point at the right upper corner of the base surface is marked as the 2nd mark point, the mark point at the right lower corner of the base surface is marked as the 3rd mark point, the mark point on the third column is marked as the 4th mark point, the mark point at the left lower corner of the base surface is marked as the 5th mark point, and the mark point at the left upper corner of the base surface is marked as the 6th mark point; The projection point of the 0th mark point on the base surface is defined as the origin of the marker coordinate system, the direction from the 6th mark point to the 2nd mark point is defined as the positive direction of the X-axis of the marker coordinate system, and the direction from the 6th mark point to the 5th mark point is defined as the positive direction of the Y-axis of the marker coordinate system.

6. An out-of-plane point marker according to claim 1, wherein In the step 2), the multiple groups of minimum recognition units are set as: {0,2,3},{0,3,5},{0,5,6},{0,6,2},{0,2,5},{0,6,3},{1,6,2},{4,3,5}； Wherein, the numbers 0 to 6 correspond to the 0th mark point to the 6th mark point, respectively; In the first four groups {0, 2, 3}, {0, 3, 5}, {0, 5, 6}, and {0, 6, 2}, the three mark points in each group are arranged in a clockwise order on the marker; In the last four groups {0, 2, 5}, {0, 6, 3}, {1, 6, 2}, and {4, 3, 5}, the projection of the first mark point on the base surface is located at the middle position between the second and third mark points; The candidate minimum recognition units are screened by setting the constraint conditions.

7. An out-of-plane point marker according to claim 1, wherein In the step 6), the matching is performed on the mark points found in the image, and the mark points with successful matching are obtained, which is specifically: If the Euclidean distance between the imaging coordinates and the coordinates of a certain mark point is less than a set threshold, that is, the mark point is successfully matched with the image feature point, if the matching points meet the set number constraint, the target has been recognized; otherwise, the pose initial value of the marker coordinate system relative to the camera coordinate system of the other mark points solved by the P3P algorithm is continuously judged; If none of the poses solved by the P3P algorithm meets the matching point number constraint condition, the candidate minimum recognition units are continuously assumed to be other situations of the multiple groups of minimum recognition units set in the step 2), until all situations in the multiple groups of minimum recognition units are traversed, and until the target is correctly recognized.

8. An out-of-plane point marker according to claim 1, wherein, The step 7) is specifically: An initial value of a pose of the marker coordinate system relative to the camera coordinate system is obtained, a re-projection error is taken as a target function E, a relative pose between the marker coordinate system and the camera coordinate system is taken as an optimization variable, and a rotation and translation relationship (R t) of the marker coordinate system relative to the camera coordinate system is obtained through nonlinear optimization, that is: wherein (X i ,Y i ,Z i ) is the coordinate of the i-th marker point in the marker coordinate system, (u i ,v i ) is the imaging coordinate of the corresponding i-th marker point, (R t) is the rotation and translation relationship of the marker coordinate system relative to the camera coordinate system, the corresponding position quantity of (R t) is tx, ty, tz, the Euler angle is ax, ay, az, fu, fv, u0, v0 is the internal parameter of the camera, is the imaging coordinate of the marker point recalculated by (X i ,Y i ,Z i ), the relative rotation and translation relationship (R t) and the camera internal parameter.

9. An out-of-plane point marker according to claim 1, wherein, The step 8) is specifically: Let the rotation translation matrix represent the relative pose relationship between the passive end coordinate system and the marker coordinate system, let the rotation translation matrix represent the relative pose relationship between the camera coordinate system and the active end coordinate system, let the rotation translation matrix represent the relative pose of the marker coordinate system relative to the camera coordinate system, let the rotation translation matrix represent the relative pose of the passive end coordinate system relative to the active end coordinate system: wherein and are fixed parameters known by calibration.

Images