A vision-based method for determining the pose of a Stewart mechanism

Abstract

The present invention is applicable to the cross - technical field of mechanism theory and machine vision, and provides a method for determining the pose of a Stewart mechanism based on vision, including steps such as the construction of a binocular vision system, the calculation of the optimal field of view, the calibration of the binocular vision system, the depth calculation, and experimental verification. In the method for determining the pose of a Stewart mechanism based on vision in the present invention, based on the workspace characteristics of the Stewart mechanism, combined with parameters related to the characteristics of the camera such as depth of field, focal length, field of view angle, and optical center distance, the optimal field of view and ArUco markers are established; the Zhang - Zhengyou method is combined with the workspace characteristics of the Stewart mechanism for the calibration of the binocular vision system; the least - squares method is used to calculate the depth information of the object, and the position information of the upper surface of the platform is obtained through the P4P method. The contradiction between the image - processing speed and the positioning accuracy is solved by the method of optimizing the field of view, and the defect of insufficient accuracy of the depth information of the ArUco code in the monocular vision technology is overcome by the binocular vision technology, so as to obtain a high - speed and high - precision visual servo effect.

Description

Technical Field

[0001] The present invention belongs to the cross - technical field of mechanism theory and machine vision, and particularly relates to a method for determining the pose of a Stewart mechanism based on vision. Background Art

[0002] Due to characteristics such as high control precision, small working space, and strong load - carrying capacity, parallel mechanisms have been widely used in fields such as motion simulation, robotics, and space technology. Vision sensors have become one of the most important sensors due to characteristics such as large amount of information, wide application range, and non - contact. Although extensive research has been carried out on the vision - based positioning technology of parallel mechanisms, there are still few cases truly applied in the industrial field. The mutual restriction among accuracy, working range, and speed is one of the important reasons for this problem.

[0003] Vision technology needs to process data of images containing a large number of pixels by a computer, and the current processing ability of the computer is limited. This results in that if vision technology wants to be real - time, only a limited number of pixels can be used to depict the image. And high - precision positioning requires high - resolution images to describe the working area in detail. Under the current technical conditions, without excessive cost increase, if it is required to run in real - time according to the requirements of robot control, the accuracy cannot be very high. On the contrary, in order to ensure accuracy, the speed generally cannot meet the real - time requirement.

[0004] In addition to the number of pixels, another adjustable factor is the field - of - view range. If the field - of - view range is reduced by a narrow - angle camera, higher accuracy can also be obtained with the same number of pixels and image - processing speed. The configuration of the parallel robot and the characteristics of the working space exactly provide this possibility.

[0005] ArUco is an open - source augmented reality library developed and maintained by the V·A team at the University of Cordoba, Spain. It uses binary codes with black borders as identifiers for visual positioning and has a series of functions for augmented reality applications, including visual calibration, generating artificial markers, and pose estimation of cameras.

[0006] However, the current positioning methods of ArUco devices mostly use monocular vision devices, and the acquisition of depth information is obtained by comparing with the characteristics of the original graphics, which has the defect of insufficient accuracy. While a stereo vision system can obtain higher accuracy by adjusting the relative positions of vision devices. Therefore, under the condition that the computing power of the computer allows, this patent explores the advantages and disadvantages of high - precision vision - based positioning methods using stereo vision methods, and in view of the above situation, develops a method for determining the pose of a Stewart mechanism based on vision to overcome the deficiencies in current practical applications. Summary of the Invention

[0007] Aiming at the deficiencies of the existing technology, the purpose of the embodiments of the present invention is to provide a method for determining the pose of a Stewart mechanism based on vision to solve the problems in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A method for determining the pose of a Stewart mechanism based on vision includes the following steps:

[0010] Step (1), construction of a binocular vision system: Through the calculation of the working space of the Stewart mechanism, the size information and coding method of the required ArUco markers are initially determined;

[0011] Step (2), obtaining the optimal field of view: Conducting field of view acquisition, including obtaining the relationship between the marker working space and the field of view, obtaining the position of the optical center, and obtaining the depth of field information;

[0012] Step (3), calibration of the binocular vision system: Calibrating the internal and external parameters and distortion coefficients of the camera using the Zhang Zhengyou method, and obtaining the relative position and angle of the binocular cameras through the rotation and translation of feature points;

[0013] Step (4), depth acquisition: Conducting depth information acquisition and platform positioning to obtain the spatial position information of the target marker corner points, and further obtaining the position information of the upper platform of the Stewart mechanism;

[0014] Step (5), experimentally verifying the working performance of the system through step (4) to verify whether the obtained accuracy and speed are optimal. Otherwise, repeat the steps from step (2) to step (4) until the optimal accuracy and speed are obtained.

[0015] As a further technical solution of the present invention, in step (1), the method for determining the size information and coding method of the ArUco marker includes the following steps:

[0016] Step (11), generation of the device with ArUco markers: Generating markers with different encodings using the ArUco image generator according to the design requirements, printing them out and pasting them onto the marker board;

[0017] Step (12), layout of the marker board with ArUco encoding: Placing the marker board on the upper surface of the moving platform, with its center point coinciding with the center point of the upper platform, the direction parallel to the x-axis of the moving platform, and obtaining the position relationship between the marker corner points and the center of the upper platform through the method of ruler measurement.

[0018] As a further technical solution of the present invention, in step (2), the identified working space refers to the active range of the corner points of the captured object on the identification, and the field of view refers to the overlapping area of the camera pinhole models for three-dimensional positioning of the identification through the images sampled by two cameras.

[0019] As a further technical solution of the present invention, in step (2), the optical center position refers to the optical center position of the camera pinhole model, and the determination steps of the optical center position are as follows:

[0020] Step (21): When building the platform, it is necessary to determine the size of the horizontal distance B of the optical center. When it is above x0 that can cover the entire area, that is, when B ≤ W sh When Within this range, the change of B has no influence on the system characteristics;

[0021] Step (22): When B > W sh When, the image size of the identified working space is reduced by adjusting the distance. Otherwise, the corner points will exceed the working range of the vision system. The ratio of the size of the adjusted new image space to the unadjusted image space is

[0022] As a further technical solution of the present invention, in step (2), the constraint method formula for the identification in the identified working space is:

[0023] Above x0, the center point of the moving platform can reach all positions of the circular area on the working space plane. At this time, the horizontal direction constraint is W > R s +r + B / 2 = R m ;

[0024] Below x0, the working space plane cannot cover all positions, but is a discontinuous arc surface. At this time, the horizontal direction constraint is W > R s cosα + r = R m ;

[0025] Where: the x0 coordinate axis is the coordinate axis taken from the parallel plane of all connecting rods of the Stewart mechanism in the middle position, R s is the radius of each cross-sectional circle in the working space. The maximum radius distance r of the identification is determined by the distance between the captured corner points. W is the width of the field of view section, and R m is the constraint value.

[0026] As a further technical solution of the present invention, in step (2), the method for obtaining depth of field information is to calculate the depth of field before and after the camera. Since the calculation formula for the depth of field is obtained:

[0027]

[0028] Where: ΔL1 and ΔL2 are the front and rear depths of field respectively; L is the object distance, F is the shooting aperture value of the lens, f is the focal length of the lens, and δ is the allowable diameter of the circle of confusion on the image plane.

[0029] As a further technical solution of the present invention, in step (3), the Zhang-Zhengyou method is used for the calibration of the binocular stereo vision system. Through the printed checkerboard and adjusting the orientation of the checkerboard, maximum likelihood estimation is performed to obtain five internal parameters, three external parameters, and two distortion coefficients of a single camera with high estimation accuracy. By translating and rotating the coordinates of the feature points of the images obtained by the two cameras, the rotation and translation matrices between the two cameras are obtained.

[0030] As a further technical solution of the present invention, for the sampling position of the checkerboard image by the Zhang-Zhengyou method, to ensure the calibration accuracy, the image acquisition positions include:

[0031] (31) Determine the focal length f based on the middle position of the connecting rod in the working space of the platform, and collect images at the limit angles of the XYZ axes that the platform can reach respectively;

[0032] (32) Collect images at the upper and lower limit positions of the upper platform in the Z direction;

[0033] (33) To improve the Z-direction accuracy, collect images at different positions in the Z direction.

[0034] As a further technical solution of the present invention, in step (4), the depth information is obtained by calculating the distance of the scene point along the principal optical axis relative to the camera according to the pixel coordinates of the corresponding image points of the scene point in the stereo vision system. The steps of the depth information obtaining method are as follows:

[0035] Step (41): Obtain the high-precision depth information through the forward kinematic solution obtained by the displacement sensor and the three-dimensional space coordinates obtained by the coordinate measuring machine, perform calibration by the least squares method, and obtain the relationship model between the scene stereo depth and the disparity of the given binocular stereo vision system: Where, a and b are constants; its solution is obtained by using the least squares method by collecting the depth information and the corresponding disparity information of some calibration points;

[0036] Step (42): By the method of corresponding definite matching, find the scene disparity in the left and right images, and obtain the accurate information of the marked depth.

[0037] As a further technical solution of the present invention, in step (4), the platform positioning is to obtain the spatial positions of the corner points of the black and white grid marks through binocular vision technology, and obtain the spatial coordinates of the four circular mark points P1, P2, P3, and P4 on the moving platform. Through the rotation angles and translation vectors of the four-point spatial coordinates, the rotation matrix and displacement vector of the platform are calculated.

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] 1. According to the motion characteristics of the moving platform of the parallel robot, this method freely adjusts the spatial position angle, focal length and other field-of-view parameters of the camera, and performs high-precision three-dimensional spatial positioning on the moving platform with the highest frame rate and the least number of pixels;

[0040] 2. This method fully considers the working space, configuration characteristics of the parallel robot and the characteristics of visual servo. The camera can be accurately positioned through a simple mechanical structure, and the error can be corrected to a certain extent according to the effect of visual servo;

[0041] 3. This method combines the ArUco code and the binocular vision positioning method, and improves the accuracy of the ArUco code positioning method based on monocular vision by introducing the lens distance parameter of binocular vision.

[0042] To more clearly elaborate on the structural features and effects of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Description of the Drawings

[0043] Figure 1 It is a schematic diagram of the working scenario of the system in the method for determining the pose of a Stewart mechanism based on vision provided by an embodiment of the present invention.

[0044] Figure 2 It is a flowchart of the method for determining the pose of a Stewart mechanism based on vision provided by an embodiment of the present invention.

[0045] Figure 3 It is a calculation diagram of the field-of-view angle in the method for determining the pose of a Stewart mechanism based on vision provided by an embodiment of the present invention.

[0046] Figure 4 It is a top view of the relationship between the working space of the Stewart mechanism and the binocular vision field-of-view in the method for determining the pose of a Stewart mechanism based on vision provided by an embodiment of the present invention.

[0047] Figure 5 It is a plan view of the relationship between the working space of the Stewart mechanism and the binocular vision field-of-view in the method for determining the pose of a Stewart mechanism based on vision provided by an embodiment of the present invention. Detailed Embodiments

[0048] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0049] The following describes the specific implementation of the present invention in detail in conjunction with specific embodiments.

[0050] As Figure 1 and 2 shown, as a method for determining the pose of a Stewart mechanism based on vision provided in an embodiment of the present invention, it includes the following steps:

[0051] Step (1), construction of a binocular vision system: Through the calculation of the working space of the Stewart mechanism, preliminarily determine the size information and coding method of the required ArUco markers.

[0052] Step (2), obtaining the optimal field of view: Conduct field of view acquisition, including obtaining the relationship between the marker working space and the field of view, obtaining the position of the optical center, and obtaining the depth of field information.

[0053] Step (3), calibration of the binocular vision system: Calibrate the internal and external parameters and distortion coefficients of the camera using the Zhang Zhengyou method, and obtain the relative position and angle of the binocular cameras through the rotation and translation of the feature points.

[0054] Step (4), depth acquisition: Conduct depth information acquisition and platform positioning to obtain the spatial position information of the target marker corner points, and then obtain the position information of the upper platform of the Stewart mechanism.

[0055] Step (5), experimentally verify the working performance of the system through step (4), verify whether the obtained accuracy and speed are optimal, otherwise repeat the steps from step (2) to step (4) until the optimal accuracy and speed are obtained.

[0056] In this embodiment, based on the working space characteristics of the Stewart mechanism, combined with parameters related to the camera characteristics such as depth of field, focal length, field of view angle, and optical center distance, an optimal field of view and ArUco markers are established; the binocular vision system is calibrated using the Zhang Zhengyou method in combination with the working space characteristics of the Stewart mechanism; the least squares method is used to obtain the depth information of the object, and the position information of the upper surface of the platform is obtained through the P4P method. The contradiction between the image processing speed and the positioning accuracy is solved by the method of field of view optimization, and the defect of insufficient accuracy of the depth information of the ArUco code in the monocular vision technology is overcome by the binocular vision technology, so as to obtain a high-speed and high-precision visual servo effect.

[0057] Embodiment 1

[0058] As Figure 1As shown in the figure, as a preferred embodiment of the present invention, the ArUco marker board A is installed on the front of the moving platform D, and the moving platform D objectively reflects the movement process of the parallel robot E. The binocular vision platform B is placed directly above the moving platform D, facing the marker board A. Two cameras C are installed directly in front of the binocular vision platform, facing the marker board A on the moving platform D.

[0059] According to the structural characteristics of the Stewart mechanism, the end effector has characteristics such as a relatively large planar area.

[0060] In this system, the marker board A uses a black and white grid (chessboard) marker board. This marker board is installed on the operating platform at the end effector of the parallel robot. The spatial positions of the corner points of the black and white grid markers are obtained through binocular vision technology, and then the P4P (Perspective-4-Point) algorithm is used to obtain the three-dimensional spatial position information of the operating platform of the parallel mechanism.

[0061] In addition, the marker board A also uses digital coding identification. According to the identification code, the relative position information of the marker on the marker disk can be obtained. In this way, the spatial position information of the end effector of the parallel mechanism can be obtained through the positioning of a few marker corner points. And when the distance between the vision device and the end of the parallel mechanism changes, markers with different areas can be used for positioning.

[0062] Using the above black and white grid marker board with ArUco coding, first, the corner point positions with relatively high accuracy can be obtained by calculating the gradient of the black and white image; second, the relative position between the corner point to be obtained and the moving platform can be obtained through coding, and the corner points of the square grid can be tracked in a smaller field of view to achieve the positioning of the platform; finally, there are multiple groups of the target corner points of the marker, and the target corner points can be adjusted as the distance between the platform and the camera changes.

[0063] When the operating platform is relatively close to the camera, the corner points of the smaller square grid can be positioned, and when the moving platform is relatively far from the camera, the corner points of the larger square grid can be positioned; the binocular vision measurement platform corresponding to the marker board is a mechanical structure used to install and adjust the two cameras; the binocular vision platform can move in six degrees of freedom, including translation in the X, Y, Z (vertical, horizontal, longitudinal) directions and rotation in the X, Y, Z directions.

[0064] In this embodiment, the characteristics of the marker are as follows:

[0065] 1) According to the characteristics of the relatively small working space of the parallel mechanism and the relatively large planar area of the end effector, etc.; an ArUco-coded marker is placed on the end effector of the parallel mechanism, and the spatial position information of the marker feature points is obtained through a binocular stereo vision device, and then this information is converted into the position information of the end effector of the parallel mechanism.

[0066] 2) The identification plate placed at the execution end is an identification plate with ArUco codes. By positioning the spatial positions of the ArUco identification corner points, the graphics placed at the centers of the squares of the identification are arranged according to the generated ArUco codes. Since the stereo vision positioning technology of P4P can obtain the position of the platform only through 4 points in space, the identification by the camera does not need to pass through the entire identification plate, but only needs to obtain the four corner points of a certain identification among them;

[0067] 3) Adjust the position of the corner points of the coded identification plate in combination with the distance. Use the corner points with a relatively close distance to each other at a relatively close distance, and use the corner points with a relatively far distance at a far distance, so as to reduce the relative error during positioning and obtain the best accuracy;

[0068] 4) Combining the characteristics that the z-direction amplitude of the Stewart mechanism is small and it is difficult to obtain the depth information of the vision device, place the vision device at the z-direction position of the parallel mechanism, and use its depth information to correspond to the z-direction movement of the parallel mechanism.

[0069] Embodiment 2

[0070] As a preferred embodiment of the present invention, through the analysis of the working space of the Stewart mechanism, the size of the identification and the coding method adopted are initially determined.

[0071] The working space of the robot is the set of points that a given reference point on the manipulator can reach. The reference point of the Stewart mechanism is selected as the center point of the moving platform, that is, the origin of the coordinate system; after the pose of the upper platform is given, the lengths of the connecting rods, the rotation angles of the joints, and the distances between adjacent rods can all be calculated by the methods discussed above, and then these calculation results are respectively compared with the corresponding allowable values. When any one of the values exceeds its allowable value, the pose of the manipulator at this time cannot be reached, that is, the reference point is outside the working space. If any one of the values is equal to its allowable value, the reference point of this manipulator is located on the boundary of the working space. If all the parameter values are less than the allowable values, the reference point of the manipulator is located inside the working space at this time.

[0072] The working space is obtained by using the search method, that is, a certain space that the upper platform may reach is defined as the search space, and this space is divided into micro sub-spaces by planes parallel to the XY plane, and it is assumed that this sub-space is a cylinder. For each micro sub-space, according to the constraint conditions given above, search for its boundary corresponding to the given pose.

[0073] The identification plate A adopts a black and white grid (checkerboard) identification plate, which is installed on the operating platform at the execution end of the parallel robot. The spatial positions of the corner points of the black and white grid identification are obtained through binocular vision technology, and then the three-dimensional spatial position information of the operating platform of the parallel mechanism is obtained by using the P4P (Perspective-4-Point) algorithm.

[0074] The identification board A also uses ArUco coding identification, and the relative position information of the identification on the identification disk can be obtained; in this way, the spatial position information of the execution end of the parallel mechanism can be obtained through the positioning of the identification corner points; and when the distance between the vision device and the end of the parallel mechanism changes, identifications with different areas can be used for positioning.

[0075] By using the above-mentioned black and white grid identification board with coding, first, the corner point positions with relatively high accuracy can be obtained by calculating the gradient of the black and white image; second, the relative position between the required corner points and the moving platform can be obtained through coding, and the positioning of the platform can be realized by tracking the corner points of the square grid in a smaller field of view; finally, the target corner points of the identification are variable. When the operating platform is relatively close to the camera, the corner points of the smaller square grid can be positioned, and when the operating platform is relatively far from the camera, the corner points of the larger square grid can be positioned; through the identification of the coding information, the relative position relationship between the corner points and the center point of the platform can be obtained.

[0076] Embodiment 3

[0077] As a preferred embodiment of the present invention, the field of view is the overlapping area of the camera pinhole models that can perform three-dimensional positioning of the identification through the images sampled by two cameras. The goal of obtaining the field of view is to ensure that this area can cover all possible positions of the identification corner points placed on the platform as much as possible.

[0078] Embodiment 4

[0079] As a preferred embodiment of the present invention, the working space of the identification is defined as the activity range of the captured object corner points on the identification; the working space of the Stewart mechanism is all the positions that the center point of the upper platform can reach; this system uses narrow-angle cameras to position the identification as accurately as possible with the least number of pixels. Therefore, the working space of the vision system is the calculation formula of the intersection of the allowable working areas of the left camera and the right camera.

[0080] To realize the positioning of the moving platform through the corner point positions of the identification, it is necessary to ensure the working of the identification. With the current binocular vision positioning method, if A is defined as the allowable area of the left camera and B is the allowable area of the right camera, the camera can meet the working requirements within the allowable area. Then the available area is the intersection A∩B of the two areas.

[0081] As Figure 3 shown, for the method of obtaining the field of view, the size of the field of view should be determined according to the distance from the lens to the object to be captured, the focal length of the lens, and the required imaging size:

[0082]

[0083]

[0084] Among them, H is the field of view height, W is the field of view width, L is the distance from the lens to the object to be captured (viewing distance), unit: m; f is the focal length; a is the image field height (vertical dimension); b is the image field width (horizontal dimension), unit: mm.

[0085] Example 5

[0086] As Figure 4 shown, A is the plane of the pinhole model of the left camera, Ol is the optical center of the pinhole model of the left camera, and C is the workspace of the Stewart mechanism. B is the plane of the pinhole model of the right camera, and Or is the optical center of the pinhole model of the right camera.

[0087] As Figure 4 shown, the workspace of the Stewart mechanism is an area close to a hexagon. Therefore, the cameras are arranged parallel to the y-axis of the workspace. This can ensure obtaining the largest workspace.

[0088] For this system, the identified workspace should be, on the basis of the workspace of the Stewart mechanism, the range of position changes of all rotation angles of the identified corner points on the points of this workspace. It is very difficult to accurately describe this space completely by the analytical method. However, the size of the identification is a relatively small value relative to the field of view range. Therefore, it is proposed to use the distance from the axis of the workspace to this point plus the maximum radius distance r of the identification as a constraint to ensure, as much as possible, that the identified corner points are within the working area of the vision system.

[0089] As Figure 5 shown, Figure 5 is a plan view of the relationship between the field of view and the workspace of the Stewart mechanism. The field of view angle of the camera is determined through this figure. B is the optical center of the pinhole model of the camera. The constraint in the vertical direction is relatively simple:

[0090] H>R s +r;

[0091] Among them, H is the height of the field of view section.

[0092] Figure 4 The x0 coordinate axis in it is the coordinate axis taken from the parallel plane when all the connecting rods of the Stewart mechanism are in the middle position. Above x0, the center point of the moving platform can reach all positions of the circular area of the workspace plane. The horizontal direction constraint at this time is

[0093] W>R s +r+B / 2=R m ;

[0094] Among them, R sis the radius on each cross-sectional circle of the working space. The marked maximum radius distance r is determined by the distance between the captured corner points. W is the width of the field-of-view cutting plane, and R m is the constraint value.

[0095] Below x0, the working space plane cannot cover all positions but is a discontinuous arc surface. At this time, the horizontal direction constraint is:

[0096] W > R s cosα + r = R m .

[0097] Embodiment 6

[0098] As Figure 5 shown, in the plan view of the relationship between the field of view and the working space of the Stewart mechanism, the following relationships exist:

[0099]

[0100] Among them, d is the parallax of the two images, z is the depth of the target object, and B is the distance between the optical centers of the two cameras.

[0101] It can be seen from the above formula that there is a proportional relationship between the depth information z of the binocular vision system and the optical center distance. Also, because

[0102]

[0103]

[0104] Among them, u and v are the horizontal and vertical image positions of the target point respectively, and x and y are the spatial positions of the target point respectively.

[0105] The above two formulas show that the xy information of the system is directly obtained from the depth information z. From this, it can be seen that increasing the optical center position is beneficial to improving the positioning accuracy. For this system, since the target object is the marked working space, when increasing the optical center position, the two cameras need to be symmetrically moved along the z-axis of the marked working space. The relationship between the number of pixels and the positioning accuracy of the vision system is as follows:

[0106] 1) Above x0 where the entire area can be covered, that is, when B ≤ W sh at this time,

[0107]

[0108] Within this range, the change of B has no effect on the system characteristics. Therefore, if possible, B can be increased as much as possible.

[0109] 2) When B > W shWhen this happens, it is necessary to reduce the image size of the identification working space by adjusting the distance. Otherwise, the corner points will exceed the working range of the vision system. Then, the ratio of the size of the adjusted new image space to the unadjusted image space is:

[0110]

[0111] When building the platform, the size of B needs to be determined according to the above two formulas. For larger Stewart mechanisms, appropriate reduction methods can be used to improve the accuracy. For micro and small Stewart mechanisms, they should be kept outside the range as much as possible.

[0112] Example 7

[0113] Before and after the focus, the light begins to converge and diverge, and the image of the point becomes blurred, forming an enlarged circle, which is called the circle of confusion. There is an allowable circle of confusion before and after the focus. The distance between these two circles of confusion is called the depth of field, that is: in front of and behind the subject to be photographed (the focus point), there is still a clear range for its image, which is the depth of field. In this system, to ensure the accuracy of the binocular vision positioning system, it is necessary to ensure that the identification working space is within the range permitted by the depth of field.

[0114] Quantitatively speaking, as shown in the figure, ΔL1 and ΔL2 are the front and rear depths of field respectively, L is the object distance, F is the shooting aperture value of the lens, f is the focal length of the lens, and δ is the diameter of the allowable circle of confusion on the image plane. Then the calculation formulas for the front and rear depths of field of the camera are:

[0115]

[0116]

[0117] From this, the size of the depth of field can be obtained as:

[0118]

[0119] According to the calculation results of the above formula, the working space can be ensured to be within the area where the depth of field is satisfied to ensure the accuracy of the image corner point information.

[0120] Example 8

[0121] As a preferred embodiment of the present invention, the camera calibration of this system includes the internal parameter calibration of the camera and the stereo vision calibration, etc.

[0122] The calibration of the camera adopts Zhang Zhengyou's method, which can obtain the internal and external parameters and distortion coefficients of the camera, and obtain the relative position and rotation of the two cameras through the translation and rotation of the image feature points. Different from general visual calibration, this method requires adjusting the position and angle of the camera and needs to cover the entire movement space of the checkerboard, rather than all areas of the binocular vision system. Therefore, the checkerboard image sampling in this method must include the following three positions:

[0123] 1. Determine the focal length f based on the middle position of the connecting rod in the working space of the platform, and take images at the limit angles of the xyz axes that the platform may reach respectively;

[0124] 2. The extreme positions of the upper platform in the z direction upward and downward;

[0125] 3. To improve the z-direction accuracy, add images at different z positions.

[0126] Example 9

[0127] As a preferred embodiment of the present invention, the depth information is obtained by calculating the distance of the scene point along the principal optical axis direction relative to a certain camera according to the pixel coordinates of the corresponding image point of the scene point in the stereo vision system, which is an important content of stereo vision. And three-dimensional reconstruction is to further calculate the three-dimensional coordinates of the scene point in a certain camera coordinate system on the basis of depth recovery.

[0128] Since there is no method to directly obtain the scene depth information and perform three-dimensional reconstruction, the existing methods all use indirect methods to obtain the scene depth information and perform three-dimensional reconstruction, which are mainly divided into two categories:

[0129] The first category is to convert the image in the non-parallel axis stereo vision system into the image in the standard parallel axis stereo vision system through the method of image correction, and then call the depth information acquisition and three-dimensional reconstruction of the standard parallel axis stereo vision system;

[0130] The second category is to extend the projection lines of the scene point in the two cameras in the reverse direction, and then find the point closest to the two projection lines in the scene through the optimization method for depth recovery and three-dimensional reconstruction. The second type of method usually requires initial value estimation first and then iterative optimization, which is time-consuming and easy to converge to the local optimal solution, so it is rarely used in practical applications.

[0131] The present invention adopts the first type of method. The present invention obtains the accurate spatial position information of a certain number of identification points through an external sensor; obtains high-precision depth information through the forward kinematic solution obtained by a displacement sensor, the three-dimensional spatial coordinates obtained by a coordinate measuring machine, etc., performs calibration based on the least squares method, and obtains the relationship model between the stereo depth and the disparity of the given binocular stereo vision system scene. Finally, through the method of corresponding definite matching, the scene disparity in the left and right images is obtained, so as to accurately obtain the stereo depth of the scene.

[0132] Embodiment 10

[0133] As a preferred embodiment of the present invention, the spatial positions of the corner points of the black and white grid markings are obtained through binocular vision technology, and the spatial coordinates of the four circular marking points P1, P2, P3, and P4 on the moving platform can be obtained. The pose of the upper platform can be obtained from the coordinates of these four points by the following method:

[0134] According to the motion characteristics of a rigid body: the rotation angles and translation vectors of these four marking points are the same. The rotation matrix and translation vector of the marking points in the fixed coordinate system (o-xyz) and the moving coordinate system (o d -x d y d z d ) for the fixed coordinate system.

[0135]

[0136] Then

[0137] P i = RP id + t

[0138] In the formula, R and t are respectively the rotation matrix and translation vector of the moving coordinate system relative to the fixed coordinate system.

[0139]

[0140] Among them, s = sin(), c = cos().

[0141] Subtracting the formulas in the above formula pairwise can solve the attitude angles θ, ψ, and then the rotation matrix R of the moving platform can be calculated. On this basis, the translation vector is expressed as

[0142] From this, t can be solved.

[0143] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A vision-based method for determining the pose of a Stewart mechanism, characterized in that, It includes the following steps: Step (1), construction of the binocular vision system: Through the calculation of the working space of the Stewart mechanism, preliminarily determine the size information and coding method of the required ArUco markers; Step (2), obtaining the optimal field of view: Conduct field of view acquisition, including obtaining the relationship between the marker working space and the field of view, obtaining the position of the optical center, and obtaining the depth of field information; The position of the optical center refers to the position of the optical center of the camera pinhole model. The steps to determine the position of the optical center are as follows: Step (21), when building the platform, it is necessary to determine the size of the horizontal distance B of the optical center. When it is above x0 that can cover the entire area, that is, when When , within this range, the change of B has no effect on the system characteristics; Step (22), when occurs, reduce the image size of the identification working space by adjusting the distance. Otherwise, the corner points will exceed the working range of the vision system. The ratio of the adjusted new image space to the unadjusted image space is ; The formula for the constraint method of the marker in the marker working space is: Above x0, the center point of the moving platform can reach all positions in the circular area of the working space plane, and the horizontal constraint at this time is ; Below x0, the workspace plane cannot cover all positions but is a discontinuous arc surface, and the horizontal direction constraint at this time is ; wherein: the x0 coordinate axis is the coordinate axis taken from the parallel plane when all the connecting rods of the Stewart mechanism are in the middle position, is the radius on each cross-sectional circle of the working space. It is indicated that the maximum radius distance r is determined by the distance between the captured corner points, and W is the width of the field-of-view section plane, is the constraint value; The method for obtaining the depth of field information is to calculate the depth of field before and after the camera. Since the calculation formula for the depth of field is obtained: ； Wherein: and are the front and rear depths of field respectively; L is the object distance, F is the shooting aperture value of the lens, f is the focal length of the lens, is the allowable diameter of the circle of confusion on the image plane; Step (3), calibration of the binocular vision system: Calibrate the internal and external parameters and distortion coefficients of the camera using the Zhang Zhengyou method, and obtain the relative position and angle of the binocular cameras through the rotation and translation of feature points; Step (4), depth calculation: Conduct depth information calculation and platform positioning to obtain the spatial position information of the target marker corner points, and then obtain the position information of the upper platform of the Stewart mechanism; Step (5), experimentally verify the working performance of the system through Step (4), verify whether the obtained accuracy and speed are optimal. Otherwise, repeat the steps from Step (2) to Step (4) until the optimal accuracy and speed are obtained.

2. The method for determining the pose of the Stewart mechanism based on vision according to claim 1, wherein In Step (1), the method for determining the size information and coding method of the ArUco marker includes the following steps: Step (11), generation of the device with ArUco markers: Generate markers with different encodings using the ArUco image generator according to the design requirements, print them out and paste them on the marker board; Step (12), layout of the marker board with ArUco encoding: Place the marker board on the upper surface of the moving platform, and its center point coincides with the center point of the upper platform. The direction is parallel to the x-axis of the moving platform, and the position relationship between the marker corner points and the center of the upper platform is obtained by the method of ruler measurement.

3. The method for determining the pose of the Stewart mechanism based on vision according to claim 1, wherein In Step (2), the marker working space refers to the active range of the captured object corner points on the marker, and the field of view refers to the overlapping area of the camera pinhole models for three-dimensional positioning of the marker through the images sampled by two cameras.

4. The method for determining the pose of the Stewart mechanism based on vision according to claim 1, wherein In Step (3), the Zhang Zhengyou method is used for the calibration of the binocular stereo vision system. Through the printed checkerboard and adjusting the orientation of the checkerboard, maximum likelihood estimation is performed to obtain five internal parameters, three external parameters, and two distortion coefficients of a single camera with high estimation accuracy; through the coordinate translation and rotation of the feature points of the images obtained by the two cameras, the rotation and translation matrices between the two cameras are obtained.

5. The method for determining the pose of the Stewart mechanism based on vision according to claim 4, characterized in that, Sampling of the checkerboard image is carried out by the Zhang Zhengyou method. To ensure the calibration accuracy, the image acquisition positions include: (31), Determine the focal length f based on the middle position of the connecting rod in the platform working space, and collect images at the limit angles of the XYZ axes that the platform can reach respectively; (32), Collect images at the upper and lower limit positions of the upper platform in the Z direction; (33), To improve the accuracy in the Z direction, collect images at different positions in the Z direction.

6. The method for determining the pose of the Stewart mechanism based on vision according to claim 1, characterized in that, In step (4), the depth information is obtained based on the pixel coordinates of the corresponding image points of the scene points in the stereo vision system, and the distance of the scene points relative to the camera along the principal optical axis direction is calculated. The steps of the depth information obtaining method are as follows: Step (41): Calibrate using the least squares method with the high-precision depth information obtained from the forward kinematic solution acquired by the displacement sensor and the three-dimensional spatial coordinates obtained by the coordinate measuring machine, and obtain the relationship model between the stereo depth and the disparity of the given binocular stereo vision system scene: , where a and b are constants; its solution is obtained by using the least squares method with the depth information of some calibration points and their corresponding disparity information collected. Step (42): By using the corresponding fixed matching method, obtain the scene disparity in the left and right images, and obtain the accurate information indicating the depth.

7. The method for determining the pose of the Stewart mechanism based on vision according to claim 1, characterized in that, In step (4), the platform positioning is to obtain the spatial positions of the corner points of the black and white grid markings through binocular vision technology, and the four circular marker points on the moving platform can be obtained , , and spatial coordinates. Through the rotation angles and translation vectors of the four-point spatial coordinates, the rotation matrix and displacement vector of the platform are calculated.

Images