Monocular vision attitude estimation method and system based on EPnP algorithm
By combining EPnP algorithm with monocular camera, parallel perspective projection model and iterative optimization algorithm are used to realize accurate tracking of the robotic arm under monocular camera, solving the problem of error accumulation and calculation volume caused by multiple Marker markers, reducing the Cost and error, suitable for orthopedic surgery and other scenarios.
Patent Information
- Application Number
- CN202411019817.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-06-18
- Filing Date
- 2024-07-29
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-07-29
AI Technical Summary
In the existing orthopedic surgical robot navigation system, optical navigation positioning and tracking technology requires multiple Marker markers, resulting in large error accumulation, large calculation amount and high cost, and it is difficult to achieve accurate robotic arm tracking in the case of a monocular camera.
Using a monocular visual pose estimation method based on EPnP algorithm, the monocular camera is combined with a non-iteration algorithm, and the monocular camera is fixed to the end of the robotic arm, and the bits of the spatial reference marker are captured in real time through parallel perspective projection model and iterative optimization algorithm. Position, calculate the target position at the end of the robot arm.
It reduces registration error, improves accuracy, reduces calculation amount and cost, and realizes precise tracking of robotic arms in the case of a monocular camera, and is suitable for dynamic scenarios such as orthopedic surgery.
Smart Images

Figure CN119090958A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of visual positioning technology, and in particular to a monocular visual posture estimation method and system based on an EPnP algorithm. Background Art
[0002] The development of digital medical technology has promoted the progress of orthopedic surgical robot navigation technology. The existing orthopedic surgical robot navigation system mostly uses NDI optical navigation positioning and tracking technology to monitor the movement position of patients during surgery. It first uses multiple markers to represent the spatial posture information of surgical instruments, patient pain points and positioning scales. Then NDI unifies the above-mentioned markers into the same coordinate system, and uses the spatial data provided by the coordinate system to determine the starting and ending points of the surgical path planning, the robot positioning position and real-time tracking of the patient's movement during surgery. Therefore, the accuracy of the spatial posture information of each marker directly affects the surgical path planning and the accuracy of the robot. The more markers there are, the greater the accumulated error and the greater the amount of calculation. To this end, the present invention proposes a monocular visual posture estimation method based on the EPnP algorithm, which combines the EPnP non-iterative algorithm with a monocular camera, and uses a monocular camera to realize the tracking of the patient's movement process by the robotic arm with only one marker;
[0003] The existing optical navigation positioning and tracking technology uses NDI to track and display the positions of multiple Marker objects in real time. There are problems such as a large number of Marker markers, large cumulative errors, large amount of calculation, and high cost. The present invention proposes a monocular visual posture estimation method based on the EPnP algorithm, which combines the EPnP non-iterative algorithm with a monocular camera, and in the case of only one Marker marker, solves the shortcomings of the NDI camera itself that needs to match multiple Markers for intraoperative positioning, such as error accumulation and large amount of calculation. This method reduces the registration error, improves the registration accuracy, reduces the amount of calculation and reduces the cost. The application of the present invention can at least solve some of the problems of the prior art. Summary of the invention
[0004] The embodiments of the present invention provide a monocular vision posture estimation method and system based on the EPnP algorithm, which can at least solve some problems in the prior art.
[0005] According to a first aspect of the embodiments of the present invention,
[0006] Provides a monocular visual pose estimation method based on the EPnP algorithm, including:
[0007] Obtain the internal parameters of the monocular camera through camera calibration; determine the world coordinates of the feature points on the spatial reference markers arranged in the field of view of the monocular camera, and the pixel coordinates of the feature points on the camera image; express the world coordinates of the feature points as a linear combination of four non-coplanar virtual control points, establish a camera perspective projection model, form a linear equation group about the camera coordinates of the virtual control points, and determine the camera coordinates corresponding to the virtual control points in the camera coordinate system; wherein the monocular camera is fixed to the end of the robotic arm, and the optical axis of the monocular camera coincides with the center point of the end of the robotic arm;
[0008] A parallel perspective projection model is introduced, and an initial estimation of the camera pose is obtained through an iterative parallel perspective solution process. Based on the initial estimation of the camera pose, the coordinates of the control points in the camera coordinate system are determined, and the initial values of the homogeneous barycentric coordinate coefficients of the control points are calculated. With the optimization goal of reducing the distance difference between the control points in the camera coordinate system and the world coordinate system, the initial values of the homogeneous barycentric coordinate coefficients of the control points are used to solve through an iterative optimization algorithm to obtain the precise pose of the spatial reference marker relative to the camera.
[0009] The initial posture of the spatial reference marker relative to the end of the robotic arm is used as the reference posture; when the spatial reference marker moves, the camera captures the new posture of the spatial reference marker in real time, calculates the current posture of the end of the robotic arm, combines the reference posture and the current posture of the end of the robotic arm, calculates the target posture of the end of the robotic arm in real time, and moves the robotic arm according to the target position.
[0010] In an optional embodiment,
[0011] Obtaining the internal parameters of a monocular camera through camera calibration; determining the world coordinates of a feature point on a spatial reference marker arranged within the field of view of the monocular camera, and the pixel coordinates of the feature point on the camera image; expressing the world coordinates of the feature point as a linear combination of four non-coplanar virtual control points, establishing a camera perspective projection model, forming a linear equation group about the camera coordinates of the virtual control point, and determining the camera coordinates corresponding to the virtual control point in the camera coordinate system includes:
[0012] The monocular camera is calibrated by using the Zhang Zhengyou calibration method to obtain an intrinsic parameter matrix of the monocular camera, including the focal length and the optical center coordinates of the camera;
[0013] The checkerboard in the camera field of view is used as a plane reference marker, the pixel coordinates of the corner points on the plane reference marker are detected by a feature extraction algorithm, n corner points are selected as feature points, and based on the size of the checkerboard and the arrangement of the corner points, the world coordinates of the feature points in the checkerboard coordinate system corresponding to the checkerboard are calculated;
[0014] In the chessboard coordinate system, four non-coplanar points are selected as virtual control points, and a transformation relationship from the chessboard coordinate system to the virtual control point coordinate system corresponding to the virtual control point is established. The world coordinates of the feature point are expressed by coordinate transformation and homogeneous barycentric coordinates to obtain the coordinate representation of the feature point in the virtual control point coordinate system;
[0015] Based on the intrinsic parameter matrix, combined with the world coordinates of the feature points and the coordinate representation of the feature points in the virtual control point coordinate system, a camera imaging perspective projection model equation group is established;
[0016] In the camera imaging perspective projection model equation group, the unknown coordinate representation of the virtual control point in the camera coordinate system is introduced, and the camera imaging perspective projection model equation group is converted into a linear equation group of the control points in the camera coordinate system. The linear equation group is solved by the singular value decomposition numerical optimization method to obtain the coordinates of the four virtual control points in the camera coordinate system.
[0017] In an optional embodiment,
[0018] Also includes:
[0019] The formula of the intrinsic parameter matrix of the monocular camera is as follows:
[0020]
[0021] Among them, K represents the internal parameter matrix, f x Indicates the focal length of the camera in the x-axis direction, f y represents the focal length of the camera in the y-axis direction, u0 represents the horizontal coordinate of the optical center of the camera, and v0 represents the vertical coordinate of the optical center of the camera;
[0022] The camera imaging perspective projection model equation group has the following formula:
[0023]
[0024] Among them, j represents the virtual control point index, i represents the feature point index, α ij represents the homogeneous centroid coordinates of the i-th feature point in the virtual control point coordinate system, x j c represents the x component of the jth virtual control point in the camera coordinate system, y j c represents the y component of the jth virtual control point in the camera coordinate system, z j c represents the z component of the jth virtual control point in the camera coordinate system, u i represents the horizontal coordinate of the i-th feature point on the image plane, v iRepresents the vertical coordinate of the i-th feature point on the image plane.
[0025] In an optional embodiment,
[0026] The parallel perspective projection model is introduced, and the initial estimation of the camera pose is obtained through an iterative parallel perspective solution process. Based on the initial estimation of the camera pose, the coordinates of the control points in the camera coordinate system are determined, and the initial values of the homogeneous barycentric coordinate coefficients of the control points are calculated. With the optimization goal of reducing the distance difference between the control points in the camera coordinate system and the world coordinate system, the initial values of the homogeneous barycentric coordinate coefficients of the control points are used to solve through an iterative optimization algorithm to obtain the precise pose of the spatial reference marker relative to the camera, including:
[0027] By establishing a parallel perspective projection model, the equation group of the camera imaging perspective projection model is reconstructed into a linear equation group about the control point coordinates and the homogeneous barycentric coordinate coefficients;
[0028] By iteratively solving the equation group of the parallel perspective projection model, an initial estimate of the camera pose is obtained, and based on the initial estimate, the initial coordinates of the control point in the camera coordinate system are calculated, and based on the initial coordinates of the control point in the camera coordinate system, the homogeneous barycentric coordinate equation group is solved by the least squares method to obtain the initial values of the homogeneous barycentric coordinate coefficients of the control point;
[0029] Based on the camera pose and the initial values of the homogeneous centroid coordinate coefficients of the control points, an objective function is constructed based on the distance difference between the control points in the camera coordinate system and the world coordinate system to minimize the objective function, thereby obtaining the optimal camera pose estimation and the precise pose of the spatial reference marker relative to the camera.
[0030] In an optional embodiment,
[0031] The objective function includes:
[0032] The formula is as follows:
[0033]
[0034] Among them, β represents the optimization target, j and l represent the virtual control point index, and C j c represents the coordinates of the jth virtual control point in the camera coordinate system, C l c represents the coordinates of the lth virtual control point in the camera coordinate system, C j w represents the coordinates of the jth virtual control point in the world coordinate system, C l w Represents the coordinates of the lth virtual control point in the world coordinate system.
[0035] In an optional embodiment,
[0036] The initial position of the spatial reference marker relative to the end of the robotic arm is used as a reference position; when the spatial reference marker moves, the camera captures the new position of the spatial reference marker in real time, calculates the current position of the end of the robotic arm, combines the reference position and the current position of the end of the robotic arm, calculates the target position of the end of the robotic arm in real time, and moves the robotic arm according to the target position, including:
[0037] Placing a spatial reference marker in a workspace corresponding to the end of the robotic arm, wherein the spatial reference marker is composed of a plurality of control points with clear coordinates;
[0038] Based on the monocular camera completely photographing the spatial reference marker as a reference, when the robotic arm is located at an initial position, the initial joint angle of the robotic arm is recorded, the initial posture of the spatial reference marker relative to the monocular camera is calculated, and the initial posture is used as the reference posture;
[0039] The monocular camera captures an image corresponding to the spatial reference marker in real time, and calculates a current position of the spatial reference marker relative to the monocular camera;
[0040] According to the reference posture and the current posture, the posture change of the spatial reference marker is calculated, and the joint angle change is calculated according to the posture change and the initial joint angle, and the joint angle change is compensated to the current joint angle of the robotic arm to obtain the target joint angle of the end of the robotic arm, and the robotic arm is moved based on the target joint angle.
[0041] In an optional embodiment,
[0042] The monocular camera taking real-time images corresponding to the spatial reference markers includes:
[0043] Transform the matrix into a 4x4 homogeneous matrix To represent the relative position between coordinate systems, the homogeneous transformation matrix The meaning is the position of coordinate system j relative to coordinate system i.
[0044] At the initial stage, the relative position reference between the end of the robotic arm and the spatial reference marker is as follows:
[0045]
[0046] in, Represents the constant homogeneous transformation matrix between the camera coordinate system and the target pose coordinate system, represents the initial homogeneous transformation matrix between the camera coordinate system and the robot base coordinate system, Represents the initial homogeneous transformation matrix between the robot base coordinate system and the target pose coordinate system;
[0047] After the spatial reference marker moves, the relative position between the spatial reference marker and the end of the robotic arm remains unchanged, satisfying the following formula:
[0048]
[0049] in, Represents the homogeneous transformation matrix of the camera coordinate system relative to the robot base coordinate system, Represents the homogeneous transformation matrix of the target pose coordinate system relative to the robot base coordinate system;
[0050] At the same time, the position and posture of the end of the robotic arm in the base coordinate system of the robotic arm satisfies the following formula:
[0051]
[0052] in, Represents the homogeneous transformation matrix of the robot end coordinate system relative to the robot base coordinate system, A homogeneous transformation matrix representing the transformation matrix of the camera coordinate system relative to the robot end coordinate system;
[0053] The spatial reference marker moves to a new position, and based on the base coordinate system, the new pose formula is as follows:
[0054]
[0055] in, Represents the homogeneous transformation matrix of the new position of the target pose relative to the camera coordinate system, Represents the homogeneous transformation matrix of the new position of the target pose relative to the coordinate system of the robot base, Represents the homogeneous transformation matrix of the new position of the target pose relative to the camera coordinate system.
[0056] According to a second aspect of the embodiments of the present invention,
[0057] Provides a monocular visual pose estimation system based on the EPnP algorithm, including:
[0058] The first unit is used to obtain the internal parameters of the monocular camera through camera calibration; determine the world coordinates of the feature points on the spatial reference markers arranged in the field of view of the monocular camera, and the pixel coordinates of the feature points on the camera image; express the world coordinates of the feature points as a linear combination of four non-coplanar virtual control points, establish a camera perspective projection model, form a linear equation group about the camera coordinates of the virtual control points, and determine the camera coordinates corresponding to the virtual control points in the camera coordinate system; wherein the monocular camera is fixed to the end of the robotic arm, and the optical axis of the monocular camera coincides with the center point of the end of the robotic arm;
[0059] The second unit is used to introduce a parallel perspective projection model, obtain an initial estimate of the camera pose through an iterative parallel perspective solution process, determine the coordinates of the control points in the camera coordinate system based on the initial estimate of the camera pose, and calculate the initial values of the homogeneous barycentric coordinate coefficients of the control points; with the optimization goal of reducing the distance difference between the control points in the camera coordinate system and the world coordinate system, use the initial values of the homogeneous barycentric coordinate coefficients of the control points to solve through an iterative optimization algorithm to obtain the precise pose of the spatial reference marker relative to the camera;
[0060] The third unit is used to use the initial posture of the spatial reference marker relative to the end of the robotic arm as a reference posture; when the spatial reference marker moves, the camera captures the new posture of the spatial reference marker in real time, calculates the current posture of the end of the robotic arm, combines the reference posture and the current posture of the end of the robotic arm, calculates the target posture of the end of the robotic arm in real time, and moves the robotic arm according to the target position.
[0061] According to a third aspect of the embodiments of the present invention,
[0062] An electronic device is provided, comprising:
[0063] processor;
[0064] a memory for storing processor-executable instructions;
[0065] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0066] According to a fourth aspect of the embodiments of the present invention,
[0067] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0068] In the embodiments of the present invention, the precise pose of the spatial reference marker relative to the camera can be obtained through an iterative optimization algorithm, thereby improving the accuracy of pose estimation; by introducing a parallel perspective projection model and an iterative parallel perspective solution process, an initial estimate of the camera pose can be quickly obtained without an initial pose, thereby providing an effective starting point for further iterative optimization, thereby improving not only the convergence speed of the algorithm but also the accuracy of the final pose estimation; by fixing a monocular camera at the end of a robotic arm and capturing the new pose of the spatial reference marker in real time, the method can calculate and adjust the target pose of the end of the robotic arm in real time to adapt to the position change of the spatial reference marker; the real-time pose tracking and adjustment mechanism provides strong support for the precise operation and automated operation of the robotic arm; since it does not rely on a specific shape or size of the spatial reference marker, it only needs to arrange identifiable feature points within the camera's field of view, thereby having high flexibility. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] Figure 1 A schematic diagram of a monocular visual posture estimation method based on the EPnP algorithm according to an embodiment of the present invention;
[0070] Figure 2 The figure is a schematic diagram of the structure of a monocular vision posture estimation system based on the EPnP algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION
[0071] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0072] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0073] Figure 1 FIG. 1 is a flow chart of a monocular visual posture estimation method based on the EPnP algorithm according to an embodiment of the present invention. Figure 1 As shown, the method includes:
[0074] S101. Obtain the internal parameters of the monocular camera through camera calibration; determine the world coordinates of the feature points on the spatial reference markers arranged in the field of view of the monocular camera, and the pixel coordinates of the feature points on the camera image; express the world coordinates of the feature points as a linear combination of four non-coplanar virtual control points, establish a camera perspective projection model, form a linear equation group about the camera coordinates of the virtual control points, and determine the camera coordinates corresponding to the virtual control points in the camera coordinate system; wherein the monocular camera is fixed to the end of the robotic arm, and the optical axis of the monocular camera coincides with the center point of the end of the robotic arm;
[0075] S102. Introduce a parallel perspective projection model, obtain an initial estimate of the camera pose through an iterative parallel perspective solution process, determine the coordinates of the control points in the camera coordinate system based on the initial estimate of the camera pose, and calculate the initial values of the homogeneous barycentric coordinate coefficients of the control points; take reducing the distance difference between the control points in the camera coordinate system and the world coordinate system as the optimization goal, use the initial values of the homogeneous barycentric coordinate coefficients of the control points, and solve through an iterative optimization algorithm to obtain the precise pose of the spatial reference marker relative to the camera;
[0076] S103. Take the initial posture of the spatial reference marker relative to the end of the robotic arm as the reference posture; when the spatial reference marker moves, the camera captures the new posture of the spatial reference marker in real time, calculates the current posture of the end of the robotic arm, combines the reference posture and the current posture of the end of the robotic arm, calculates the target posture of the end of the robotic arm in real time, and moves the robotic arm according to the target position.
[0077] In an optional embodiment,
[0078] Obtaining the internal parameters of a monocular camera through camera calibration; determining the world coordinates of a feature point on a spatial reference marker arranged within the field of view of the monocular camera, and the pixel coordinates of the feature point on the camera image; expressing the world coordinates of the feature point as a linear combination of four non-coplanar virtual control points, establishing a camera perspective projection model, forming a linear equation group about the camera coordinates of the virtual control point, and determining the camera coordinates corresponding to the virtual control point in the camera coordinate system includes:
[0079] The monocular camera is calibrated by using the Zhang Zhengyou calibration method to obtain an intrinsic parameter matrix of the monocular camera, including the focal length and the optical center coordinates of the camera;
[0080] The checkerboard in the camera field of view is used as a plane reference marker, the pixel coordinates of the corner points on the plane reference marker are detected by a feature extraction algorithm, n corner points are selected as feature points, and based on the size of the checkerboard and the arrangement of the corner points, the world coordinates of the feature points in the checkerboard coordinate system corresponding to the checkerboard are calculated;
[0081] In the chessboard coordinate system, four non-coplanar points are selected as virtual control points, and a transformation relationship from the chessboard coordinate system to the virtual control point coordinate system corresponding to the virtual control point is established. The world coordinates of the feature point are expressed by coordinate transformation and homogeneous barycentric coordinates to obtain the coordinate representation of the feature point in the virtual control point coordinate system;
[0082] Based on the intrinsic parameter matrix, combined with the world coordinates of the feature points and the coordinate representation of the feature points in the virtual control point coordinate system, a camera imaging perspective projection model equation group is established;
[0083] In the camera imaging perspective projection model equation group, the unknown coordinate representation of the virtual control point in the camera coordinate system is introduced, and the camera imaging perspective projection model equation group is converted into a linear equation group of the control points in the camera coordinate system. The linear equation group is solved by the singular value decomposition numerical optimization method to obtain the coordinates of the four virtual control points in the camera coordinate system.
[0084] In an optional embodiment,
[0085] Also includes:
[0086] The formula of the intrinsic parameter matrix of the monocular camera is as follows:
[0087]
[0088] Among them, K represents the internal parameter matrix, f x Indicates the focal length of the camera in the x-axis direction, f y represents the focal length of the camera in the y-axis direction, u0 represents the horizontal coordinate of the optical center of the camera, and v0 represents the vertical coordinate of the optical center of the camera;
[0089] The camera imaging perspective projection model equation group has the following formula:
[0090]
[0091] Among them, j represents the virtual control point index, i represents the feature point index, α ij represents the homogeneous centroid coordinates of the i-th feature point in the virtual control point coordinate system, x j c represents the x component of the jth virtual control point in the camera coordinate system, y j c represents the y component of the jth virtual control point in the camera coordinate system, z j c represents the z component of the jth virtual control point in the camera coordinate system, u i represents the horizontal coordinate of the i-th feature point on the image plane, v iRepresents the vertical coordinate of the i-th feature point on the image plane.
[0092] In an optional embodiment,
[0093] The parallel perspective projection model is introduced, and the initial estimation of the camera pose is obtained through an iterative parallel perspective solution process. Based on the initial estimation of the camera pose, the coordinates of the control points in the camera coordinate system are determined, and the initial values of the homogeneous barycentric coordinate coefficients of the control points are calculated. With the optimization goal of reducing the distance difference between the control points in the camera coordinate system and the world coordinate system, the initial values of the homogeneous barycentric coordinate coefficients of the control points are used to solve through an iterative optimization algorithm to obtain the precise pose of the spatial reference marker relative to the camera, including:
[0094] By establishing a parallel perspective projection model, the equation group of the camera imaging perspective projection model is reconstructed into a linear equation group about the control point coordinates and the homogeneous barycentric coordinate coefficients;
[0095] By iteratively solving the equation group of the parallel perspective projection model, an initial estimate of the camera pose is obtained, and based on the initial estimate, the initial coordinates of the control point in the camera coordinate system are calculated, and based on the initial coordinates of the control point in the camera coordinate system, the homogeneous barycentric coordinate equation group is solved by the least squares method to obtain the initial values of the homogeneous barycentric coordinate coefficients of the control point;
[0096] Based on the camera pose and the initial values of the homogeneous centroid coordinate coefficients of the control points, an objective function is constructed based on the distance difference between the control points in the camera coordinate system and the world coordinate system to minimize the objective function, thereby obtaining the optimal camera pose estimation and the precise pose of the spatial reference marker relative to the camera.
[0097] In an optional embodiment,
[0098] The objective function includes:
[0099] The formula is as follows:
[0100]
[0101] Among them, β represents the optimization target, l and j represent the virtual control point index, and C l c represents the coordinates of the lth virtual control point in the camera coordinate system, C j c represents the coordinates of the jth virtual control point in the camera coordinate system, C l w represents the coordinates of the lth virtual control point in the world coordinate system, C j w Represents the coordinates of the jth virtual control point in the world coordinate system.
[0102] In an optional embodiment,
[0103] The initial position of the spatial reference marker relative to the end of the robotic arm is used as a reference position; when the spatial reference marker moves, the camera captures the new position of the spatial reference marker in real time, calculates the current position of the end of the robotic arm, combines the reference position and the current position of the end of the robotic arm, calculates the target position of the end of the robotic arm in real time, and moves the robotic arm according to the target position, including:
[0104] Placing a spatial reference marker in a workspace corresponding to the end of the robotic arm, wherein the spatial reference marker is composed of a plurality of control points with clear coordinates;
[0105] Based on the monocular camera completely photographing the spatial reference marker as a reference, when the robotic arm is located at an initial position, the initial joint angle of the robotic arm is recorded, the initial posture of the spatial reference marker relative to the monocular camera is calculated, and the initial posture is used as the reference posture;
[0106] The monocular camera captures an image corresponding to the spatial reference marker in real time, and calculates a current position of the spatial reference marker relative to the monocular camera;
[0107] According to the reference posture and the current posture, the posture change of the spatial reference marker is calculated, and the joint angle change is calculated according to the posture change and the initial joint angle, and the joint angle change is compensated to the current joint angle of the robotic arm to obtain the target joint angle of the end of the robotic arm, and the robotic arm is moved based on the target joint angle.
[0108] In an optional embodiment,
[0109] The monocular camera taking real-time images corresponding to the spatial reference markers includes:
[0110] Transform the matrix into a 4x4 homogeneous matrix To represent the relative position between coordinate systems, the homogeneous transformation matrix The meaning is the position of coordinate system j relative to coordinate system i.
[0111] At the initial stage, the relative position reference between the end of the robotic arm and the spatial reference marker is as follows:
[0112]
[0113] in, Represents the constant homogeneous transformation matrix between the camera coordinate system and the target pose coordinate system, represents the initial homogeneous transformation matrix between the camera coordinate system and the robot base coordinate system, Represents the initial homogeneous transformation matrix between the robot base coordinate system and the target pose coordinate system;
[0114] After the spatial reference marker moves, the relative position between the spatial reference marker and the end of the robotic arm remains unchanged, satisfying the following formula:
[0115]
[0116] in, Represents the homogeneous transformation matrix of the camera coordinate system relative to the robot base coordinate system, Represents the homogeneous transformation matrix of the target pose coordinate system relative to the robot base coordinate system;
[0117] At the same time, the position and posture of the end of the robotic arm in the base coordinate system of the robotic arm satisfies the following formula:
[0118]
[0119] in, Represents the homogeneous transformation matrix of the robot end coordinate system relative to the robot base coordinate system, A homogeneous transformation matrix representing the transformation matrix of the camera coordinate system relative to the robot end coordinate system;
[0120] The spatial reference marker moves to a new position, and based on the base coordinate system, the new pose formula is as follows:
[0121]
[0122] in, Represents the homogeneous transformation matrix of the new position of the target pose relative to the camera coordinate system, Represents the homogeneous transformation matrix of the new position of the target pose relative to the coordinate system of the robot base, Represents the homogeneous transformation matrix of the new position of the target pose relative to the camera coordinate system.
[0123] In this embodiment, by expressing the world coordinates of the feature points as a linear combination of non-coplanar virtual control points and establishing an accurate camera perspective projection model, the method can obtain the accurate position and posture of the spatial reference marker relative to the camera through an iterative optimization algorithm, thereby improving the accuracy of the posture estimation; by introducing a parallel perspective projection model and an iterative parallel perspective solution process, the method can quickly obtain an initial estimate of the camera posture without an initial posture, thereby providing an effective starting point for further iterative optimization, thereby improving not only the convergence speed of the algorithm but also the accuracy of the final posture estimation; by fixing the monocular camera at the end of the robotic arm and capturing the new position and posture of the spatial reference marker in real time, the method can .... The method can calculate and adjust the target posture of the end of the robot arm in real time to adapt to the position change of the spatial reference marker. The real-time posture tracking and adjustment mechanism provides strong support for the precise operation and automated operation of the robot arm; the method does not rely on the specific shape or size of the spatial reference marker, but only needs to arrange identifiable feature points within the camera's field of view, and has high flexibility; since the method only needs a monocular camera as a visual sensor and realizes posture estimation and posture tracking through software algorithms, compared with systems that require multiple sensors or high-cost hardware, the method can reduce the hardware cost and complexity of the system while ensuring high accuracy.
[0124] A specific embodiment of the present application is as follows:
[0125] The robotic arm tracking of patient movement mainly includes two key steps. The first is that the monocular camera at the end of the robotic arm obtains the position and posture of the target marker. When the target moves, the displacement and direction of the target movement can be calculated. The second is that the robotic arm moves the same distance in the same direction according to the displacement and direction of the target movement, keeping the relative position between the end of the robotic arm and the target unchanged, thereby tracking the patient's movement.
[0126] First, the monocular camera obtains the position information of the target landmark;
[0127] Monocular estimation of target posture is to estimate the position and posture of the camera based on a set of spatial point and image point information. In the target coordinate system {O w -X w Y w Z w The coordinates are In the camera coordinate system {O c -X c Y c Z c The coordinates of} are In the image plane coordinate system {O f -uv} is the pixel coordinate p i (u i , vi After the camera is calibrated, the parameters of the camera internal parameter matrix can be obtained: focal length f, optical center coordinates (u0, v0), physical length of unit pixel (d x , d y ), let f x =f / d x 、f u =f / d y , construct the camera's intrinsic parameter matrix:
[0128]
[0129] According to the camera perspective projection model, the coordinates of the space point in the camera coordinate system The projection coordinates p on the image plane i (u i , v i )The relationship is as follows:
[0130]
[0131] Where λ i represents the depth factor, K represents the intrinsic parameter matrix of the camera;
[0132] Landmark point P i Coordinates in the camera coordinate system and the coordinates in the world coordinate system The satisfying relationship is as follows:
[0133]
[0134] Where R∈SO(3) represents the rotation relationship between the world coordinate system and the camera coordinate system, and t∈R 3 Represents the translation relationship between the world coordinate system and the camera coordinate system.
[0135] Through the above relationship model, when the camera intrinsic parameter K is known, through n pairs of matching image coordinates {p i , i = 1, 2, ..., n} and its corresponding world coordinate system coordinates To calculate the relative pose R and t between the camera coordinate system and the world coordinate system; for a point in the world coordinate system Can be represented as 4 non-coplanar virtual control points A linear combination of:
[0136]
[0137] where α ij For feature points The second centroid coordinates of In theory, non-coplanar virtual control points You can choose any one;
[0138] In particular, the same landmark point P i The relationship between the homogeneous centroid coordinates of the control point and the camera coordinate system remains unchanged. i Homogeneous Barycentric Coordinates and Control Points The same relationship is satisfied:
[0139]
[0140] Therefore, the camera perspective projection model can be modified as follows:
[0141]
[0142] Rearranging this formula gives The system of equations:
[0143]
[0144] For n landmarks we can get 2n equations:
[0145] M 2n×12 x 12×1 =0 (8)
[0146] In the formula The solution x of the system of equations lies in the null space of the coefficient matrix M and can be expressed as:
[0147]
[0148] Where v i M T The eigenvector corresponding to the zero eigenvalue of M is i After confirmation, the coordinates of the four virtual control points in the camera coordinate system is also determined, so that the coordinates of the marker point in the camera coordinate system can be calculated After obtaining the coordinates of the marker point in the world coordinate system and the camera coordinate system, the camera's position and posture can be calculated based on the solution to the absolute orientation problem.
[0149] Solving for the coefficient β i After the value of , the accuracy of the solution can be further improved through Gauss-Newton optimization. The goal of Gauss-Newton optimization is to reduce the distance difference between the control points in the camera coordinate system and the world coordinate system. Its objective function is:
[0150]
[0151] The EPnP algorithm can achieve good results when the number of 3D-2D points n>6, but when the number of 3D-2D points is small, such as n=4 or 5, the EPnP+GN algorithm is unstable. The factor causing the instability of the algorithm is the coefficient β obtained by the EPnP algorithm. i The initial value of is sometimes seriously deviated from the correct value, which makes the subsequent Gauss-Newton optimization unable to converge correctly. Therefore, the EPnP algorithm is improved by introducing a parallel perspective projection model, through which an initial estimate of the pose R0 and t0 is obtained. According to the relationship The coordinates of the control point in the camera coordinate system can be obtained Then we can get because:
[0152]
[0153] Therefore, when V and x are known, the coefficient β=[β1,β2,β3,β4] T The initial value is β0=V T V) -1 V T x;
[0154] The parallel perspective projection model is used to approximate the pose solution under the perspective projection model through an iterative parallel perspective solution process. It can be regarded as a first-order approximation of perspective projection. Let the camera pose R = [i, j, k] T and t=[t x ,t y ,t z ] T , then the parallel perspective projection equation is expressed as:
[0155]
[0156] Where u0 = t x / t z , v0=t y / t z ,definition:
[0157]
[0158] Then I p and J p It can be solved linearly by equation (12), thus I p and J p Modulo t z The two solutions of , take the average of the two solutions as t z The three elements of the translation vector can be obtained as follows:
[0159]
[0160] According to the orthogonality of R, we have:
[0161]
[0162] If you order[·] × Represents the antisymmetric matrix corresponding to the three-dimensional vector, then the above equation is solved to obtain k:
[0163]
[0164] Substituting it into formula (12) we can get the corresponding i and j;
[0165] After obtaining i and j, we can get an initial estimate of the camera pose R0 and t0. According to the relationship The coordinates of the control point in the camera coordinate system can be obtained Then we can get Then, by optimizing the Gauss-Newton optimization objective function, the accurate posture R and t can be optimized:
[0166]
[0167] Second, the robotic arm tracks the moving target marker;
[0168] The camera is fixed at the end of the robotic arm and is used to track the marker (goal). During this process, the camera will record the position of the target marker. When the target marker moves, the end of the robotic arm will track the target marker so that the relative position between the end and the target remains unchanged.
[0169] Now assume that the position of the landmark recorded by the camera (cam) in space is Initially, the relative position between the end of the robot arm and the target is After moving the target marker, the relative position between the end and the target remains unchanged. The position of the camera in the robot base coordinate system (base) Should meet:
[0170]
[0171] The position of the end in the base coordinate system (base) Should meet:
[0172]
[0173] When the target marker moves to a new position After that, the new position of the end in the base coordinate system (base) Should meet:
[0174]
[0175] In the above formula,
[0176] (1) is the current position of the end of the robot arm in the base coordinate system, which is known;
[0177] (2) is the position of the end marker relative to the end of the robot arm, a constant that can be obtained by hand-eye calibration;
[0178] (3) are the positions and poses of the two markers in the camera coordinate system, which are known;
[0179] (4) is the relative position reference recorded at the initial time, which is known;
[0180] Therefore, according to formula (20), the homogeneous matrix required for the robot arm to track the target movement when the target moves can be calculated.
[0181] In this embodiment, by using the EPnP algorithm and its improvement, combined with the Gauss-Newton optimization method, the method can accurately estimate the position and posture of the target marker in the case of a small number of 3D-2D corresponding points; the method is particularly suitable for practical application scenarios with a limited number of markers; the introduced parallel perspective projection model provides a more accurate initial estimate of the position and posture for the EPnP algorithm, making the subsequent optimization process more robust, greatly improving the stability and accuracy of the algorithm when the number of markers is small; the method captures the position and posture changes of the target marker in real time, and calculates the distance and direction that the robot arm needs to move, keeping the relative position with the target unchanged; the real-time response capability enables the method to be applicable to dynamically changing The invention can be applied to scenarios such as instrument tracking in medical surgery or sample manipulation in the laboratory; by automatically capturing the position and posture of the target marker and calculating the corresponding robot arm movement, the complexity of the operation and the dependence on professional operators are reduced, and the safety and efficiency of the operation are improved; the method is not limited by the shape and size of the specific marker. As long as the position and posture of the marker can be accurately captured by a monocular camera, accurate tracking can be achieved. The method is flexible and can be widely used in a variety of different occasions and tasks; the embodiment demonstrates a solution that combines advanced visual algorithms and mechanical control technology, provides a new technical route for the fields of robot arm control and machine vision recognition, and has good promotion and application prospects.
[0182] Figure 2 FIG. 1 is a schematic diagram of the structure of a monocular visual posture estimation system based on the EPnP algorithm according to an embodiment of the present invention. Figure 2 As shown, the system comprises:
[0183] The first unit is used to obtain the internal parameters of the monocular camera through camera calibration; determine the world coordinates of the feature points on the spatial reference markers arranged in the field of view of the monocular camera, and the pixel coordinates of the feature points on the camera image; express the world coordinates of the feature points as a linear combination of four non-coplanar virtual control points, establish a camera perspective projection model, form a linear equation group about the camera coordinates of the virtual control points, and determine the camera coordinates corresponding to the virtual control points in the camera coordinate system; wherein the monocular camera is fixed to the end of the robotic arm, and the optical axis of the monocular camera coincides with the center point of the end of the robotic arm;
[0184] The second unit is used to introduce a parallel perspective projection model, obtain an initial estimate of the camera pose through an iterative parallel perspective solution process, determine the coordinates of the control points in the camera coordinate system based on the initial estimate of the camera pose, and calculate the initial values of the homogeneous barycentric coordinate coefficients of the control points; with the optimization goal of reducing the distance difference between the control points in the camera coordinate system and the world coordinate system, use the initial values of the homogeneous barycentric coordinate coefficients of the control points to solve through an iterative optimization algorithm to obtain the precise pose of the spatial reference marker relative to the camera;
[0185] The third unit is used to use the initial posture of the spatial reference marker relative to the end of the robotic arm as a reference posture; when the spatial reference marker moves, the camera captures the new posture of the spatial reference marker in real time, calculates the current posture of the end of the robotic arm, combines the reference posture and the current posture of the end of the robotic arm, calculates the target posture of the end of the robotic arm in real time, and moves the robotic arm according to the target position.
[0186] According to a third aspect of the embodiments of the present invention,
[0187] An electronic device is provided, comprising:
[0188] processor;
[0189] a memory for storing processor-executable instructions;
[0190] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0191] According to a fourth aspect of the embodiments of the present invention,
[0192] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0193] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.
[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A monocular visual pose estimation method based on EPnP algorithm, It is characterized in that include: Obtain the monocular camera intrinsic parameters through camera calibration; Determine the world coordinates of the feature points on the spatial reference markers arranged within the field of view of the monocular camera, and the pixel coordinates of the feature points on the camera image; express the world coordinates of the feature points as a linear combination of four non-coplanar virtual control points, establish a camera perspective projection model, form a linear equation group about the camera coordinates of the virtual control points, and determine the camera coordinates corresponding to the virtual control points in the camera coordinate system; wherein the monocular camera is fixed to the end of the robotic arm, and the optical axis of the monocular camera coincides with the center point of the end of the robotic arm; A parallel perspective projection model is introduced, and an initial estimation of the camera pose is obtained through an iterative parallel perspective solution process. Based on the initial estimation of the camera pose, the coordinates of the control points in the camera coordinate system are determined, and the initial values of the homogeneous barycentric coordinate coefficients of the control points are calculated. With the optimization goal of reducing the distance difference between the control points in the camera coordinate system and the world coordinate system, the initial values of the homogeneous barycentric coordinate coefficients of the control points are used to solve through an iterative optimization algorithm to obtain the precise pose of the spatial reference marker relative to the camera. The initial posture of the spatial reference marker relative to the end of the robotic arm is used as the reference posture; when the spatial reference marker moves, the camera captures the new posture of the spatial reference marker in real time, calculates the current posture of the end of the robotic arm, combines the reference posture and the current posture of the end of the robotic arm, calculates the target posture of the end of the robotic arm in real time, and moves the robotic arm according to the target position.
2. The method according to claim 1, It is characterized in that Obtaining the internal parameters of a monocular camera through camera calibration; determining the world coordinates of a feature point on a spatial reference marker arranged within the field of view of the monocular camera, and the pixel coordinates of the feature point on the camera image; expressing the world coordinates of the feature point as a linear combination of four non-coplanar virtual control points, establishing a camera perspective projection model, forming a linear equation group about the camera coordinates of the virtual control point, and determining the camera coordinates corresponding to the virtual control point in the camera coordinate system includes: The monocular camera is calibrated by using the Zhang Zhengyou calibration method to obtain an intrinsic parameter matrix of the monocular camera, including the focal length and the optical center coordinates of the camera; The checkerboard in the camera field of view is used as a plane reference marker, the pixel coordinates of the corner points on the plane reference marker are detected by a feature extraction algorithm, n corner points are selected as feature points, and based on the size of the checkerboard and the arrangement of the corner points, the world coordinates of the feature points in the checkerboard coordinate system corresponding to the checkerboard are calculated; In the chessboard coordinate system, four non-coplanar points are selected as virtual control points, and a transformation relationship from the chessboard coordinate system to the virtual control point coordinate system corresponding to the virtual control point is established. The world coordinates of the feature point are expressed by coordinate transformation and homogeneous barycentric coordinates to obtain the coordinate representation of the feature point in the virtual control point coordinate system; Based on the intrinsic parameter matrix, combined with the world coordinates of the feature points and the coordinate representation of the feature points in the virtual control point coordinate system, a camera imaging perspective projection model equation group is established; In the camera imaging perspective projection model equation group, the unknown coordinate representation of the virtual control point in the camera coordinate system is introduced, and the camera imaging perspective projection model equation group is converted into a linear equation group of the control points in the camera coordinate system. The linear equation group is solved by the singular value decomposition numerical optimization method to obtain the coordinates of the four virtual control points in the camera coordinate system.
3. The method according to claim 2, It is characterized in that Also includes: The formula of the intrinsic parameter matrix of the monocular camera is as follows: Among them, K represents the internal parameter matrix, f x Indicates the focal length of the camera in the x-axis direction, f y Indicates the focal length of the camera in the y-axis direction, u 0 represents the horizontal coordinate of the optical center of the camera, v 0 Indicates the ordinate of the camera optical center; The camera imaging perspective projection model equation group has the following formula: Among them, j represents the virtual control point index, i represents the feature point index, α ij represents the homogeneous centroid coordinates of the i-th feature point in the virtual control point coordinate system, x j c represents the x component of the jth virtual control point in the camera coordinate system, y j c represents the y component of the jth virtual control point in the camera coordinate system, z j c represents the z component of the jth virtual control point in the camera coordinate system, u i represents the horizontal coordinate of the i-th feature point on the image plane, v i Represents the vertical coordinate of the i-th feature point on the image plane.
4. The method according to claim 1, It is characterized in that A parallel perspective projection model is introduced to obtain an initial estimate of the camera pose through an iterative parallel perspective solution process. Based on the initial estimate of the camera pose, the coordinates of the control points in the camera coordinate system are determined, and the initial values of the homogeneous barycentric coordinate coefficients of the control points are calculated. Taking reducing the distance difference between the control point in the camera coordinate system and the world coordinate system as the optimization goal, the initial value of the homogeneous barycentric coordinate coefficient of the control point is used to solve through an iterative optimization algorithm to obtain the precise position and posture of the spatial reference marker relative to the camera, including: By establishing a parallel perspective projection model, the equation group of the camera imaging perspective projection model is reconstructed into a linear equation group about the control point coordinates and the homogeneous barycentric coordinate coefficients; By iteratively solving the equation group of the parallel perspective projection model, an initial estimate of the camera pose is obtained, and based on the initial estimate, the initial coordinates of the control point in the camera coordinate system are calculated, and based on the initial coordinates of the control point in the camera coordinate system, the homogeneous barycentric coordinate equation group is solved by the least squares method to obtain the initial values of the homogeneous barycentric coordinate coefficients of the control point; Based on the camera pose and the initial values of the homogeneous centroid coordinate coefficients of the control points, an objective function is constructed based on the distance difference between the control points in the camera coordinate system and the world coordinate system to minimize the objective function, thereby obtaining the optimal camera pose estimation and the precise pose of the spatial reference marker relative to the camera.
5. The method according to claim 4, It is characterized in that The objective function includes: The formula is as follows: Among them, β represents the optimization target, j and l represent the virtual control point index, and C j c represents the coordinates of the jth virtual control point in the camera coordinate system, C l c represents the coordinates of the lth virtual control point in the camera coordinate system, C j w represents the coordinates of the jth virtual control point in the world coordinate system, C l w Represents the coordinates of the lth virtual control point in the world coordinate system.
6. The method according to claim 1, It is characterized in that The initial position of the spatial reference marker relative to the end of the robotic arm is used as a reference position; when the spatial reference marker moves, the camera captures the new position of the spatial reference marker in real time, calculates the current position of the end of the robotic arm, combines the reference position and the current position of the end of the robotic arm, calculates the target position of the end of the robotic arm in real time, and moves the robotic arm according to the target position, including: Placing a spatial reference marker in a workspace corresponding to the end of the robotic arm, wherein the spatial reference marker is composed of a plurality of control points with clear coordinates; Based on the monocular camera completely photographing the spatial reference marker as a reference, when the robotic arm is located at an initial position, the initial joint angle of the robotic arm is recorded, the initial posture of the spatial reference marker relative to the monocular camera is calculated, and the initial posture is used as the reference posture; The monocular camera captures an image corresponding to the spatial reference marker in real time, and calculates a current position of the spatial reference marker relative to the monocular camera; According to the reference posture and the current posture, the posture change of the spatial reference marker is calculated, and the joint angle change is calculated according to the posture change and the initial joint angle, and the joint angle change is compensated to the current joint angle of the robotic arm to obtain the target joint angle of the end of the robotic arm, and the robotic arm is moved based on the target joint angle.
7. The method according to claim 1, It is characterized in that The monocular camera taking real-time images corresponding to the spatial reference markers includes: Transform the matrix into a 4x4 homogeneous matrix To represent the relative position between coordinate systems, the homogeneous transformation matrix The meaning is the position of coordinate system j relative to coordinate system i. At the initial stage, the relative position reference between the end of the robotic arm and the spatial reference marker is as follows: in, Represents the constant homogeneous transformation matrix between the camera coordinate system and the target pose coordinate system, represents the initial homogeneous transformation matrix between the camera coordinate system and the robot base coordinate system, Represents the initial homogeneous transformation matrix between the robot base coordinate system and the target pose coordinate system; After the spatial reference marker moves, the relative position between the spatial reference marker and the end of the robotic arm remains unchanged, satisfying the following formula: in, Represents the homogeneous transformation matrix of the camera coordinate system relative to the robot base coordinate system, Represents the homogeneous transformation matrix of the target pose coordinate system relative to the robot base coordinate system; At the same time, the position and posture of the end of the robotic arm in the base coordinate system of the robotic arm satisfies the following formula: in, Represents the homogeneous transformation matrix of the robot end coordinate system relative to the robot base coordinate system, A homogeneous transformation matrix representing the transformation matrix of the camera coordinate system relative to the robot end coordinate system; The spatial reference marker moves to a new position, and based on the base coordinate system, the new pose formula is as follows: in, Represents the homogeneous transformation matrix of the new position of the target pose relative to the camera coordinate system, Represents the homogeneous transformation matrix of the new position of the target pose relative to the coordinate system of the robot base, Represents the homogeneous transformation matrix of the new position of the target pose relative to the camera coordinate system.
8. A monocular visual pose estimation system based on EPnP algorithm, It is characterized in that include: The first unit is used to obtain the intrinsic parameters of the monocular camera through camera calibration; Determine the world coordinates of the feature points on the spatial reference markers arranged within the field of view of the monocular camera, and the pixel coordinates of the feature points on the camera image; express the world coordinates of the feature points as a linear combination of four non-coplanar virtual control points, establish a camera perspective projection model, form a linear equation group about the camera coordinates of the virtual control points, and determine the camera coordinates corresponding to the virtual control points in the camera coordinate system; wherein the monocular camera is fixed to the end of the robotic arm, and the optical axis of the monocular camera coincides with the center point of the end of the robotic arm; The second unit is used to introduce a parallel perspective projection model, obtain an initial estimate of the camera pose through an iterative parallel perspective solution process, determine the coordinates of the control points in the camera coordinate system based on the initial estimate of the camera pose, and calculate the initial values of the homogeneous barycentric coordinate coefficients of the control points; with the optimization goal of reducing the distance difference between the control points in the camera coordinate system and the world coordinate system, use the initial values of the homogeneous barycentric coordinate coefficients of the control points to solve through an iterative optimization algorithm to obtain the precise pose of the spatial reference marker relative to the camera; The third unit is used to use the initial posture of the spatial reference marker relative to the end of the robotic arm as a reference posture; when the spatial reference marker moves, the camera captures the new posture of the spatial reference marker in real time, calculates the current posture of the end of the robotic arm, combines the reference posture and the current posture of the end of the robotic arm, calculates the target posture of the end of the robotic arm in real time, and moves the robotic arm according to the target position.
9. An electronic device, It is characterized in that include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.
10. A computer readable storage medium having computer program instructions stored thereon, It is characterized in that When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Robot positioning device and method based on novel visual guidance
CN108098762A
Motion capture method and device, electronic equipment and mechanical arm control system
CN113246131A
Parameter calibration method and system for hand-eye coordinated grabbing equipment
CN117047756A
Robot 3D visual guidance grabbing method
CN118143929A
Multi-sensor combined calibration device and method
JP2022039906A
