Calibration method for rotation center based on feature point trajectory fitting and dihedral constraint

Abstract

The application discloses a rotation center calibration method based on feature point trajectory fitting and space curve constraint. The method acquires camera parameters, controls a camera moving synchronously with a rotating mechanism to collect images containing a calibration object at multiple rotation angles, and synchronously records physical position information; extracts image coordinates of the same target feature to form an image trajectory; performs quadratic curve fitting on the image trajectory to obtain trajectory parameters representing the tilt of the rotation axis, and determines the rotation axis direction in combination with the camera parameters; constructs a projection correction transformation based on the rotation axis direction, corrects the image trajectory into a circular trajectory, inversely transforms the correction center to the original image coordinate system, and obtains the rotation center pixel coordinates; constructs a rotation axis space straight line cluster according to the physical position information and the transformed rotation axis direction, and solves the rotation center physical coordinates based on the space curve constraint. The application can reduce the calibration error caused by the tilt of the rotation axis.

Description

Technical Field

[0001] This application relates to the field of machine vision calibration technology, and in particular to a rotation center calibration method, electronic device and storage medium based on feature point trajectory fitting and non-plane constraints. Background Technology

[0002] In industrial automation, vision alignment, precision assembly, and automated inspection, it is typically necessary to acquire target images using cameras and combine this with the motion control results of motion mechanisms to achieve target positioning, attitude correction, or rotational alignment. For motion mechanisms with rotational degrees of freedom, the calibration accuracy of the rotation center directly affects the accuracy of subsequent vision guidance, platform correction, and assembly alignment. In scenarios where cameras are used in conjunction with rotating mechanisms, the camera can be mounted on the rotating mechanism and move synchronously with it, or it can form a relatively fixed imaging relationship with the rotating mechanism. Ideally, if the rotation axis is not tilted and the camera optical axis satisfies an ideal mounting relationship with the rotation axis, the features on the calibrated object will typically appear as a circular trajectory in multi-angle rotational images. By performing circle fitting on this circular trajectory, the position of the rotation center in the image coordinate system can be obtained.

[0003] However, in actual equipment, the rotating shaft is often difficult to maintain an ideal state due to factors such as machining errors, assembly deviations, installation tilt, shaft eccentricity, or long-term wear. When the rotating shaft tilts, the motion trajectory of the same feature on the calibration object in the image is no longer an ideal circular trajectory, but rather an elliptical or approximately elliptical quadratic curve trajectory. If fitting is still performed according to a two-dimensional circular trajectory, the center of the obtained circle is prone to deviate from the true rotation center, thus introducing systematic errors related to the degree of rotation shaft tilt, camera imaging relationship, and feature position.

[0004] Among existing methods for calibrating the rotation center, one type typically determines the rotation center based on multi-angle image acquisition and circle fitting. This type of method is simple to implement, but it usually implicitly assumes that the rotational motion is an ideal planar circular motion, resulting in insufficient compensation for projection distortion caused by the tilt of the rotation axis. Another type of method determines the rotation axis position through hand-eye calibration, solving coordinate transformation matrices, or external measuring equipment. While this type of method can obtain relatively complete spatial pose information, it usually requires complex motion data, numerous calibration steps, or additional measuring equipment, leading to high costs for on-site deployment and periodic re-inspection. Some methods can estimate inter-axis offset using elliptical trajectories, but these typically focus on estimating the offset based on the ellipse center, elliptical path, or proportional relationships, failing to fully utilize the projection relationship of the rotation axis tilt on the image trajectory, and also struggling to simultaneously and stably output the image coordinates and physical coordinates of the rotation center.

[0005] Therefore, how to compensate for the tilt of the image trajectory by utilizing the rotation axis tilt information contained in the feature trajectory in multi-angle images without relying on complex hand-eye calibration procedures and expensive external measuring equipment, and further combine the physical position information of the motion mechanism to solve the physical coordinates of the rotation center, is a problem that needs to be solved in the current rotation center calibration. Summary of the Invention

[0006] This application provides a rotation center calibration method, electronic device, and computer-readable storage medium based on feature point trajectory fitting and non-plane constraints, to solve the problem that existing rotation center calibration methods cannot simultaneously take into account rotation axis tilt compensation, rotation center pixel coordinate solution, and rotation center physical coordinate solution.

[0007] On one hand, this application provides a rotation center calibration method based on feature point trajectory fitting and skew constraints, comprising: acquiring camera parameters and controlling a camera that moves synchronously with a rotating mechanism to acquire images containing a calibration object at multiple rotation angles, synchronously recording the physical position information of the motion mechanism corresponding to each rotation angle, wherein the position of the calibration object remains unchanged during the acquisition process; extracting image coordinates of the same target feature on the calibration object from multiple images to obtain the image trajectory of the target feature; performing quadratic curve fitting on the image trajectory to obtain trajectory parameters for characterizing the tilt of the rotation axis, and determining the rotation axis direction based on the trajectory parameters and the camera parameters; and constructing a projection correction method for compensating for the tilt of the rotation axis based on the rotation axis direction. A forward transformation is performed to correct the image trajectory, resulting in a circular trajectory. A correction center is determined based on the circular trajectory, and the inverse transformation of the projection correction transformation is used to map the correction center back to the original image coordinate system, obtaining the rotation center pixel coordinates. The rotation axis direction is transformed to the physical coordinate system of the motion mechanism. Based on the physical position information of the motion mechanism, the rotation axis reference point corresponding to each rotation angle is determined, and a rotation axis spatial line cluster is constructed based on the rotation axis reference point and the transformed rotation axis direction. Based on the minimum distance relationship between multiple spatial lines in the rotation axis spatial line cluster and the rotation center to be determined, skew line constraints are formed to solve for the rotation center physical coordinates, and the rotation center pixel coordinates and rotation center physical coordinates are output.

[0008] Furthermore, the camera parameters include camera intrinsic parameters and distortion parameters; the multiple rotation angles are distributed along a preset rotation range; each rotation angle corresponds to a set of calibration data, and each set of calibration data includes the rotation angle corresponding to the set of calibration data, the image acquired at that rotation angle, and the physical position information of the corresponding motion mechanism.

[0009] Furthermore, the target feature is a point feature or local image feature that can be repeatedly located in multiple images; when extracting the image coordinates, the image coordinates of the target feature at different rotation angles are matched so that the image trajectory is formed by associating multiple image coordinates of the same target feature according to the rotation angle.

[0010] Further, the trajectory parameters include ellipse equation, ellipse matrix, or equivalent quadratic curve parameters; determining the rotation axis direction based on the trajectory parameters and the camera parameters includes: determining the spatial circle projection relationship corresponding to the image trajectory according to the camera parameters, and determining the normal vector of the plane containing the spatial circle based on the spatial circle projection relationship, and determining the normal vector as the rotation axis direction in the camera coordinate system.

[0011] Furthermore, the projection correction transformation is determined by the direction of the rotation axis and is used to correct the plane containing the spatial circle corresponding to the image trajectory to a reference plane parallel to the image plane, so that the image trajectory is converted into the circular trajectory on the reference plane.

[0012] Further, determining the correction center based on the circular trajectory includes: performing circle fitting on the circular trajectory to obtain the center coordinates in the correction coordinate system, and using the center coordinates as the correction center; mapping the correction center to the original image coordinate system using the inverse transformation of the projection correction transformation to obtain the rotation center pixel coordinates.

[0013] Furthermore, each rotation axis spatial line in the cluster of rotation axis spatial lines is determined by the rotation axis reference point at the corresponding rotation angle and the transformed rotation axis direction; the rotation axis reference point is determined based on the physical position information of the motion mechanism at the corresponding rotation angle, and the transformed rotation axis direction is the direction of the rotation axis direction in the camera coordinate system after coordinate transformation in the physical coordinate system of the motion mechanism.

[0014] Further, solving for the physical coordinates of the rotation center includes: using the perpendicular distance from the rotation center to each of the spatial lines of the rotation axis as the residual, establishing a distance sum of squares objective function; performing least squares solution on the distance sum of squares objective function to obtain the coordinates of the spatial points that satisfy the skew line constraints, and determining the spatial point coordinates as the physical coordinates of the rotation center.

[0015] On the other hand, this application provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-mentioned rotation center calibration method based on feature point trajectory fitting and non-plane constraints.

[0016] In another aspect, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described rotation center calibration method based on feature point trajectory fitting and non-plane constraints.

[0017] Compared with existing technologies, the above technical solution has the following advantages: By controlling a camera that moves synchronously with the rotating mechanism to acquire images at multiple rotation angles and extracting the image trajectory of the same target feature, the image trajectory is subjected to quadratic curve fitting to obtain trajectory parameters used to characterize the rotation axis tilt. Then, the rotation axis direction is determined in combination with the camera parameters. This allows the influence of the rotation axis tilt on the image trajectory to be incorporated into the calibration process, avoiding the systematic error caused by directly treating the target feature trajectory as an ideal circular trajectory. Furthermore, a projection correction transformation is constructed based on the rotation axis direction to correct the image trajectory affected by the rotation axis tilt into a circular trajectory. The correction center determined based on the circular trajectory is then inversely transformed to the original image coordinate system. This reduces the influence of projection distortion caused by the rotation axis tilt on the solution of the rotation center pixel coordinates and improves the calibration accuracy of the rotation center pixel coordinates.

[0018] The above technical solution also has the following advantages: by converting the rotation axis direction to the physical coordinate system of the motion mechanism and combining the physical position information of the motion mechanism corresponding to each rotation angle to determine the reference point of the rotation axis, a cluster of spatial straight lines of the rotation axis can be constructed to characterize the spatial position of the rotation axis under different acquisition states; further, by forming skew line constraints based on the minimum distance relationship between multiple spatial straight lines and the rotation center to be determined and solving the physical coordinates of the rotation center, the physical coordinates of the rotation center can be obtained at the same time as obtaining the pixel coordinates of the rotation center, reducing the error accumulation and operational complexity caused by simply relying on pixel coordinate transformation or complex hand-eye calibration process, thus making it more suitable for rapid deployment and periodic re-inspection in industrial sites. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the rotation center calibration system provided in this application; Figure 2 A schematic diagram of the trajectory of a target feature image under the tilt of the rotation axis provided in this application; Figure 3 A flowchart of the rotation center calibration method provided in this application; Figure 4 A schematic diagram illustrating the principle of determining the rotation center pixel coordinates for projection correction provided in this application; Figure 5 A schematic diagram illustrating the determination of the physical coordinates of the rotation center using skew-plane linear constraints provided in this application; Figure 6 A schematic diagram of the electronic device structure provided in this application.

[0020] In the diagram: 1. Motion mechanism base; 2. Translation mechanism; 3. Rotation mechanism; 31. Rotation axis; 4. Camera; 5. Lens; 6. Calibration object; 7. Control and processing unit; 8. Physical position information acquisition unit; 9. Image acquisition connection line; 10. Motion control connection line; 20. Image plane; 21. Ideal rotation axis; 22. Tilt rotation axis; 23. Image coordinates; 24. Image trajectory; 25. Ideal circular trajectory; 26. Rotation axis tilt angle; 27. Rotation center projection position; 40. Original image coordinate system; 41. Image trajectory; 42. Image coordinates; 43. Projection correction. 44. Transformation; 45. Correction coordinate system; 46. Circular trajectory; 47. Correction center; 48. Inverse transformation; 59. Rotation center pixel coordinates; 50. Physical coordinate system of motion mechanism; 51. Rotation axis reference point; 52. Transformed rotation axis direction; 53. Rotation axis spatial line; 54. Rotation axis spatial line cluster; 55. Rotation center to be determined; 56. Vertical distance residual; 57. Rotation center physical coordinates; 60. Electronic equipment; 61. Processor; 62. Memory; 63. Communication interface; 64. Bus; 65. Image acquisition interface; 66. Motion control interface; 67. Computer program. Detailed Implementation

[0021] The embodiments of this application will be further described below with reference to the accompanying drawings. It should be understood that the following embodiments are used to illustrate the technical solutions of this application and should not be construed as limiting the scope of protection of this application. Where there is no conflict, the technical features of the following embodiments can be combined with each other.

[0022] In the description of this application, the terms "first," "second," etc., are used only to distinguish different objects and do not indicate any difference in order or importance between the objects. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method, apparatus, device, or medium that comprises a list of elements may include not only the expressly listed elements but also other elements not expressly listed. The terms "acquire," "determine," "generate," "build," "output," etc., can be implemented by a processor executing corresponding program instructions, or by a control and processing unit, an image processing unit, a motion control unit, or a combination thereof. The term "connection" can refer to an electrical connection, a communication connection, a data connection, a control connection, or a mechanical connection, the specific form of which is determined according to the functional relationship between the relevant objects.

[0023] In this application, the calibration object refers to an object whose position remains unchanged during the calibration data acquisition process and which has repeatable target features in the image. The calibration object can be a checkerboard calibration board, a dot array calibration board, or a workpiece, target, or other reference object with high-contrast edges, corners, or local textures. Target features refer to image features that can be repeatedly identified and correlated in images acquired from multiple rotation angles, such as corners, dot centers, edge intersections, and local texture feature points. Image coordinates refer to the position of the target feature in the image plane, which can be pixel coordinates, normalized image coordinates corrected by camera parameters, or other equivalent image plane coordinates. The image trajectory refers to the trajectory formed by associating the image coordinates of the same target feature at multiple rotation angles according to the rotation angle.

[0024] Quadratic curve fitting refers to fitting an image trajectory to obtain trajectory parameters that characterize the trajectory shape and the effect of rotation axis tilt. Trajectory parameters can include ellipse equations, ellipse matrices, or equivalent quadratic curve parameters. In practical installations, when the rotation axis is tilted, the trajectory of the target feature typically exhibits an elliptical or approximately elliptical quadratic curve trajectory. It should be noted that this application does not directly use the geometric center of the quadratic curve as the rotation center. Instead, it utilizes the spatial circle projection relationship contained within the quadratic curve to determine the rotation axis direction, and then performs projection correction based on the rotation axis direction before calculating the pixel coordinates of the rotation center.

[0025] Projection correction transformation refers to a planar projection transformation constructed based on the rotation axis direction to compensate for the effects of rotation axis tilt. This projection correction transformation can be a perspective transformation, homography transformation, or equivalent planar projection correction transformation. Its function is to correct the plane containing the spatial circle corresponding to the image trajectory to a reference plane parallel to the image plane, thus transforming the quadratic curve trajectory affected by rotation axis tilt into a circular trajectory. The rotation axis reference point refers to a point located on a straight line in the rotation axis space, determined based on the physical position information of the motion mechanism at the corresponding rotation angle. The rotation axis spatial line cluster refers to a set of multiple spatial lines composed of multiple rotation axis reference points at multiple rotation angles and the transformed rotation axis direction. Skew line constraints refer to spatial geometric constraints formed based on minimizing the distances from the center of rotation to multiple rotation axis spatial lines.

[0026] Representation based on elliptic matrix The coefficient terms or coefficient matrix constructed from the elements corresponding to the homogeneous coordinate constant term and related terms are used to participate in the solution of the generalized eigenvalues ​​of the normal vector of the plane containing the spatial circle; in different equivalent derivation forms, It can be represented as matrix elements, submatrices, or elliptic matrices. The constructed computational terms.

[0027] Figure 1This is a schematic diagram of the rotation center calibration system provided in this application. Figure 1 As shown, the rotation center calibration system may include a motion mechanism base 1, a translation mechanism 2, a rotation mechanism 3, a camera 4, a lens 5, a calibration object 6, a control and processing unit 7, and a physical position information acquisition unit 8. The rotation mechanism 3 is mounted on the translation mechanism 2, which is mounted on the motion mechanism base 1. The rotation mechanism 3 has a rotation axis 31. The camera 4 and lens 5 can be mounted on the rotation mechanism 3 and move synchronously with it. The calibration object 6 is positioned within the imaging range of the camera 4 and maintains its position during calibration data acquisition.

[0028] The control and processing unit 7 can be connected to the camera 4 via the image acquisition cable 9 to receive images acquired by the camera 4. The control and processing unit 7 can also be connected to the translation mechanism 2 and the rotation mechanism 3 via the motion control cable 10 to control the movement of the translation mechanism 2 and the rotation mechanism 3. The physical position information acquisition unit 8 can acquire or output the physical position information of the motion mechanism, which may include the position readings of the translation mechanism 2, the position readings of the rotation mechanism 3, the grating ruler readings, the encoder readings, the converted physical coordinates of the motion mechanism, the position of the rotation axis base, or a combination thereof. The control and processing unit 7 can determine the physical position information of the motion mechanism corresponding to each rotation angle based on the data output by the physical position information acquisition unit 8.

[0029] In one embodiment, camera 4 is a monocular camera, and lens 5 is mounted on camera 4. The optical axis of camera 4 can be approximately parallel to the rotation axis 31, but translational eccentricity, angular deviation, or axial tilt are permissible in actual installation. Calibration object 6 can be a planar calibration object with multiple identifiable target features. When camera 4 moves with the rotation mechanism 3, calibration object 6 remains stationary in physical space; therefore, the image coordinates of the same target feature on calibration object 6 at different rotation angles can form an image trajectory.

[0030] Figure 2 This is a schematic diagram of the trajectory of the target feature image under the tilt of the rotation axis provided in this application. Figure 2As shown, on the image plane 20, the ideal rotation axis 21 corresponds to the ideal installation state, and the tilted rotation axis 22 has a rotation axis tilt angle 26 with the ideal rotation axis 21. In the ideal state, the trajectory formed by the target feature at multiple rotation angles can approximate an ideal circular trajectory 25; however, when the rotation axis is tilted, the multiple image coordinates 23 of the target feature form an image trajectory 24 on the image plane 20, which can be represented as an ellipse or a quadratic curve trajectory approximately elliptical. The projection position 27 of the rotation center does not necessarily coincide with the geometric center of the image trajectory 24. If a normal circle fitting is directly applied to the image trajectory 24, or if the center of the ellipse is directly used as the rotation center, systematic errors caused by the tilt of the rotation axis are easily introduced.

[0031] Therefore, this application first determines the rotation axis direction through the image trajectory 24, then constructs a projection correction transformation using the rotation axis direction to correct the image trajectory 24 into a circular trajectory, and then solves for the rotation center pixel coordinates based on the circular trajectory. This avoids directly treating the quadratic curve trajectory in the tilted state as an ideal circular trajectory.

[0032] Figure 3 A flowchart illustrating the rotation center calibration method provided in this application. Figure 3 As shown, the rotation center calibration method based on feature point trajectory fitting and non-plane constraints provided in this embodiment may include steps S101 to S107.

[0033] In step S101, camera parameters are acquired, multi-angle images are captured, and physical position information is recorded. Specifically, camera 4 can be pre-calibrated to obtain camera parameters. Camera parameters may include camera intrinsic parameters and distortion parameters. Camera intrinsic parameters may include parameters such as focal length, principal point position, and pixel scale, while distortion parameters may include radial distortion parameters and tangential distortion parameters. The control and processing unit 7 controls the camera 4, which moves synchronously with the rotating mechanism 3, to acquire images containing the calibration object 6 at multiple rotation angles and simultaneously records the physical position information of the motion mechanism corresponding to each rotation angle. The multiple rotation angles can be distributed along a preset rotation range, and each rotation angle corresponds to a set of calibration data. Each set of calibration data may include the rotation angle corresponding to the set of calibration data, the image acquired at that rotation angle, and the corresponding physical position information of the motion mechanism.

[0034] In one example, the rotating mechanism 3 can be controlled to rotate sequentially to N rotation angles, where N is an integer greater than or equal to 5. Multiple rotation angles can cover a large angle range, such as multiple angles within the range of 0° to 360°, or a preset angle range can be selected based on the device structure and calibration space. Each time the rotating mechanism 3 reaches a rotation angle, the camera 4 acquires a frame image, and the physical position information acquisition unit 8 records the corresponding physical position information of the moving mechanism. In this way, the image data, rotation angles, and physical position information of the moving mechanism correspond one-to-one in time and space, which is beneficial for subsequent joint calculation of the rotation center pixel coordinates and rotation center physical coordinates.

[0035] In step S102, image coordinates of the same target feature are extracted to obtain an image trajectory. Specifically, image coordinates of the same target feature on the calibration object 6 can be extracted from multiple images to obtain the image trajectory of the target feature. The target feature is a point feature or local image feature that can be repeatedly located in multiple images. When extracting image coordinates, the image coordinates of the target feature at different rotation angles can be matched so that the image trajectory is formed by associating multiple image coordinates of the same target feature according to the rotation angle.

[0036] In one example, calibration object 6 is a checkerboard calibration board, and the target feature is the checkerboard corner point. The image coordinates of the same checkerboard corner point can be extracted in each frame of the image. In another example, calibration object 6 is a dot array calibration board, and the target feature is the center of the same dot. The image coordinates of the center of that dot can be extracted in each frame of the image. In yet another example, calibration object 6 is a workpiece with local texture or high-contrast edges, and the target features are local texture feature points, edge intersections, or corner points. Image coordinates belonging to the same target feature in different images can be determined through feature matching. Each image trajectory is formed by the image coordinates of the same target feature at multiple rotation angles. When multiple target features exist, multiple image trajectories can be formed separately, and the image trajectory used for calibration can be selected based on fitting residuals, trajectory span, or feature stability. Alternatively, the calibration results of multiple image trajectories can be fused.

[0037] In step S103, a quadratic curve is fitted to the image trajectory, and the rotation axis direction is determined. Specifically, a quadratic curve is fitted to the image trajectory to obtain trajectory parameters characterizing the tilt of the rotation axis, and the rotation axis direction is determined based on the trajectory parameters and camera parameters. The trajectory parameters may include the equation of an ellipse, an ellipse matrix, or equivalent quadratic curve parameters. Since the projection of a spatial circle onto the image plane 20 can be represented as an ellipse or an approximate ellipse, the trajectory parameters and camera parameters can jointly reflect the pose of the plane containing the spatial circle. Based on the projection relationship of the spatial circle, the normal vector of the plane containing the spatial circle can be determined based on the trajectory parameters and camera parameters, and this normal vector is determined as the rotation axis direction in the camera coordinate system.

[0038] In one specific embodiment, let the same target feature be in the th... The image coordinates at each rotation angle are: For multiple image coordinates By performing quadratic curve fitting, the equation of the ellipse is obtained. or elliptic matrix According to photographic geometry, the projection of a spatial circle under perspective transformation is an ellipse. The ratio of its major and minor axes, its orientation angles, and its matrix parameters contain the pose information of the plane containing the spatial circle. Let the fitted ellipse matrix be... The camera intrinsic parameter matrix is The image of the absolute conic section is ,and Then the normal vector of the circle in space In the camera coordinate system, the following generalized eigenvalue relationship can be used to solve for the problem: ; in, Represents an elliptic matrix The relevant matrix terms in the matrix may be derived from elliptic matrices. The constructed coefficient terms, This represents the normal vector of the plane containing the spatial circle in the camera coordinate system. This represents the eigenvalues. After solving the above generalized eigenvalue relationships, we can obtain the direction vector of the rotation axis in the camera coordinate system. In this embodiment, the direction vector is... This refers to the rotation axis direction in the camera coordinate system. It should be noted that the above generalized eigenvalue relationship is an exemplary computational relationship. In actual implementations, the rotation axis direction can also be obtained using numerical optimization, matrix decomposition, or iterative solutions that are geometrically equivalent to quadratic curve projection.

[0039] In step S104, a projection correction transformation is constructed to correct the image trajectory, resulting in a circular trajectory. Specifically, a projection correction transformation can be constructed based on the rotation axis direction to compensate for the tilt of the rotation axis and correct the image trajectory. The projection correction transformation is determined by the rotation axis direction and is used to correct the plane containing the spatial circle corresponding to the image trajectory to a reference plane parallel to the image plane 20, so that the image trajectory is converted into a circular trajectory on the reference plane.

[0040] Figure 4 A schematic diagram illustrating the principle of determining the rotation center pixel coordinates for projection correction provided in this application. (See attached diagram.) Figure 4As shown, in the original image coordinate system 40, multiple image coordinates 42 of the target feature are distributed along the image trajectory 41. Due to the tilt of the rotation axis, the image trajectory 41 is an elliptical or approximately elliptical quadratic curve trajectory. After constructing the projection correction transformation 43 based on the rotation axis direction, the multiple image coordinates 42 in the image trajectory 41 can be mapped to the correction coordinate system 44, making it form a circular trajectory 45. The purpose of the projection correction transformation 43 is to compensate for the projection distortion caused by the tilt of the rotation axis, rather than to perform ordinary image enhancement or filtering.

[0041] In one specific embodiment, the projection correction transformation can be expressed as: ,in Let be the direction of the rotation axis in the camera coordinate system. For the ... Image coordinates The coordinates after projection correction can be expressed as It satisfies: ; in, Indicates the direction of rotation axis A defined projection correction transformation. Through this projection correction transformation, the plane containing the spatial circle can be mapped to a reference plane parallel to the image plane 20, transforming the image trajectory 41, which was originally a quadratic curve, into a circular trajectory 45.

[0042] In step S105, the correction center is determined and inversely transformed to obtain the rotation center pixel coordinates. Specifically, the correction center 46 is determined based on the circular trajectory 45, and the correction center 46 is mapped to the original image coordinate system 40 using the inverse transformation 47 of the projection correction transformation 43 to obtain the rotation center pixel coordinates 48.

[0043] In one specific embodiment, the points after projection correction transformation The following circular trajectory equation is satisfied: ; in, This represents the coordinates of the center of the circle in the corrected coordinate system 44. This represents the radius of the circular trajectory (45). Multiple corrected points can be used. A circle fitting is performed to obtain the center coordinates of the circle in the correction coordinate system 44, and these center coordinates are used as the correction center 46. Then, the correction center 46 is mapped to the original image coordinate system 40 using the inverse transformation 47 of the projection correction transformation 43, resulting in the rotation center pixel coordinates 48. This process can be represented as: ; in, This indicates that the rotation center pixel coordinates are 48 in the original image coordinate system 40. Indicates projection correction transformation The inverse transformation. Through steps S104 and S105, this embodiment first corrects the image trajectory 41 affected by tilt to a circular trajectory 45, then determines the correction center 46 from the circular trajectory 45 and inversely transforms it back to the original image coordinate system 40, avoiding the introduction of errors by directly using the geometric center of the image trajectory 41 as the rotation center.

[0044] In step S106, the rotation axis direction is transformed, and a cluster of spatial lines in the rotation axis space is constructed. Specifically, the rotation axis direction is transformed to the physical coordinate system 50 of the motion mechanism, the rotation axis reference point 51 corresponding to each rotation angle is determined according to the physical position information of the motion mechanism, and a cluster of spatial lines in the rotation axis space 54 is constructed based on the rotation axis reference point 51 and the transformed rotation axis direction 52.

[0045] Figure 5 A schematic diagram illustrating the determination of the physical coordinates of the rotation center for the skew-plane linear constraint provided in this application. (See diagram below.) Figure 5 As shown, in the physical coordinate system 50 of the motion mechanism, each rotation angle corresponds to a rotation axis reference point 51. After transforming the rotation axis direction in the camera coordinate system to the physical coordinate system 50 of the motion mechanism, the transformed rotation axis direction 52 is obtained. Each rotation axis spatial line 53 can be determined by the rotation axis reference point 51 at the corresponding rotation angle and the transformed rotation axis direction 52. Multiple rotation axis spatial lines 53 constitute a rotation axis spatial line cluster 54.

[0046] In one embodiment, if the coordinate transformation relationship between the camera coordinate system and the physical coordinate system 50 of the motion mechanism is known, the rotation axis direction in the camera coordinate system can be directly converted to the rotation axis direction 52 in the physical coordinate system 50 of the motion mechanism based on this coordinate transformation relationship. The coordinate transformation relationship can be determined by camera mounting parameters, extrinsic parameters of the calibration object, geometric relationship between the calibration object and the motion mechanism, translation calibration results, or a combination thereof. If the mounting posture parameters between the camera 4 and the rotating mechanism 3 are known, the direction conversion can be completed based on these mounting posture parameters; if the extrinsic parameters of the camera 4 relative to the calibration object 6 are obtained from the image of the calibration object 6, and the relationship between the calibration object 6 and the physical coordinate system 50 of the motion mechanism is known, the direction conversion can also be completed by combining the above extrinsic parameters and geometric relationship; if the local transformation relationship between the image coordinate system and the physical coordinate system 50 of the motion mechanism is obtained through translation calibration, the rotation axis direction conversion can also be completed by combining this local transformation relationship.

[0047] The reference point 51 of the rotation axis can be determined based on the physical position information of the motion mechanism at the corresponding rotation angle. If the physical position information of the motion mechanism directly represents the position of the rotation axis base, then the position of the rotation axis base can be used as the reference point 51 of the rotation axis. If the physical position information of the motion mechanism is the position reading of the translation mechanism 2, the encoder reading, or the grating ruler reading, it can be converted into the reference point 51 of the rotation axis based on the geometric relationship of the mechanism or the installation offset. If there is a camera installation offset or a rotation mechanism installation offset, the physical position information of the motion mechanism can be corrected according to the preset installation offset to obtain the reference point 51 of the rotation axis at the corresponding rotation angle.

[0048] In one specific embodiment, let the first... The reference point of the rotation axis corresponding to each rotation angle is: The direction of the converted rotation axis is ,in Let be a unit direction vector, satisfying . No. A straight line in space with a rotation axis It can be represented as: ; in, Represents linear parameters. Multiple rotational axes in space. Together they form a cluster of linear lines in the rotation axis space 54.

[0049] In step S107, the physical coordinates of the rotation center are solved based on the non-planar straight line constraint, and the pixel coordinates and physical coordinates of the rotation center are output. Specifically, the non-planar straight line constraint is formed based on the minimum distance relationship between multiple spatial straight lines in the rotation axis spatial straight line cluster 54 and the rotation center 55 to be determined, the physical coordinates 57 of the rotation center are solved, and the pixel coordinates 48 and physical coordinates 57 of the rotation center are output.

[0050] like Figure 5 As shown, due to factors such as image measurement errors, mechanism reading errors, installation errors, or slight oscillations of the rotation axes, multiple spatial lines 53 of the rotation axes may not strictly intersect at the same point, but may form skew lines or approximately skew lines. In this embodiment, the vertical distance from the center of rotation 55 to each spatial line 53 of the rotation axes is used as the vertical distance residual 56, and a sum of squared distance objective function is established. Let the physical coordinates of the center of rotation be... ,but To the A straight line in space with a rotation axis The vertical residual can be expressed as: ; in, Indicates the center of rotation to be determined To the A straight line in space with a rotation axis The vertical distance residual vector, " represents the vector dot product, Indicates the first The reference point of the rotation axis corresponding to each rotation angle Represents the first in the physical coordinate system of the motion mechanism One rotation axis direction vector. Based on multiple vertical distance residuals. The objective function for the sum of squared distances can be established as follows: ; in, This represents the number of lines in the rotation axis space. The objective function is minimized. The coordinates of the spatial points that satisfy the constraints of skew lines can be obtained, and the obtained spatial point coordinates can be determined as the physical coordinates of the rotation center 57.

[0051] In a specific solution method, let and order Let the identity matrix be denoted by the objective function. right Taking the derivative and setting it to zero, we get: ; This can be further summarized as follows: ; Find the above formula. That is, the physical coordinates of the rotation center are 57. Because... Indicates the direction perpendicular to the axis of rotation The projection matrix can yield a stable solution when the distribution of the rotation axis reference points 51 provides sufficient spatial constraints. To improve the stability of the solution, the number of rotation angles can be increased, the coverage of rotation angles can be expanded, or multiple sets of calibration data can be collected at different translation positions to reduce the impact of the degradation of the spatial line cluster 54 on the solution results.

[0052] Through the above steps, this embodiment can simultaneously output the rotation center pixel coordinates 48 and the rotation center physical coordinates 57. The rotation center pixel coordinates 48 can be used for visual alignment, image rotation compensation, or target feature centering in the image coordinate system; the rotation center physical coordinates 57 can be used for motion compensation, rotation alignment, platform correction, or robot end-effector position control in the motion mechanism physical coordinate system 50.

[0053] In a further embodiment, after outputting the rotation center pixel coordinates 48 and rotation center physical coordinates 57, the rotation mechanism 3 can be controlled to rotate to the verification angle, and a verification image containing the calibration object 6 can be acquired. The control and processing unit 7 can predict the position of the target feature at the verification angle based on the rotation center pixel coordinates 48 or rotation center physical coordinates 57, and compare the predicted position with the actual extracted position to obtain the verification residual. The verification residual can be used to evaluate the reliability of the current calibration result. This verification process does not change the solution logic of the above rotation center calibration result and can be used as a quality assessment process after calibration is completed.

[0054] In another embodiment, multiple image trajectories can be generated using multiple target features, and quadratic curve fitting, rotation axis direction estimation, projection correction, and center determination can be performed on each image trajectory. If there are differences in the rotation axis direction or rotation center results corresponding to multiple target features, the multiple results can be weighted and fused based on the fitting residual, target feature positioning confidence, trajectory span, or image quality to obtain the final rotation axis direction, rotation center pixel coordinates 48, or rotation center physical coordinates 57. This can improve the calibration stability under image noise, local occlusion, or single target feature positioning errors. Each image trajectory is formed by the image coordinates of the same target feature at multiple rotation angles, and multiple target features correspond to multiple independent image trajectories.

[0055] In another embodiment, the quadratic curve fitting and rotation axis direction determination process can also be achieved through 3D reconstruction or overall optimization. For example, after matching multiple target features in multi-angle images, the camera motion trajectory or target feature spatial trajectory can be obtained based on motion reconstruction or sparse reconstruction, and then the rotation axis direction can be determined based on spatial circle fitting. Alternatively, the rotation axis direction, a point on the rotation axis, the installation relationship between the camera and the rotating mechanism, and the spatial position of the target features can be used as variables to be optimized, a reprojection error function can be established, and the solution can be obtained through nonlinear optimization. Both the above methods and the aforementioned quadratic curve fitting-based methods aim to compensate for the influence of rotation axis tilt and combine image space and physical space constraints.

[0056] Figure 6 This is a schematic diagram of the electronic device structure provided in this application. Figure 6 As shown, the electronic device 60 may include a processor 61, a memory 62, a communication interface 63, a bus 64, an image acquisition interface 65, and a motion control interface 66. The processor 61, memory 62, communication interface 63, image acquisition interface 65, and motion control interface 66 can be communicatively connected via the bus 64. The memory 62 stores a computer program 67. When the processor 61 executes the computer program 67, it can implement the aforementioned rotation center calibration method based on feature point trajectory fitting and skew constraints.

[0057] Image acquisition interface 65 can be connected to camera 4 to receive images acquired by camera 4. Motion control interface 66 can be connected to translation mechanism 2, rotation mechanism 3, or physical position information acquisition unit 8 to send motion control commands and receive physical position information of the motion mechanism. Communication interface 63 can be used to communicate with external controllers, host computers, display devices, or storage devices. Memory 62 may include non-volatile memory, volatile memory, or a combination thereof. Computer program 67 may include program instructions for performing image acquisition control, target feature extraction, quadratic curve fitting, rotation axis direction determination, projection correction, circular trajectory fitting, spatial straight line family construction, skew line constraint solving, and result output.

[0058] The computer-readable storage medium may store a computer program 67. When the computer program 67 is executed by the processor 61, the above-described method for rotation center calibration based on feature point trajectory fitting and non-plane constraints can be implemented. The computer-readable storage medium may include a read-only memory, random access memory, magnetic disk, optical disk, flash memory, removable storage medium, or other non-transitory storage medium capable of storing program instructions.

[0059] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Those skilled in the art can make modifications or equivalent substitutions to the above embodiments without departing from the concept of this application, and such modifications or equivalent substitutions should all be covered within the protection scope of this application.

Claims

1. A method for calibrating the center of rotation based on feature point trajectory fitting and skew-plane constraints, characterized in that, include: The camera parameters are acquired, and the camera, which moves synchronously with the rotating mechanism, is controlled to acquire images containing the calibration object at multiple rotation angles. The physical position information of the moving mechanism corresponding to each rotation angle is recorded synchronously, and the position of the calibration object remains unchanged during the acquisition process. The image coordinates of the same target feature on the calibrated object are extracted from multiple images to obtain the image trajectory of the target feature; The image trajectory is fitted with a quadratic curve to obtain trajectory parameters that characterize the tilt of the rotation axis, and the direction of the rotation axis is determined based on the trajectory parameters and the camera parameters. Based on the rotation axis direction, a projection correction transformation is constructed to compensate for the tilt of the rotation axis, and the image trajectory is corrected to obtain a circular trajectory; The correction center is determined based on the circular trajectory, and the correction center is mapped to the original image coordinate system using the inverse transformation of the projection correction transformation to obtain the rotation center pixel coordinates; The rotation axis direction is transformed to the physical coordinate system of the motion mechanism. The rotation axis reference point corresponding to each rotation angle is determined according to the physical position information of the motion mechanism. A cluster of rotation axis spatial lines is constructed based on the rotation axis reference point and the transformed rotation axis direction. Based on the minimum distance relationship between multiple spatial lines in the rotation axis spatial line cluster and the rotation center to be determined, skew line constraints are formed. The physical coordinates of the rotation center are solved, and the pixel coordinates and physical coordinates of the rotation center are output.

2. The rotation center calibration method based on feature point trajectory fitting and skew-plane constraints according to claim 1, characterized in that, The camera parameters include camera intrinsic parameters and distortion parameters; The multiple rotation angles are distributed along a preset rotation range; Each rotation angle corresponds to a set of calibration data. Each set of calibration data includes the rotation angle corresponding to the set of calibration data, the image acquired at that rotation angle, and the physical position information of the corresponding motion mechanism.

3. The rotation center calibration method based on feature point trajectory fitting and skew-plane constraints according to claim 1, characterized in that, The target feature is a point feature or local image feature that can be repeatedly located in multiple images; When extracting the image coordinates, the image coordinates of the target feature under different rotation angles are matched so that the image trajectory is formed by associating multiple image coordinates of the same target feature according to the rotation angle.

4. The rotation center calibration method based on feature point trajectory fitting and skew-plane constraints according to claim 1, characterized in that, The trajectory parameters include ellipse equations, ellipse matrices, or equivalent quadratic curve parameters; Determining the rotation axis direction based on the trajectory parameters and the camera parameters includes: determining the spatial circle projection relationship corresponding to the image trajectory according to the camera parameters, determining the normal vector of the plane containing the spatial circle based on the spatial circle projection relationship, and determining the normal vector as the rotation axis direction in the camera coordinate system.

5. The rotation center calibration method based on feature point trajectory fitting and skew-plane constraints according to claim 1, characterized in that, The projection correction transformation is determined by the direction of the rotation axis and is used to correct the plane containing the spatial circle corresponding to the image trajectory to a reference plane parallel to the image plane, so that the image trajectory is converted into the circular trajectory on the reference plane.

6. The rotation center calibration method based on feature point trajectory fitting and skew-plane constraints according to claim 1, characterized in that, Determining the correction center based on the circular trajectory includes: The circular trajectory is fitted to obtain the center coordinates of the circle in the corrected coordinate system, and the center coordinates of the circle are used as the correction center. The correction center is mapped to the original image coordinate system using the inverse transformation of the projection correction transformation to obtain the pixel coordinates of the rotation center.

7. The rotation center calibration method based on feature point trajectory fitting and skew-plane constraints according to claim 1, characterized in that, Each rotation axis spatial line in the rotation axis spatial line cluster is determined by the rotation axis reference point at the corresponding rotation angle and the transformed rotation axis direction; The reference point of the rotation axis is determined based on the physical position information of the motion mechanism under the corresponding rotation angle, and the transformed rotation axis direction is the direction of the rotation axis direction in the camera coordinate system after coordinate transformation in the physical coordinate system of the motion mechanism.

8. The rotation center calibration method based on feature point trajectory fitting and skew-plane constraints according to claim 7, characterized in that, Solving for the physical coordinates of the center of rotation includes: Using the perpendicular distance from the center of rotation to each of the spatial lines of rotation axes as the residual, a target function of sum of squared distances is established; The least squares solution is performed on the objective function of the sum of squared distances to obtain the coordinates of the spatial points that satisfy the constraints of the skew lines, and these spatial point coordinates are determined as the physical coordinates of the rotation center.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the rotation center calibration method based on feature point trajectory fitting and skew constraints as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the rotation center calibration method based on feature point trajectory fitting and skew constraints as described in any one of claims 1 to 8.

Images