Method and apparatus for calibrating camera pose under motion capture system

Abstract

The application discloses a method for calibrating camera pose under a motion capture system, and aims at the traditional camera calibration method of directly pasting points on the camera shell or relying on the machining precision, so as to solve the problem of inaccurate pose of packaged industrial cameras caused by manufacturing errors. By taking the motion capture system as an external absolute reference system, using the known rigid body coordinate system to motion capture coordinate system conversion, the camera pose calibration is upgraded from a 'camera-calibration board' two-body system to a 'camera-calibration board-motion capture' three-body system. The double-mode markers of 'four coplanar point light sources + four reflective balls' are integrated on the calibration board at one time, and a rectangular four-ball geometric constraint model is proposed. Only by using the geometric prior of 4 coplanar points and adjacent edge perpendicularity and opposite edge equal length, 8 equations (4 orthogonal + 4 module length) are constructed to close the solution of 4 unknown distances, and the complete 6DoF of the calibration board in the camera coordinate system is solved at one time by a single image, so that the calculation amount is small and the robustness is high.

Description

Technical Field

[0001] This invention belongs to the technical field of camera calibration, and particularly relates to a method and apparatus for calibrating camera pose in a motion capture system. Background Technology

[0002] The camera's attitude and position significantly impact the positioning accuracy of binocular or multi-view vision. In motion capture systems, the attitude and position of an object are typically obtained by attaching reflective markers to its surface. However, this method, when used with industrial cameras enclosed in a housing, can lead to inaccurate camera pose calibration due to manufacturing errors.

[0003] Therefore, a calibration method that uses external references and combines the transformation relationships of various coordinate systems under the motion capture system is needed to achieve accurate calibration of the camera pose. Summary of the Invention

[0004] The purpose of this invention is to provide a method and apparatus for calibrating camera pose in a motion capture system. By introducing the motion capture system as an external absolute reference system and utilizing its known rigid body coordinate system to motion capture coordinate system transformation, the camera pose calibration is upgraded from a "camera-calibration board" two-body system to a "camera-calibration board-motion capture" three-body system. This completely eliminates the traditional practice of directly attaching points to the camera shell or relying on machining precision, and solves the problem of inaccurate pose caused by manufacturing errors in packaged industrial cameras.

[0005] To solve the above problems, the technical solution of the present invention is as follows: A method for calibrating camera pose in a motion capture system includes: Step 1: Obtain a rigid substrate, on which the following are rigidly fixed: Four coplanar point light sources arranged in a rectangle are used for imaging by the camera; Four reflective spheres, each corresponding to a point light source and arranged coaxially, are used to be identified by the motion capture system. The relative three-dimensional position error between each point light source and the corresponding reflective sphere is ≤0.1mm, and the included angle between adjacent sides of the rectangle is 90°±0.1° and the difference in length between opposite sides is ≤0.1mm; Step 2: Place the rigid substrate within the field of view of the camera to be calibrated, and perform the following simultaneously: The camera captures a single frame image to obtain two-dimensional angle measurement data from four point light sources; The motion capture system acquires the three-dimensional coordinates of four reflective spheres at the same time to obtain the 6DoF pose of the rigid substrate in the motion capture world coordinate system; Step 3: Using the two-dimensional angle measurement data and the known side length of the rectangle, construct a system of closed equations for metric geometry. The system of equations includes: Orthogonal constraint where the dot product of adjacent edge vectors is zero; The length constraint is that the magnitude of the edge vector is equal to the known side length. The distances from the camera to the four point light sources are analytically solved by combining the closed system of equations, thereby determining the complete 6DoF attitude of the rigid substrate in the camera coordinate system; Step 4: Based on the 6DoF attitude of the rigid substrate in the camera coordinate system and the 6DoF attitude in the motion capture world coordinate system, calculate the attitude of the camera coordinate system relative to the motion capture world coordinate system through chain coordinate transformation. Step 5: Using any distance and corresponding unit vector obtained from Step 3, combined with the three-dimensional coordinates of the corresponding reflector in the motion capture world coordinate system, calculate the three-dimensional coordinates of the camera's optical center in the motion capture world coordinate system, and complete the camera position calibration.

[0006] According to one embodiment of the present invention, the closed system of equations is an 8-equation, 4-unknown system, wherein 4 equations are given by the adjacent edge perpendicularity condition and 4 equations are given by the opposite edge length condition, and the Sylvester elimination method is used to solve analytically within a single frame image.

[0007] According to one embodiment of the present invention, when any light spot is blocked, it automatically switches to a three-sphere redundancy mode: using the remaining three spheres and the known diagonal length of the rectangle as additional constraints to maintain a closed and solvable state.

[0008] According to an embodiment of the present invention, a brightness threshold segmentation method is used to extract candidate regions of light spots in an image, morphological processing is performed on the candidate regions of light spots to extract their geometric centers, and their spot areas are calculated. If the spot area is less than the threshold, it is determined to be occluded, and the three-sphere redundancy mode is triggered. From the known rectangular geometric parameters, select two adjacent sides and one diagonal corresponding to the remaining three spheres as new constraints to construct a closed system of 7 equations and 3 unknowns, where: The three equations are derived from the fact that the dot product of the pairwise vectors of the three spheres is 0; The three equations are derived from the fact that the pairwise distances between the three spheres equal the known side lengths; One equation is derived from the fact that the magnitude of the diagonal vector equals the known length of the diagonal. The Sylvester elimination method, the same as that used in the four-sphere model, was employed to solve the problem analytically. The distances between the three spheres were obtained, and then the virtual coordinates of the fourth point were reconstructed using the triangle area method. This ensured that the 6DoF attitude of the rigid substrate in the camera coordinate system was still solved in a single closed loop.

[0009] According to one embodiment of the present invention, the rigid substrate has high contrast reflection / emission characteristics in both visible and infrared bands, and the point light source adopts an 850nm VCSEL and is equipped with a condenser cup with a half-angle ≤8°.

[0010] According to an embodiment of the present invention, the three-dimensional coordinates of the camera's optical center are obtained by the following formula: (x_c, y_c, z_c)^T = (x_p, y_p, z_p)^T - a·

[0011] Where (x_p, y_p, z_p)^T represents the three-dimensional coordinates of the reflector given by the motion capture system, and a is the corresponding distance obtained from the solution of the closed system of equations. Let be the unit direction vector of the point light source.

[0012] According to one embodiment of the present invention, the attitude deviation is represented by Euler angles, and the attitude angle of the rigid substrate relative to the camera is first solved, and then the difference is made with the attitude angle of the rigid substrate in the motion capture world coordinate system to obtain the roll angle, pitch angle and yaw angle deviation of the camera relative to the motion capture world.

[0013] A calibration apparatus for performing a method according to an embodiment of the present invention, comprising: Rigid substrate; Four coplanar point light sources arranged in a rectangular pattern; Four reflective spheres arranged coaxially with the point light source; The processor module stores algorithms for solving closed systems of equations and is configured to receive camera images and motion capture data and output camera 6DoF pose.

[0014] According to one embodiment of the present invention, the processor module is further configured to: Real-time reception of camera images triggers hardware timestamps, ensuring that the time deviation between image frames and motion capture data is less than 1 ms; The image is subjected to brightness threshold segmentation and ellipse fitting to extract the sub-pixel centroids of four light points and calculate the corresponding two-dimensional angle measurement data. The rigid substrate 6DoF pose broadcast by the motion capture system is received, and the three-dimensional coordinates of the four reflective spheres in the motion capture world coordinate system are resolved. If any missing spot is detected or the signal-to-noise ratio is below 20 dB, the system automatically switches to the 3-sphere redundant equation set and calls the Sylvester junction elimination method to solve for the unknown distance. The calculated 6DoF transformation from the camera coordinate system to the rigid substrate coordinate system is chained multiplied with the 6DoF transformation from the rigid substrate coordinate system to the motion capture world coordinate system to output the 6DoF pose of the camera in the motion capture world coordinate system.

[0015] Because the present invention adopts the above technical solution, it has the following advantages and positive effects compared with the prior art: The method for calibrating camera pose in a motion capture system according to one embodiment of the present invention addresses the problem of inaccurate pose of packaged industrial cameras due to manufacturing errors caused by traditional camera calibration methods that rely on direct point affixing to the camera shell or depend on machining precision. By using the motion capture system as an external absolute reference system and utilizing its known rigid body coordinate system to motion capture coordinate system transformation, the camera pose calibration is upgraded from a "camera-calibration board" two-body system to a "camera-calibration board-motion capture" three-body system. The calibration board integrates "four coplanar point light sources + four reflective spheres" dual-mode markers at one time and proposes a rectangular four-sphere geometric constraint model: using only 4 points that are coplanar and whose adjacent sides are perpendicular and opposite sides are of equal length, an 8-equation (4 orthogonal + 4 modulus length) closed solution is constructed to solve for 4 unknown distances, realizing the complete 6DoF of the calibration board in the camera coordinate system in a single image, with low computational load and high robustness. The entire process can be completed in one go within the motion capture workspace. The calibration board only needs to be placed once, and the camera does not need to be moved. It is compatible with binocular, multi-camera, and array camera systems and can directly output the 6DoF of each camera relative to a unified motion capture system, providing a sub-millimeter-level reference for subsequent multi-camera fusion and dynamic compensation. Attached Figure Description

[0016] Figure 1 This is a flowchart of a method for calibrating camera pose in a motion capture system according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a rectangular plate in one embodiment of the present invention; Figure 3 This is a schematic diagram of angle measurement in one embodiment of the present invention; Figure 4 This is a schematic diagram of point target recognition graphic processing in one embodiment of the present invention. Detailed Implementation

[0017] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a method and apparatus for calibrating camera pose in a motion capture system. The advantages and features of the present invention will become more apparent from the following description and claims.

[0018] This embodiment provides a method for calibrating camera pose in a motion capture system. Please refer to [link / reference]. Figure 1 The method includes the following steps: Step 1: Obtain a rigid substrate, on which the following are rigidly fixed: Four coplanar point light sources arranged in a rectangle are used for imaging by the camera; Four reflective spheres, each corresponding to a point light source and arranged coaxially, are used to be identified by the motion capture system. The relative three-dimensional position error between each point light source and the corresponding reflective sphere is ≤0.1mm, and the included angle between adjacent sides of the rectangle is 90°±0.1° and the difference in length between opposite sides is ≤0.1mm; Step 2: Place the rigid substrate within the field of view of the camera to be calibrated, and perform the following simultaneously: The camera captures a single frame image to obtain two-dimensional angle measurement data from four point light sources; The motion capture system acquires the three-dimensional coordinates of four reflective spheres at the same time to obtain the 6DoF pose of the rigid substrate in the motion capture world coordinate system; Step 3: Using the two-dimensional angle measurement data and the known side length of the rectangle, construct a closed system of equations for metric geometry. This closed system of equations includes: Orthogonal constraint where the dot product of adjacent edge vectors is zero; The length constraint is that the magnitude of the edge vector is equal to the known side length. The distances from the camera to the four point light sources are analytically solved by combining the closed system of equations, thereby determining the complete 6DoF attitude of the rigid substrate in the camera coordinate system; Step 4: Based on the 6DoF attitude of the rigid substrate in the camera coordinate system and the 6DoF attitude in the motion capture world coordinate system, calculate the attitude of the camera coordinate system relative to the motion capture world coordinate system through chain coordinate transformation. Step 5: Using any distance and corresponding unit vector obtained from Step 3, combined with the three-dimensional coordinates of the corresponding reflector in the motion capture world coordinate system, calculate the three-dimensional coordinates of the camera's optical center in the motion capture world coordinate system, and complete the camera position calibration.

[0019] The method for calibrating camera pose in a motion capture system mainly consists of two parts: Part 1, camera pose calibration: calibrating the angular deviation between the rigid body coordinate system and the camera coordinate system; Part 2, camera position calibration: calibrating the coincidence of the origin of the rigid body coordinate system and the rear principal point (image-side principal point or optical center) of the camera. Part 1 is primarily achieved through a rectangular four-sphere calibration method, the key to which is determining the angular deviation between the rigid body coordinate system and the camera coordinate system introduced by the manufacturing or installation process. Part 2 is mainly calculated by solving for one of the obtained module lengths and its corresponding unit vector. Because once the four module lengths of the rectangular plate are known, its pose in the camera coordinate system is completely determined.

[0020] Specifically, before implementing the functions in Part 1, it is necessary to establish a rigid body coordinate system → a motion capture coordinate system ([g]). T ), calibration plate (rectangular plate, such as Figure 2 (As shown) Coordinate system → Motion capture coordinate system [b] T First, as described in step 1, a rigid substrate is obtained, on which the following are rigidly fixed: Four coplanar point light sources arranged in a rectangle are used for imaging by the camera; Four reflective spheres, each corresponding to a point light source and arranged coaxially, are used to be recognized by the motion capture system. The relative three-dimensional positional error between each point light source and its corresponding reflective sphere is ≤0.1mm, and the included angle between adjacent sides of the rectangle is 90°±0.1°, with a difference in length between opposite sides ≤0.1mm. Furthermore, the rigid substrate exhibits high contrast reflection / emission characteristics in both visible and infrared bands, and the point light source uses an 850nm VCSEL equipped with a condenser cup, with a half-angle ≤8°.

[0021] Then, the rigid substrate is placed within the field of view of the camera to be calibrated, and the following steps are performed simultaneously: The camera captures a single frame image to obtain two-dimensional angle measurement data from four point light sources; The motion capture system acquires the three-dimensional coordinates of four reflective spheres at the same time to obtain the 6DoF pose of the rigid substrate in the motion capture world coordinate system. The 6DoF pose is the minimum of 6 independent parameters required to completely describe the position and attitude of a rigid body (or coordinate system) in three-dimensional space: translation position (X, Y, Z) and rotation attitude (roll angle, pitch angle, yaw angle).

[0022] Specifically, the motion capture system (hereinafter referred to as motion capture) emits a 100Hz pulse, which is simultaneously connected to the camera trigger port (TTL active high) and the motion capture synchronization interface (SYNC IN) via a BNC coaxial cable; after receiving the pulse, the camera immediately outputs a single frame image and writes the pulse sequence number in the first 4 bytes of the image header; the motion capture records the rigid substrate at the same sequence number at 6DoF, with a time deviation of ≤ 0.1 ms.

[0023] The point light source driving current is supplied by the processor module through I 2 C control (0–50 mA, 0.2 mA increments): The camera first pre-exposes with a 5% duty cycle and reads the maximum grayscale value; if < 128 DN, the current is increased in increments of 10 mA, and if > 220 DN, it is decreased to ensure that the spot grayscale is in the 180–200 DN range and to prevent overexposure and underexposure.

[0024] Using the origin (X, Y, Z) and dimensions of the rigid substrate returned in real time by motion capture, an 80px × 80px prediction window is generated by projecting it onto the image plane according to the pinhole model. An 11×11 adaptive threshold segmentation is performed only within this window, extracting light spot pixels less than 0.5% of the entire image, with CPU time less than 0.3 ms. Ellipse fitting is performed on the light spots within the window to remove pseudo-spots with eccentricity > 0.8. Forster sub-pixel iteration is used, with centroid repeatability σ ≤ 0.05 pixels. Two-dimensional angle measurement (α) is output simultaneously. i ,βi ) = (arctan((u−cx) / fx), arctan((v−cy) / fy)), where (u, v) are the centroid pixel coordinates, and (cx, cy, fx, fy) are taken from the current camera intrinsic parameters.

[0025] The four-point two-dimensional angle measurement, current value, image sequence number, and PTP timestamp are packaged into a 64-byte UDP frame and transmitted to the processor module via gigabit network; the motion capture end also outputs a 64-byte frame (sequence number + 4 reflector 3D coordinates + 6DoF). Only when the sequence numbers of the two frames are consistent can the solution be obtained in step 3, ensuring that the single frame strictly corresponds.

[0026] Step 3: Using the two-dimensional angle measurement data and the known side length of the rectangle, construct a closed system of equations for metric geometry. This closed system of equations includes: Orthogonal constraint where the dot product of adjacent edge vectors is zero; The length constraint is that the magnitude of the edge vector is equal to the known side length. The distances from the camera to the four point light sources are analytically solved by solving a system of closed equations, thereby determining the complete 6DoF attitude of the rigid substrate in the camera coordinate system.

[0027] Furthermore, the closed system of equations is an 8-equation, 4-unknown system, in which 4 equations are given by the condition of adjacent perpendicularity and 4 equations are given by the condition of opposite side length. The Sylvester elimination method is used to solve it analytically within a single frame image.

[0028] Specifically, the coordinate systems with known relative relationships are as follows: Rigid body coordinate system → Dynamic coordinate system ([g]) T ), Calibration plate (rectangular plate) coordinate system → Motion capture coordinate system [b] T Camera coordinate system → Calibration plate (rectangular plate) coordinate system [c] T Once the relative angles between the aforementioned coordinate systems are known, the three-axis angle difference between the camera coordinate system and the rigid body coordinate system, i.e., [r], can be calculated. T It can be expressed as Equation 1. Among them, the coordinate system transformation only includes three attitude angles, i.e., Equation 2.

[0029] 1

[0030] 2 Although the point light source and the reflector sphere are not in the same position, the coordinate offset between them is known. For ease of explanation, the conversion process from the reflector sphere coordinates to the point light source coordinates will be omitted in the subsequent calculations.

[0031] After completing the intrinsic parameter calibration, the camera photographs a rectangular board. Using OpenCV, four light points are identified, and the angle measurement data of the point light source is calculated based on the light point positions and the camera's intrinsic parameters. Figure 3 As shown.

[0032] The angular measurement data of the four light points, the spatial positions of the four reflectors, and the side length of the rectangular plate are known. The distance from the camera to the four light points is unknown. Let the angular measurement data of light point 1 be α1 and β1 (yaw angle and pitch angle, respectively). Then, the unit vector from this light point to the camera is constructed as shown in Equation 3, and the magnitude of the vector from the light point to the camera is set to a. It should be noted that, for ease of calculation, the camera coordinate system is set to be parallel to the motion capture coordinate system. (The forward direction of the optical axis is the positive y-axis, the x-axis is to the right, and the z-axis is upward.) 3 Similarly, the unit vectors of the other three light points are given by Equation 4, with magnitudes of b, c, and d, respectively. The four vectors constructed from the camera's optical center to the four point light sources are respectively represented by... , , , express.

[0033] On the rectangular plate, adjacent virtual edges composed of light spots are perpendicular. According to the property of vector dot product, we can obtain the system of equations in equation 4. Expanding this system and substituting the unit vector from equation 3, we can obtain a system of four equations in terms of a, b, c, and d.

[0034] 4 However, the number of independent equations in this system is less than 4, so it cannot be solved directly and requires additional constraints. The length of the virtual side formed by the light points on the rectangular plate is known, that is, the square of the magnitude of the virtual side vector is known. Let the virtual long side be vl1 and the virtual short side be vl2, and we get the system of equations 5.

[0035] 5 After adding this system of equations, the number of independent equations out of the total of 8 equations becomes 4, which allows us to solve for the vector magnitudes corresponding to the 4 angle measurement data, and thus determine the attitude of the rectangular plate in the camera coordinate system. Let n be the normal vector of the rectangular plate calibration plane in the camera coordinate system. c For (c x , c y , c z If n c Equation 6 is satisfied.

[0036] 6 Determine n c Then, the orientation of the rectangular plate in the camera coordinate system. Calculated according to Equation 7. Wherein, Let be the projection vector of a vertical side (side ab) on the rectangular plate onto the xy plane in the camera coordinate system, calculated according to Equation 8. Wherein, For roll angle, For pitch angle, This is the yaw angle.

[0037] 7 8 Step 4: Based on the 6DoF attitude of the rigid substrate in the camera coordinate system and the 6DoF attitude in the motion capture world coordinate system, calculate the attitude of the camera coordinate system relative to the motion capture world coordinate system through chained coordinate transformation.

[0038] After obtaining the attitude of the rectangular plate in the camera coordinate system, the inverse of the three attitude angles is taken to calculate the camera attitude in the rectangular plate coordinate system. Combined with the attitude of the rectangular plate in the motion capture coordinate system, the camera attitude in the motion capture coordinate system is calculated simultaneously. For ease of calculation, the coordinate system of the rectangular plate is set to be the same as that of the motion capture coordinate system; that is, when the light source surface of the rectangular plate is perpendicular to the ground and parallel to the xz plane of the motion capture coordinate system, the three coordinate axes of the rectangular plate coordinate system and the motion capture coordinate system are parallel and point in the same direction.

[0039] Based on the above settings, the inverse attitude angles need to be negative for roll, pitch, and yaw. The two coordinate systems are parallel and do not involve exchanging different angles. In the rectangular plate coordinate system, the camera's rotation around the x, y, and z axes is given by Equation 9.

[0040] 9 Let the attitude angles of the rectangular plate in the motion capture coordinate system be respectively... , , Then the camera's attitude angle in the motion capture coordinate system can be expressed by Equation 10.

[0041] 10 Therefore, we can derive [c] in Equation 1, expressed in terms of Euler angles. T That is, Equation 11, which is then substituted into the equation to calculate the camera's attitude relative to the carrier. .

[0042] 11 The above method uses rectangular four-sphere geometric constraints to transform the machining errors of "coplanar + orthogonal + side length" into known quantities and substitute them into the closed equation. This allows the obtained camera pose to eliminate the systematic errors introduced by the shell mounting and lens tilt, and the attitude accuracy can be improved to the 0.05 level.

[0043] Furthermore, when any light spot is blocked, it automatically switches to the three-sphere redundancy mode: using the remaining three spheres and the known diagonal length of the rectangle as additional constraints to maintain a closed and solvable state.

[0044] Specifically, a brightness threshold segmentation method is used to extract candidate regions of light spots in the image. Morphological processing is then performed on these candidate regions to extract their geometric centers, and their spot areas are calculated. If the spot area is less than the threshold, it is determined to be occluded, triggering the three-sphere redundancy mode. For example, within the predicted ROI, the Otsu method is used to calculate the brightness threshold and perform secondary correction. After obtaining a binary mask, isolated noise points and internal dark gaps are removed through opening and closing operations. Subsequently, bright spots that meet the conditions of area, roundness, and eccentricity are selected using connected components, and their sub-pixel geometric centers are extracted. At the same time, the spot area drop rate and ellipse eccentricity are calculated. When the area drop rate exceeds 50% or the eccentricity is greater than 0.6, the light spot is determined to be occluded, triggering the three-sphere redundancy mode.

[0045] From the known rectangular geometric parameters, select two adjacent sides and one diagonal corresponding to the remaining three spheres as new constraints to construct a closed system of 7 equations and 3 unknowns, where: The three equations are derived from the fact that the dot product of the pairwise vectors of the three spheres is 0; The three equations are derived from the fact that the pairwise distances between the three spheres equal the known side lengths; One equation is derived from the fact that the magnitude of the diagonal vector equals the known length of the diagonal. The Sylvester elimination method, the same as that used in the four-sphere model, was employed to solve the problem analytically. The distances between the three spheres were obtained, and then the virtual coordinates of the fourth point were reconstructed using the triangle area method. This ensured that the 6DoF attitude of the rigid substrate in the camera coordinate system was still solved in a single closed loop.

[0046] The solution results are geometrically valid: the reconstructed rectangle is considered valid only if the deviation of the four interior angles from 90° is ≤0.5° and the error of the diagonal length is ≤0.3 mm; otherwise, the frame is discarded and an audio-visual prompt is triggered to change the viewpoint.

[0047] In the second part of camera position calibration, the calculation is mainly performed by solving for one of the obtained moduli and its corresponding unit vector. This is because once the four moduli of the rectangular plate are known, its pose in the camera coordinate system is completely determined.

[0048] Step 5: Using any distance and corresponding unit vector obtained from step 3, combined with the three-dimensional coordinates of the corresponding reflector in the motion capture world coordinate system, calculate the three-dimensional coordinates of the camera's optical center in the motion capture world coordinate system, and complete the camera position calibration.

[0049] After solving for the four unknowns in equations 4 and 5, taking any one of the solved moduli and multiplying it by the unit vector yields the coordinates (relative coordinates) of the corner point on the rectangular plate in the camera coordinate system. In the motion capture software, the coordinates of the corresponding reflector sphere (absolute coordinates of the reflector sphere, (…)) can be obtained. ) T Therefore, the camera's coordinates in the motion capture coordinate system can be represented by the difference between its relative coordinates and the absolute coordinates of the reflector sphere. ) T And there is the relationship shown in Equation 12.

[0050] 12 Where 'a' is the corresponding distance obtained from solving the closed system of equations. After creating a rigid body corresponding to the camera carrier in the motion capture software for the unit direction vector of the point light source, adjust the origin of the rigid body coordinate system to the calibrated camera coordinate position to complete the calibration of the camera position.

[0051] Based on the same concept, this embodiment also provides a calibration apparatus for performing the above-described method for calibrating camera pose in a motion capture system, comprising: Rigid substrate; Four coplanar point light sources arranged in a rectangular pattern; Four reflective spheres arranged coaxially with the point light source; The processor module stores algorithms for solving closed systems of equations and is configured to receive camera images and motion capture data and output camera 6DoF pose.

[0052] The processor module is further configured as follows: Real-time reception of camera images triggers hardware timestamps, ensuring that the time deviation between image frames and motion capture data is less than 50 ms; The image is subjected to brightness threshold segmentation and ellipse fitting to extract the sub-pixel centroids of four light points and calculate the corresponding two-dimensional angle measurement data. The rigid substrate 6DoF pose broadcast by the motion capture system is received, and the three-dimensional coordinates of the four reflective spheres in the motion capture world coordinate system are resolved. If any missing spot is detected or the signal-to-noise ratio is below 20 dB, the system automatically switches to the 3-sphere redundant equation set and calls the Sylvester junction elimination method to solve for the unknown distance. The calculated 6DoF transformation from the camera coordinate system to the rigid substrate coordinate system is chained multiplied with the 6DoF transformation from the rigid substrate coordinate system to the motion capture world coordinate system to output the 6DoF pose of the camera in the motion capture world coordinate system.

[0053] The calibration experiment conducted using the method described above for calibrating camera pose in a motion capture system is described below: The side lengths of the 4-sphere rectangular plate (composed of point light sources) are 170mm and 240mm respectively. Calibration is performed according to the steps, and the point target recognition results and output angle measurement data are as follows: Figure 4 As shown in Table 1.

[0054] Table 1 Angle Measurement Data

[0055] Substituting the angle measurement results into the system of simultaneous equations, the vector magnitudes corresponding to the four angle measurement data are obtained by solving the system, as shown in Table 2.

[0056] Table 2 Calculation Results of Modulus

[0057] Substituting the obtained module lengths into equations 6 to 9, we can obtain the three attitude angles of the rectangular plate in the camera coordinate system. The pitch angle is 4.60°, the yaw angle is 2.06°, and the roll angle is 2.12°. Based on the motion capture data, the corresponding three attitude angles of the rectangular plate in the motion capture coordinate system are 1.02°, 0.93°, and -5.08°, respectively. Therefore, the attitude angles of the camera in the motion capture coordinate system can be obtained. Expressed in vector form, this is given by equation 13.

[0058] 13 Furthermore, based on the motion capture data, the three attitude angles of the carrier in the motion capture coordinate system are 0.10°, -0.02°, and -0.06°, respectively. The attitude deviation of the camera relative to the carrier can be calculated, thus completing the attitude deviation calibration. As shown in Equation 14, the pitch angle, roll angle, and yaw angle of the camera relative to the carrier are 3.68°, 1.11°, and 7.14°, respectively.

[0059] 14 Calculate the camera position. Based on Equation 12, convert the modulus *a* obtained from solving the coordinates of point light source 1 (after adding an offset to the coordinates of reflector 1 obtained in the motion capture software) and the unit vector obtained from the angle measurement data. Substituting these values, we can obtain the camera's coordinates in the motion capture coordinate system. As shown in Equation 15, the distances of the camera's rear principal point relative to the origin of the motion capture coordinate system along the x-axis, y-axis, and z-axis are 670.0 mm, -3786.0 mm, and 383.3 mm, respectively.

[0060] 15 In summary, the above-mentioned method for calibrating camera pose in a motion capture system addresses the problem of inaccurate poses caused by manufacturing errors in traditional camera calibration methods that rely on direct point attachment to the camera casing or machining precision. By using the motion capture system as an external absolute reference system and leveraging its known rigid body coordinate system to motion capture coordinate system transformation, the camera pose calibration is upgraded from a "camera-calibration board" two-body system to a "camera-calibration board-motion capture" three-body system. The calibration board integrates "four coplanar point light sources + four reflective spheres" dual-mode markers at one time, and proposes a rectangular four-sphere geometric constraint model: using only the geometric priors of 4 points being coplanar with adjacent sides perpendicular and opposite sides of equal length, an 8-equation (4 orthogonal + 4 modulus length) closed solution is constructed to solve for 4 unknown distances, achieving a single image to solve the complete 6DoF of the calibration board in the camera coordinate system, with low computational cost and high robustness.

[0061] The entire process can be completed in one go within the motion capture workspace. The calibration board only needs to be placed once, and the camera does not need to be moved. It is compatible with binocular, multi-camera, and array camera systems and can directly output the 6DoF of each camera relative to a unified motion capture system, providing a sub-millimeter-level reference for subsequent multi-camera fusion and dynamic compensation.

[0062] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they shall still fall within the protection scope of the present invention.

Claims

1. A method for calibrating camera pose in a motion capture system, characterized in that, include: Step 1: Obtain a rigid substrate, on which the following are rigidly fixed: Four coplanar point light sources arranged in a rectangle are used for imaging by the camera; Four reflective spheres, each corresponding to a point light source and arranged coaxially, are used to be identified by the motion capture system. The relative three-dimensional position error between each point light source and the corresponding reflective sphere is ≤0.1mm, and the included angle between adjacent sides of the rectangle is 90°±0.1° and the difference in length between opposite sides is ≤0.1mm; Step 2: Place the rigid substrate within the field of view of the camera to be calibrated, and perform the following simultaneously: The camera captures a single frame image to obtain two-dimensional angle measurement data from four point light sources; The motion capture system acquires the three-dimensional coordinates of four reflective spheres at the same time to obtain the 6DoF pose of the rigid substrate in the motion capture world coordinate system; Step 3: Using the two-dimensional angle measurement data and the known side length of the rectangle, construct a closed system of metric geometry equations, which includes: Orthogonal constraint where the dot product of adjacent edge vectors is zero; The length constraint is that the magnitude of the edge vector is equal to the known side length. The distances from the camera to the four point light sources are analytically solved by combining the closed system of equations, thereby determining the complete 6DoF attitude of the rigid substrate in the camera coordinate system; Step 4: Based on the 6DoF attitude of the rigid substrate in the camera coordinate system and the 6DoF attitude in the motion capture world coordinate system, calculate the attitude of the camera coordinate system relative to the motion capture world coordinate system through chain coordinate transformation. Step 5: Using any distance and corresponding unit vector obtained from Step 3, combined with the three-dimensional coordinates of the corresponding reflector in the motion capture world coordinate system, calculate the three-dimensional coordinates of the camera's optical center in the motion capture world coordinate system, and complete the camera position calibration.

2. The method for calibrating camera pose in a motion capture system as described in claim 1, characterized in that, The closed system of equations is an 8-equation, 4-unknown system, in which 4 equations are given by the condition of adjacent perpendicularity and 4 equations are given by the condition of opposite side length. The Sylvester elimination method is used to solve it analytically within a single frame image.

3. The method for calibrating camera pose in a motion capture system as described in claim 1, characterized in that, When any light spot is blocked, it automatically switches to the three-sphere redundancy mode: using the remaining three spheres and the known diagonal length of the rectangle as additional constraints to maintain a closed and solvable state.

4. The method for calibrating camera pose in a motion capture system as described in claim 3, characterized in that, The brightness threshold segmentation method is used to extract candidate regions of light spots in the image. Morphological processing is performed on the candidate regions of light spots to extract their geometric centers and calculate their spot areas. If the spot area is less than the threshold, it is determined to be occluded and the three-sphere redundancy mode is triggered. From the known rectangular geometric parameters, select two adjacent sides and one diagonal corresponding to the remaining three spheres as new constraints to construct a closed system of 7 equations and 3 unknowns, where: The three equations are derived from the fact that the dot product of the pairwise vectors of the three spheres is 0; The three equations are derived from the fact that the pairwise distances between the three spheres equal the known side lengths; One equation is derived from the fact that the magnitude of the diagonal vector equals the known length of the diagonal. The Sylvester elimination method, the same as that used in the four-sphere model, was employed to solve the problem analytically. The distances between the three spheres were obtained, and then the virtual coordinates of the fourth point were reconstructed using the triangle area method. This ensured that the 6DoF attitude of the rigid substrate in the camera coordinate system was still solved in a single closed loop.

5. The method for calibrating camera pose in a motion capture system as described in claim 1, characterized in that, The rigid substrate has high contrast reflection / emission characteristics in both visible and infrared bands, and the point light source adopts an 850nm VCSEL and is equipped with a condenser cup with a half angle ≤8°.

6. The method for calibrating camera pose in a motion capture system as described in claim 1, characterized in that, The three-dimensional coordinates of the camera's optical center are obtained by the following formula: (x_c, y_c, z_c)^T = (x_p, y_p, z_p)^T - a· 7. Where (x_p, y_p, z_p)^T are the three-dimensional coordinates of the reflector given by the motion capture system, and a is the corresponding distance obtained by solving the closed system of equations. Let be the unit direction vector of the point light source.

8. The method for calibrating camera pose in a motion capture system as described in claim 1, characterized in that, The attitude deviation is represented by Euler angles. First, the attitude angle of the rigid substrate relative to the camera is solved, and then the difference is made with the attitude angle of the rigid substrate in the motion capture world coordinate system to obtain the roll angle, pitch angle and yaw angle deviation of the camera relative to the motion capture world.

9. A calibration apparatus for performing the method according to any one of claims 1-7, characterized in that, include: Rigid substrate; Four coplanar point light sources arranged in a rectangular pattern; Four reflective spheres arranged coaxially with the point light source; The processor module stores algorithms for solving closed systems of equations and is configured to receive camera images and motion capture data and output camera 6DoF pose.

10. The calibration device as described in claim 8, characterized in that, The processor module is further configured to: Real-time reception of camera images triggers hardware timestamps, ensuring that the time deviation between image frames and motion capture data is less than 50 ms; The image is subjected to brightness threshold segmentation and ellipse fitting to extract the sub-pixel centroids of four light points and calculate the corresponding two-dimensional angle measurement data. The rigid substrate 6DoF pose broadcast by the motion capture system is received, and the three-dimensional coordinates of the four reflective spheres in the motion capture world coordinate system are resolved. If any missing spot is detected or the signal-to-noise ratio is below 20 dB, the system automatically switches to the 3-sphere redundant equation set and calls the Sylvester junction elimination method to solve for the unknown distance. The calculated 6DoF transformation from the camera coordinate system to the rigid substrate coordinate system is chained multiplied with the 6DoF transformation from the rigid substrate coordinate system to the motion capture world coordinate system to output the 6DoF pose of the camera in the motion capture world coordinate system.

Images