Monocular camera internal reference calibration method under motion capture system

By using calibration points and mathematical optimization algorithms in the motion capture system, the problem that traditional calibration methods cannot accurately obtain the camera's rigid body internal parameters is solved, and the accurate calibration of camera internal parameters in the motion capture system is realized, which simplifies the system architecture and improves the calibration accuracy and efficiency.

CN119068058AInactive Publication Date: 2024-12-03BEIJING INST OF TECH

Patent Information

Application Number
CN202410953519.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The traditional checkerboard calibration method cannot accurately obtain the internal parameters of the camera's rigid body in the motion capture system, because the center of the camera's rigid body does not coincide with the optical center, resulting in inaccurate calibration results, which increases the system complexity and cost, and affects image processing and three-dimensionality. The efficiency of reconstruction.

Method used

A single-eye camera internal parameter calibration method was designed. By pasting calibration points on the calibration board, using the high-precision pose data and mathematical optimization algorithm of the motion capture system, the geometric mapping relationship between spatial points and image pixel points was established, and gradient descent was used. The internal parameters and distortion parameters are optimized to achieve accurate calibration of the camera's rigid body internal parameters without additional mechanical structure.

Benefits of technology

This method simplifies the calibration process, reduces system complexity and cost, improves the accuracy and efficiency of image processing and three-dimensional reconstruction, and improves the performance and stability of the motion capture system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068058A_ABST
    Figure CN119068058A_ABST
Patent Text Reader

Abstract

The invention provides a monocular camera internal reference calibration method under a motion capture system, and a checkerboard calibration method is characterized in that even if an additional mechanical structure is added, the optical center of the camera is aligned with the center of a rigid body, tiny errors also exist, and a final calibration result is influenced. According to the method, accurate calibration of the internal parameters of the camera rigid body under the action capture system is realized by using high-precision pose data provided by the action capture system; extra mechanical structure design is not needed, the calibration process is simplified by pasting calibration points and using a specific template design method, and the cost and complexity of the system are reduced; a mathematical optimization formula in the method uses a gradient descent method, and rapid and accurate convergence can be realized by means of a computer.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a method for calibrating intrinsic parameters of a monocular camera in a motion capture system. Background Art

[0002] Camera intrinsic calibration is the process of obtaining the camera's own parameters, which generally include focal length, principal point coordinates, and distortion parameters. These parameters determine the camera's projection center. Camera intrinsic calibration is the foundation of computer vision and one of the key technologies in the fields of visual positioning and target recognition.

[0003] The most widely used camera intrinsic calibration method in the industry is the checkerboard calibration method. The checkerboard calibration method calculates the camera's intrinsic parameters by photographing a checkerboard with a regular checkered pattern and using the geometric features of these squares. This method usually requires photographing the checkerboard from multiple angles to ensure that the camera can capture enough information to solve the intrinsic parameters. The checkerboard calibration method is simple and easy to use, and has high calculation accuracy, so it has been widely used.

[0004] Motion capture systems are widely used in film and television production, virtual reality and other fields due to their precise spatial positioning capabilities. By creating a camera rigid body in a motion capture system, the position and posture of the camera can be accurately obtained, which is very useful in motion analysis and 3D reconstruction. However, traditional calibration methods, such as the checkerboard calibration method, are not suitable for the intrinsic calibration of the camera rigid body in the motion capture system. This is because in the motion capture system, the rigid body center of the camera does not coincide with the projection center of the camera, resulting in the inability to correctly obtain the internal parameters of the camera rigid body.

[0005] The current checkerboard calibration method has obvious limitations in the application of motion capture systems. This is because in motion capture systems, the rigid center of the camera does not coincide with the optical center of the camera, resulting in the inability of traditional methods to accurately obtain the internal parameters of the camera. This inaccuracy directly affects the effect and efficiency of subsequent image processing and 3D reconstruction tasks.

[0006] Specifically, when using the traditional checkerboard calibration method to perform internal calibration in a motion capture system, an additional mechanical structure must be designed to make the camera rigid body center coincide with the optical center as much as possible. Such a design not only increases the complexity and cost of the system, but may also introduce more errors and unstable factors, affecting the reliability of the calibration results. Summary of the invention

[0007] In view of this, the present invention proposes a method for calibrating the internal parameters of a camera rigid body for a motion capture system. The method can accurately obtain the internal parameters of the camera rigid body in the motion capture scene without the need for additional mechanical structure design, thereby simplifying the system architecture, reducing costs, and improving calibration accuracy. Through the method of the present invention, the accuracy and efficiency of image processing and three-dimensional reconstruction tasks can be significantly improved, thereby improving the performance and stability of the entire motion capture system.

[0008] A method for calibrating the intrinsic parameters of a monocular camera in a motion capture system, comprising:

[0009] Calibration template design: For a (w, h) calibration plate, paste calibration points at four positions m1(0, 0), m2(w, 0), m3(0, h), and m4(w, h) in the two-dimensional plane of the calibration plate, and measure the distance d between one of the calibration points and the plane where the calibration plate is located; where ω and h are the number of corner points of the calibration plate in the horizontal and vertical directions respectively;

[0010] The positions of the four calibration points in the motion capture coordinate system are obtained by using the motion capture system, M1, M2, M3, and M4. Then, the position P of any point (i, j) in the calibration plate in the motion capture coordinate system is obtained according to formula (2): ij ;

[0011]

[0012] Among them, v and h are the unit vectors of the calibration plate in the horizontal and vertical directions respectively;

[0013] Calibration point detection: Place a calibration plate within the camera's field of view, capture the calibration plate image with the camera, use the image processing algorithm library 0penCV to detect the checkerboard corner points in the image, and record the pixel coordinates of the corner points p ij ;

[0014] Establishment of geometric relationship: A pinhole imaging model with distortion is used to establish the geometric mapping relationship between space points and image pixels. Distortion parameters k1, k2, p1, p2 are introduced, and the corner point calculation formula in the pixel coordinate system is shown in formula (3):

[0015]

[0016] Among them, p ij is the coordinate of the corner point of the chessboard, P ij is the coordinate of the corresponding corner point in the world coordinate system calculated by formula (2), r is the distance from the pixel point to the center of the image, (x, y, 1) is the coordinate of the corner point in the normalized plane system, R and t are the rotation transformation matrix and translation transformation matrix in the position of the camera rigid body in the motion capture coordinate system, and K is the internal parameter to be optimized;

[0017] Optimization parameters K, k1, k2, p1, p2:

[0018] The definition of the objective function is shown in formula (4):

[0019]

[0020] Where n represents the number of valid images; p ij is the pixel coordinate of the corner point in the i-th row and j-th column detected in the image, is the projection coordinate calculated by formula (3);

[0021] The gradient calculation of the objective function for parameters K, k1, k2, p1, p2 and the parameter update process are shown in formula (5):

[0022]

[0023] Among them, η is the learning rate, which indicates the step size of each update;

[0024] In the current iteration, the parameters K, k1, k2, p1, p2 are updated using formula (5), and then substituted into formula (3) to obtain After that, substitute it into formula (4) to update the objective function; in this way, enter the next round of iteration and use formula (5) to update the parameters again; and so on, until the value of the objective function converges to the preset threshold, or the maximum number of iterations is reached, the parameters K, k1, k2, p1, p2 obtained in the last round of iteration are output, and the monocular camera intrinsic calibration is completed.

[0025] Preferably, the initial value of the internal parameter K is obtained by singular value decomposition.

[0026] Preferably, the initial values ​​of the distortion parameters k1, k2, p1, p2 are specified as 1e-4.

[0027] Preferably, in the motion capture system, the camera rigid body is realized by pasting a certain number of calibration points on the camera and cooperating with specific motion capture software.

[0028] The present invention has the following beneficial effects:

[0029] The present invention provides a method for calibrating the internal parameters of a monocular camera under a motion capture system. Even if the checkerboard calibration method adds an additional mechanical structure to align the optical center of the camera with the center of the rigid body, there will still be a slight error, which will affect the final calibration result. However, this method uses the high-precision posture data provided by the motion capture system to achieve accurate calibration of the internal parameters of the camera rigid body under the motion capture system; no additional mechanical structure design is required, and the calibration process is simplified by pasting calibration points and using a specific template design method, reducing the cost and complexity of the system; the mathematical optimization formula in this method uses the gradient descent method, which can converge quickly and accurately with the help of a computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] FIG. 1( a ) shows the projection process under the motion capture system provided by an embodiment of the present invention, and FIG. 1( b ) shows the normal camera coordinate system and the camera coordinate system after the rigid body is created;

[0031] Figure 2 A schematic diagram showing a camera device provided by an embodiment of the present invention;

[0032] Figure 3 A schematic diagram showing a checkerboard device and corner points provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention is described in detail below with reference to the accompanying drawings and examples. The method provided by the present invention focuses on how to implement the internal parameter calibration task of a rigid camera under a motion capture system, which is an important basis for realizing key applications such as three-dimensional reconstruction and virtual reality under a motion capture system. In order to facilitate the understanding of the implementation examples of the present invention, the concepts involved in the embodiments of the present invention are first described below. A motion capture system is a device widely used to record and analyze dynamic motion information of a human body or other objects. The most commonly used is optical motion capture, which uses multiple cameras to surround the capture area to capture the movement of a reflective marker ball in space. These markers reflect the light emitted by the camera, thereby creating three-dimensional data of the action. With the help of a motion capture system, the three-dimensional position information of any marker point in the space in the world coordinate system can be easily obtained, and a rigid body can be created based on multiple marker points and the position and posture information of the six degrees of freedom of the rigid body can be obtained.

[0034] Camera intrinsic parameters refer to parameters related to the optical characteristics of the camera, including focal length, principal point coordinates, distortion coefficients, etc. The focal length describes the distance between the optical center of the camera and the imaging plane, and the principal point coordinates refer to the position of the intersection of the optical axis and the plane on the imaging plane. Camera extrinsic parameters refer to the direction and position of the camera in the world coordinate system. The direction is usually represented by a rotation matrix, and the position is located using three-dimensional coordinates (usually X, Y, Z coordinates). These parameters are crucial for accurate image processing and three-dimensional reconstruction. In a motion capture system, by creating a camera rigid body, the position of the center of the camera rigid body and its posture relative to the capture space can be obtained in real time, and the extrinsic parameters can be updated synchronously. This approach allows a high degree of accuracy and response speed to be maintained in a dynamic environment, thereby improving the efficiency and effectiveness of the entire system.

[0035] Camera imaging refers to the process of projecting points in 3D space onto a two-dimensional image plane. This process involves complex geometric and mathematical transformations and requires accurate camera intrinsic and extrinsic parameters. In order to more clearly illustrate the process and the disadvantages of traditional methods in motion capture systems, the present invention will be explained below through Figure 1(a).

[0036] Figure 1(a) shows the classic pinhole imaging model. Without considering distortion, given a point P(X, Y, Z) in the world coordinate system, its coordinates p(u, v, 1) in the pixel coordinate system are derived as shown in formula (1).

[0037] The spatial point P (X, Y, Z) is first obtained by the camera's external parameters to obtain the point P in the camera coordinate system c (X c , Y c , Z c , 1) coordinates P c =RP+t, followed by the proportionality factor Z c Get the normalized plane Finally, the coordinates of p(u, v, 1) in the pixel coordinate system are obtained through the camera internal parameters p = KP c ′.

[0038]

[0039] Among them, R|t describes the external parameters of the camera, K represents the internal parameters of the camera, and Z c Point P is the camera coordinate system c The Z coordinate value in the camera coordinate system.

[0040] In the traditional checkerboard calibration method, it is usually assumed that the optical center of the camera coincides with the origin of the camera coordinate system, and the Z axis of the camera coordinate system coincides with the optical axis. However, in the motion capture system, such an assumption is difficult to meet, as shown in Figure 1(b). In the motion capture system, the rigid body pose of the camera is bound to the rigid body center. If this pose is directly used as the transformation matrix R|t, the center of the camera coordinate system after the transformation will coincide with the rigid body center of the camera. If the rigid body center of the camera is inconsistent with the optical center of the camera, this processing method will lead to projection errors, as shown in the projection deviation of p1 and p2 in Figure 1(a).

[0041] To address this problem, the markers can often be placed at specific locations on the camera to ensure that the rigid body formed by the markers on the camera roughly coincides with the camera's optical center. However, this approach requires additional mechanical structure design and may complicate camera operation, increase system weight and cost. In addition, even with careful design, it is difficult to achieve complete accuracy in aligning the markers with the optical center, and small errors may still be introduced, affecting the final calibration accuracy.

[0042] In contrast, the present invention provides a simpler and more efficient intrinsic parameter calibration method, which can accurately obtain the intrinsic parameters of the camera without adding additional mechanical structures. The method is based on the high-precision pose data provided by the motion capture system and combines mathematical optimization algorithms to achieve accurate calibration of camera parameters. The following will be described in detail.

[0043] Camera rigid body creation: Specifically, in a motion capture system, a camera rigid body can be created by pasting a certain number of calibration points on the camera and cooperating with specific motion capture software. For example, the present invention provides a camera rigid body model, which uses a common RGBD camera and pastes 4 calibration points. Figure 2 A schematic diagram of its rigid body in motion capture is shown.

[0044] Calibration template design: The present invention provides a calibration template design method suitable for motion capture system. Figure 3 A calibration template used in the present invention is shown. Specifically, for a (ω, h) calibration plate, it is necessary to paste calibration points at four positions m1(0, 0), m2(ω, 0), m3(0, h), and m4(ω, h) in the two-dimensional plane of the calibration plate and measure the distance d between one of the calibration points and the plane where the calibration plate is located; the unit is m.

[0045] With the help of the motion capture system, the positions of the above four calibration points in the motion capture coordinate system can be obtained, M1, M2, M3, and M4. Then, the position P of any point (i, j) in the calibration plate in the motion capture coordinate system can be obtained according to formula (2): ij .

[0046]

[0047] Where ω and h are the number of corner points of the calibration plate in the horizontal and vertical directions, respectively. v and h are the unit vectors of the calibration plate in the horizontal and vertical directions, respectively.

[0048] Calibration point detection: Place a calibration plate within the camera's field of view, shoot a template image with the camera, use the image processing algorithm library OpenCV to detect the checkerboard corner points in the image, and record the pixel coordinates of the corner points p ij .

[0049] Establishment of geometric relationship: The present invention uses a pinhole imaging model with distortion to establish the geometric mapping relationship between space points and image pixels. Based on formula (1), the distortion parameters k1, k2, p1, p2 are introduced, and the final corner point calculation formula in the pixel coordinate system is shown in formula (3).

[0050]

[0051] Among them, p ij is the coordinate of the corner point of the chessboard, P ij is the coordinate of the corresponding corner point in the world coordinate system calculated by formula (2), r is the distance from the pixel point to the center of the image, (x, y, 1) is the coordinate of the corner point in the normalized plane system, R and t are the rotation transformation matrix and translation transformation matrix in the position of the camera rigid body in the motion capture coordinate system, respectively, and K is the internal parameter to be optimized.

[0052] Application of optimization algorithm: The present invention uses the gradient descent method to optimize the camera's intrinsic parameters and finds the optimal parameter value by minimizing the objective function. The camera's intrinsic parameter K and distortion parameters k1, k2, p1, p2 are set according to the initial estimated value, where the initial value of the intrinsic parameter K can be obtained by singular value decomposition, and the initial value of the distortion parameter is specified as 1e-4.

[0053] The definition of the objective function is shown in formula (4):

[0054]

[0055] Where n represents the number of valid images. ij is the pixel coordinate of the corner point in the i-th row and j-th column detected in the image, It is the projection coordinate calculated by formula (3).

[0056] The gradient calculation of the objective function for parameters K, k1, k2, p1, p2 and the parameter update process are shown in formula (5):

[0057]

[0058] Wherein, η is the learning rate, which indicates the step size of each update. In the present invention, all parameters use the same learning rate, and different learning rates can be set in practical applications.

[0059] In the current iteration, the parameters K, k1, k2, p1, p2 are updated using formula (5), and then substituted into formula (3) to obtain Then, substitute it into formula (4) to update the objective function; in this way, enter the next round of iteration, and use formula (5) to update the parameters again; and so on, until the value of the objective function converges to the preset threshold, or reaches the maximum number of iterations, and output the parameters K, k1, k2, p1, p2 obtained in the last round of iteration. Through the above method, the high-precision posture data provided by the motion capture system can be used without adding additional mechanical structures, combined with mathematical optimization algorithms, to achieve accurate calibration of camera parameters.

[0060] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for calibrating the intrinsic parameters of a monocular camera in a motion capture system, characterized in that: include: Calibration template design: For a (ω, h) calibration plate, paste calibration points at four positions m1(0, 0), m2(ω, 0), m3(0, h), and m4(ω, h) in the two-dimensional plane of the calibration plate, and measure the distance d between one of the calibration points and the plane where the calibration plate is located; where ω and h are the number of corner points of the calibration plate in the horizontal and vertical directions, respectively; The positions of the four calibration points in the motion capture coordinate system are obtained by using the motion capture system, M1, M2, M3, and M4. Then, the position P of any point (i, j) in the calibration plate in the motion capture coordinate system is obtained according to formula (2): ij ; Among them, v and h are the unit vectors of the calibration plate in the horizontal and vertical directions respectively; Calibration point detection: Place a calibration plate within the camera's field of view, capture the calibration plate image with the camera, use the image processing algorithm library OpenCV to detect the checkerboard corner points in the image, and record the pixel coordinates of the corner points p ij ; Establishment of geometric relationship: A pinhole imaging model with distortion is used to establish the geometric mapping relationship between space points and image pixels. Distortion parameters k1, k2, p1, p2 are introduced, and the corner point calculation formula in the pixel coordinate system is shown in formula (3): Among them, p ij is the coordinate of the corner point of the chessboard, P ij is the coordinate of the corresponding corner point in the world coordinate system calculated by formula (2), r is the distance from the pixel point to the center of the image, (x, y, 1) is the coordinate of the corner point in the normalized plane system, R and t are the rotation transformation matrix and translation transformation matrix in the position of the camera rigid body in the motion capture coordinate system, and K is the internal parameter to be optimized; Optimization parameters K, k1, k2, p1, p2: The definition of the objective function is shown in formula (4): Where n represents the number of valid images; p ij is the pixel coordinate of the corner point in the i-th row and j-th column detected in the image, is the projection coordinate calculated by formula (3); The gradient calculation of the objective function for parameters K, k1, k2, p1, p2 and the parameter update process are shown in formula (5): Among them, η is the learning rate, which indicates the step size of each update; In the current iteration, the parameters K, k1, k2, p1, p2 are updated using formula (5), and then substituted into formula (3) to obtain Then, substitute it into formula (4) to update the objective function; in this way, enter the next round of iteration and use formula (5) to update the parameters again; and so on, until the value of the objective function converges to the preset threshold, or the maximum number of iterations is reached, the parameters K, k1, k2, p1, p2 obtained in the last round of iteration are output, and the monocular camera intrinsic calibration is completed.

2. The method for calibrating intrinsic parameters of a monocular camera in a motion capture system according to claim 1, characterized in that: The initial value of the internal parameter K is obtained through singular value decomposition.

3. The method for calibrating intrinsic parameters of a monocular camera in a motion capture system according to claim 1, characterized in that: The initial values ​​of the distortion parameters k1, k2, p1, and p2 are specified as 1e-4.

4. The method for calibrating intrinsic parameters of a monocular camera in a motion capture system according to claim 1, wherein: In the motion capture system, the camera rigid body is realized by pasting a certain number of calibration points on the camera and cooperating with specific motion capture software.

Citation Information

Patent Citations

  • External parameter calibration method for 3D camera group

    CN108288294A

  • Calibration method for nonlinear model parameters of binocular camera

    CN111968182A

  • Camera calibration processing method and device, equipment and storage medium

    CN114445506A

  • Camera internal reference automatic calibration method and system based on unmanned aerial vehicle, and electronic equipment

    CN117252933A

  • Method for simultaneously calibrating internal reference of camera and exporting coordinate system of camera

    CN117541662A

Cited By

  • Projector and optical motion capture joint calibration system and method

    CN120512528A