A method for correcting three-dimensional coordinates of marker points in optical motion capture

Abstract

The present application relates to the technical field of optical motion capture, and especially to a method for correcting three-dimensional coordinates of marker points in optical motion capture, comprising the following steps: step S1: waving a T-shaped calibration rod with a known distance between marker points in a capture field, collecting multiple frames of multi-view images, based on multi-view geometric constraints, using bundle adjustment to jointly optimize camera internal parameters, external parameters and lens distortion coefficients, obtaining the internal parameters, external parameters and lens distortion coefficients of each camera, establishing an initial three-dimensional reconstruction model, and obtaining initial three-dimensional coordinates of the marker points; the present application uses the known distance constraint of the T-shaped calibration rod and the bundle adjustment to jointly optimize the camera internal and external parameters and distortion coefficients, effectively reduces the re-projection error, and improves the initial three-dimensional reconstruction accuracy; the present application uses the known invariant distance between the marker points on the rigid body for nonlinear optimization, controls the coordinate adjustment amplitude through a regularization term, and can improve the local deformation and scale drift while maintaining the overall structure stability.

Description

Technical Field

[0001] This invention relates to the field of optical motion capture technology, and specifically to a method for correcting the three-dimensional coordinates of marker points in optical motion capture. Background Technology

[0002] Optical motion capture systems typically consist of multiple synchronized cameras. They acquire images of markers attached to the surface of the object being measured and reconstruct the three-dimensional coordinates of these markers using multi-view geometry. These systems are widely used in motion analysis, virtual reality, film special effects production, robot control, and biomechanical research. However, in practical applications, the accuracy and robustness of the marker's three-dimensional coordinates are limited by various factors, and existing calibration methods still have the following shortcomings: System-level calibration accuracy is limited: Traditional methods use a T-shaped calibration rod to swing within the capture area, calculating the camera's intrinsic and extrinsic parameters and lens distortion coefficients from the known spacing between marker points. However, the calibration process usually relies on manual operation, the calibration rod's trajectory covers an uneven range, and the parameters are fixed after a single calibration, making it difficult to adapt to parameter drift caused by factors such as camera micro-movements and changes in ambient temperature during use, resulting in systematic errors in the 3D reconstruction coordinates.

[0003] Post-process optimization methods are limited: Existing technologies often utilize the constraint that the distance between marker points on a rigid body is known and fixed to perform nonlinear optimization of the 3D coordinates. However, such methods can only improve the accuracy of the relative distance between marker points, and have limited effect on improving the scale deviation of the overall coordinate system, local nonlinear deformation, and absolute positioning accuracy.

[0004] High-precision benchmark calibration is costly and not real-time: Using high-precision equipment such as total stations, laser trackers or robotic arms to measure the real position of marker points can obtain high-precision reference data and establish error models. However, such equipment is expensive and complex to operate, and is usually only used for offline verification in laboratory environments. It is difficult to use it directly for real-time calibration in daily capture tasks.

[0005] Occlusion and flying point issues are prominent: When marker points are occluded or interfered with by strong light, optical motion capture systems are prone to generating missing points or incorrect 3D coordinate points (i.e., "flying points"). Existing methods have limited ability to recover coordinates after occlusion and remove outliers, affecting the continuity and reliability of the final trajectory.

[0006] The lack of effective fusion of multi-source calibration results: system-level calibration, rigid body constraint optimization, high-precision benchmark calibration, and special scenario calibration are often used in isolation, each outputting independent results. Existing technologies lack a framework for adaptive fusion based on the uncertainties of each calibration result, leading to insufficient calibration accuracy and robustness.

[0007] To address the aforementioned issues, this application proposes a method for correcting the three-dimensional coordinates of marker points in optical motion capture. Summary of the Invention

[0008] The purpose of this invention is to provide a method for correcting the three-dimensional coordinates of marker points in optical motion capture, thereby addressing the problems of limited accuracy and robustness in existing correction techniques. This invention achieves multi-level, high-precision correction of the three-dimensional coordinates of marker points through system-level calibration, rigid body constraint optimization, high-precision benchmark correction, special scene correction, and adaptive weighted fusion.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a method for correcting the three-dimensional coordinates of a marker point in optical motion capture, comprising the following steps: Step S1: Swing a T-shaped calibration rod with known marker spacing within the capture area to acquire multiple frames of multi-view images. Based on multi-view geometric constraints, use bundle adjustment to jointly optimize the camera's intrinsic and extrinsic parameters and lens distortion coefficients. Obtain the intrinsic and extrinsic parameters and lens distortion coefficients of each camera, establish an initial 3D reconstruction model, and obtain the initial 3D coordinates of the marker points. Step S2: Use known rigid body constraints to perform first-level optimization and correction on the initial three-dimensional coordinates to obtain the first corrected coordinates; Step S3: Measure the true three-dimensional coordinates of at least some of the marker points using a high-precision reference device, compare the measured values ​​with the first corrected coordinates, construct a local error correction model using radial basis function interpolation, and perform a second-level correction on all marker points to obtain the second corrected coordinates; Step S4: Perform special scene correction on the first corrected coordinates to obtain the third corrected coordinates; the special scene correction includes coordinate calculation based on the virtual image of the plane mirror in occluded scenes and / or dynamic anomaly marker identification and removal; wherein, the triggering condition for occlusion recovery is that the marker is missing for at least 5 consecutive frames and the virtual image of the marker in the plane mirror is observed by at least two cameras; the triggering condition for anomaly detection is that the marker has valid observation coordinates and the Kalman filter has been continuously tracked for at least 10 frames; Step S5: Adaptively weightedly fuse the first, second, and third correction coordinates to output the final three-dimensional coordinates of the marker point; the adaptive weighted fusion is as follows: based on the uncertainty of each of the first, second, and third correction coordinates, calculate the weight of each correction coordinate in a manner that the weight is inversely proportional to the square of the uncertainty, and sum the weights to obtain the final output coordinates.

[0010] Preferably, the system-level calibration in step S1 includes: By swinging a T-shaped calibration pole with a known spacing between marker points within the capture area, multiple frames of multi-view images are acquired. Extract the two-dimensional coordinates of the marked points in the multi-frame, multi-view images; Based on multi-view geometric constraints, the camera intrinsic parameters, extrinsic parameters, and lens distortion coefficients are jointly optimized using the bundle adjustment method to minimize the sum of reprojection error and calibration rod distance constraint error. The objective function is: , In the formula, For the first The marker point at the th The observed two-dimensional coordinates in a camera image Projection function, The first The intrinsic parameter matrix, rotation matrix, and translation vector of each camera. To mark the three-dimensional coordinates of the point, Given the known distance between the marked points on the calibration rod. These are the distance constraint weighting coefficients.

[0011] Preferably, the known rigid body constraint in step S2 is that the distance between the marked points on the rigid body is known and remains constant; The first-level optimization correction includes constructing a distance-constrained objective function: , In the formula, The first one to be optimized Three-dimensional coordinates of each marker point For the first The initial three-dimensional coordinates of the marker points Mark the point and The known distance between them For a set of distance-constrained pairs, The regularization coefficient is used. The Levenberg-Marquardt algorithm is used to iteratively solve the distance constraint objective function to obtain the first corrected coordinates.

[0012] Preferably, the high-precision reference device mentioned in step S3 includes a total station, a laser tracker, or a robotic arm; The local error correction model is constructed as follows: Calculate the measured values ​​of high-precision reference equipment With the first correction coordinate Error vector between: , The error field is constructed using radial basis function interpolation: , In the formula, The number of measurement points for high-precision reference equipment. Center of radial basis functions For radial basis functions, These are linear polynomial terms; The weights are determined by solving a system of linear equations. Then, the error field is applied to all marked points for a second-level correction.

[0013] Preferably, the radial basis function is a Gaussian function: , In the formula, the shape parameter Based on the average spacing between measurement points of high-precision reference equipment Confirmed, and , The value of γ is determined by leave-one-out cross-validation.

[0014] Preferably, the coordinate deduction based on the virtual image of the plane mirror in the occlusion scene described in step S4 includes: Obtain the plane equation of a plane mirror ,in For unit normal vector, This is the plane offset; Calculate the reflection matrix: , Obtain the virtual image coordinates of the occluded marker point in the plane mirror. Then the true coordinates for , The true coordinates This serves as the three-dimensional coordinate of the marker point during the occlusion period.

[0015] Preferably, the dynamic anomaly marker identification and removal in step S4 includes: The predicted coordinates are obtained by predicting the trajectory of the marker points using Kalman filtering or extended Kalman filtering. and predicting covariance matrix ; Calculate the actual observed coordinates Mahalanobis distance between the predicted coordinates and the target coordinates: , If the Mahalanobis distance is greater than the preset threshold If the actual observed coordinates are not found, they are determined to be outliers and are either removed or replaced with predicted coordinates. The preset threshold Determined by chi-square distribution quantiles: , in For a chi-square distribution with 3 degrees of freedom Quantiles This represents the expected false positive rate.

[0016] Preferably, the adaptive weighted fusion in step S5 includes: Based on the uncertainties of the first, second, and third correction coordinates Calculate the weights: , Final output coordinates: , In the formula, The first correction coordinate, The second correction coordinate. The third correction coordinate is used; if a correction coordinate is missing, its corresponding weight is set to 0, and the remaining weights are normalized.

[0017] Preferably, the uncertainty The method for determining it is as follows: Estimation of the root mean square error of the residuals corrected by the first-level optimization; It is synthesized from the nominal accuracy of the high-precision reference equipment and the interpolation residual of the radial basis function; Square root estimate of the trace of the predicted covariance matrix by Kalman filtering.

[0018] An optical motion capture system includes multiple cameras, a processor, and a memory, characterized in that the memory stores a computer program, and the processor executes the computer program to implement the above-mentioned method for correcting the three-dimensional coordinates of marker points in optical motion capture.

[0019] Compared with the prior art, the beneficial effects of the present invention are: To address the issue of limited system-level calibration accuracy, this invention utilizes the known distance constraint of the T-shaped calibration rod and the bundle adjustment method to jointly optimize the camera's intrinsic and extrinsic parameters and distortion coefficients, effectively reducing reprojection errors and improving the initial 3D reconstruction accuracy.

[0020] To address the issue of limited post-optimization methods, this invention utilizes the known and invariant distance between marked points on a rigid body for nonlinear optimization. By controlling the coordinate adjustment range through a regularization term, it can improve local deformation and scale drift while maintaining the overall structural stability.

[0021] To address the issues of high cost and non-real-time nature of high-precision benchmark calibration, this invention only requires measuring a small number of marker points using a high-precision benchmark device during the offline phase. After constructing a local error field, all marker points can be calibrated, significantly improving absolute positioning accuracy without affecting real-time performance.

[0022] To address the issues of occlusion and flying points, this invention recovers the coordinates of occluded marker points by calculating the coordinates of the virtual image of a plane mirror, and uses Kalman filtering and Mahalanobis distance to detect dynamic anomalies, thereby improving the continuity and reliability of 3D coordinates in complex scenes.

[0023] To address the problem of ineffective fusion of multi-source correction results, this invention performs adaptive weighted fusion based on the uncertainty of each correction result, giving full play to the advantages of each correction method and further improving the accuracy and robustness of the final three-dimensional coordinates. Attached Figure Description

[0024] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is a schematic diagram of the system-level calibration process in this invention; Figure 3 This is a schematic diagram of the rigid body constraint optimization and correction process in this invention; Figure 4 This is a schematic diagram of the high-precision reference calibration process in this invention; Figure 5 This is a schematic diagram of the special scene correction process in this invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Example 1, System-level calibration System-level calibration is a fundamental correction step before the optical motion capture system can operate formally. In this embodiment, a T-shaped calibration rod with known marker spacing is swung within the capture area, and multiple cameras simultaneously acquire image sequences of the marker points on the calibration rod. Let the three-dimensional coordinates of the marker points in the world coordinate system be... Corresponding to the first The two-dimensional coordinates in each camera image are Using a pinhole camera model, the projection relationship is as follows: , in As a scale factor, For the first The intrinsic parameter matrix of each camera: , and The first The rotation matrix and translation vector of each camera coordinate system relative to the world coordinate system. Lens distortion is modeled as follows: , in , , , , , The radial distortion coefficient is... , denoted as the tangential distortion coefficient.

[0027] Example of specific parameters for the calibration rod: In this embodiment, the T-shaped calibration rod is made of carbon fiber material, with a total length of 600mm, a horizontal bar length of 400mm, and a vertical bar length of 200mm. A spherical reflective marker with a diameter of 14mm is attached to each end of the horizontal bar and the end of the vertical bar. The designed distances between the markers are as follows: distance from the ends of the horizontal bar... The distance between the left end of the horizontal bar and the end of the vertical bar The distance between the right end of the horizontal bar and the end of the vertical bar These distances were calibrated using a coordinate measuring machine, and the error was less than 0.01 mm.

[0028] The distance between the marked points on the calibration rod is known. Let the actual distance between marked points A and B be... The objective function defined at the system level is: , in This represents the projection function that projects three-dimensional points onto a two-dimensional image. These are the distance constraint weighting coefficients.

[0029] The calibration process is implemented as follows: The operator holds a T-shaped calibration rod and slowly swings it within the capture area (approximately 4 m × 3 m × 2.5 m), ensuring the rod passes through all corners and height levels of the space. A total of 200 multi-view images are acquired, covering different poses and positions of the calibration rod. For each frame, the two-dimensional coordinates of the marker points are extracted using a Blob detection algorithm, and matching and identification are performed based on the known distances between the marker points.

[0030] The Levenberg-Marquardt algorithm is used to minimize the objective function described above. Initial values ​​are set as follows: camera intrinsic parameters use factory-nominal values, extrinsic parameters use coarse estimates based on direct linear transformation, and distortion coefficients are initially set to zero. During iteration, when the rate of decrease of the objective function value is less than... Alternatively, the iteration count can be stopped at 200. After optimization, the root mean square error of reprojection decreased from the initial 1.52 pixels to 0.18 pixels, and the root mean square error of distance constraint decreased from 0.85 mm to 0.12 mm.

[0031] Distance constraint weight coefficient The determination is based on an objective method using the initial residual ratio. Firstly, using... Perform a single bundle adjustment optimization with the result set to 0, obtaining the optimization result considering only the reprojection error, and then calculate the root mean square value of the reprojection error at this point. and the root mean square value of the distance constraint error of the calibration rod Then, the initial weights are calculated based on the proportional relationship between the two in terms of their dimensions: , The above formula ensures that the two residuals contribute approximately the same order of magnitude to the objective function, preventing the optimization process from being dominated by either one. Based on this, a balancing factor can be introduced according to actual calibration requirements. ,make ,in The value range is 0.1 to 10, with a typical value of 1.0. When the known accuracy of the spacing between the marker points on the calibration rod is much higher than the image point positioning accuracy, it can be appropriately increased. To strengthen the effect of distance constraints; conversely, to reduce them. In this embodiment, the calculation is obtained ≈2.5, take =1.0, finally adopted =2.5.

[0032] Once the system-level calibration is complete, multi-view can be utilized. Figure 3 The angle measurement method performs 3D reconstruction on the subsequently captured marker points to obtain the initial 3D coordinates. After system-level calibration, a standard rigid body with a known distance of 100.00 mm was placed at the center of the capture area, and the distance error obtained from the 3D reconstruction was approximately 0.25 mm.

[0033] Example 2: First-level optimization correction using known rigid body constraints Rigid body constraint refers to a constraint where the distance between two or more marker points fixed on the same rigid body remains constant during motion. Let the marker points on the rigid body... and The known distance between them is The constraint on the set is The initial three-dimensional coordinates are Construct the following distance-constrained objective function: , in The first one to be optimized Three-dimensional coordinates of each marker point The first term is the regularization coefficient. The second term is the regularization term, used to constrain the optimized coordinates from deviating too far from the initial reconstructed coordinates, thus avoiding instability caused by overall translation or rotation.

[0034] Specific example of rigid body constraints: Four marker points are pasted onto a rigid body to form a tetrahedral structure. The six distance values ​​between the marker points are as follows: =80.00mm, =95.00mm, =70.00mm, =60.00mm, =85.00mm, =75.00mm. These distances were precisely measured beforehand using a coordinate measuring machine with an error of less than 0.02 mm. During motion capture, the initial three-dimensional coordinates of the four marker points were reconstructed by the system.

[0035] Regularization coefficient The method for determining this is as follows: based on the uncertainty of the initial three-dimensional coordinates. Setting: The greater the initial 3D coordinate uncertainty, the lower the initial reconstruction accuracy; in this case, it should be reduced. To allow for a larger range of coordinate adjustments; conversely, it should be increased. In this embodiment, the initial reconstruction accuracy, after system-level calibration, is approximately 0.2 mm. =0.05.

[0036] The Levenberg-Marquardt algorithm is used iteratively to solve the above nonlinear least squares problem. The stopping condition for iteration is: the relative change in the objective function is less than 1%. Or the number of iterations reaches 100. During the optimization process, the coordinates are updated in each iteration as follows: , in This is obtained by solving the following system of linear equations: , Let be the Jacobian matrix of the residual vector with respect to the coordinates. For the residual vector, For damping parameters. In this embodiment, the root mean square error of the distance between the marked points on the rigid body is reduced from the initial 0.35mm to 0.08mm after optimization, while the overall coordinate translation does not exceed 0.15mm and the rotation does not exceed 0.05°, ensuring structural stability.

[0037] Example 3: Second-stage calibration using high-precision reference equipment To further improve absolute positioning accuracy, this invention introduces a high-precision reference device (such as a total station, laser tracker, or robotic arm) to perform real three-dimensional coordinate measurements on some marker points within the capture area. Let the high-precision reference device measure the first... The reference coordinates of each marker point are: The corresponding first correction coordinate is Then the error vector is: , These discrete error vectors are used to construct a continuous error field within the capture region. This invention employs a radial basis function interpolation method: , in The number of measurement points for high-precision reference equipment. The center of the radial basis function is usually taken as the coordinates of the measurement point. The weights to be determined are: For radial basis functions, These are linear polynomial terms used to compensate for global linearity errors.

[0038] In this embodiment, the radial basis function is a Gaussian function: , Shape parameters The method for determining this is as follows: based on the average spacing between measurement points of the high-precision reference equipment. calculate: , in As an adjustable parameter, its value is objectively determined using leave-one-out cross-validation (LOOCV). The specific steps are as follows: For candidate value set Perform the following operations in sequence: Remove one high-precision reference measurement point, and utilize the remaining... A radial basis function error field is constructed for each measurement point, and then the interpolation residuals at the removed points are calculated. This process is repeated for all points. After measuring each point, calculate the... The value corresponds to the root mean square of the cross-validation residuals. Select the value that minimizes the root mean square of the cross-validation residuals. The value is used as the final value. When the number of measurement points is small ( When <10), it can be The candidate range is expanded to 0.3–3.0, while the step size remains unchanged, to ensure sufficient search space. In this embodiment, the number of measurement points… =20, and the optimal value is obtained by calculating using the above method. =1.0, corresponding to .

[0039] Example of high-precision benchmark calibration: 20 benchmark markers are evenly distributed within the capture area, spatially covering the entire capture area (4m×3m×2.5m), with an average spacing between adjacent markers. ≈0.8m. Using a total station (nominal angle accuracy 0.5, distance accuracy 0.6mm + 1ppm), the reference coordinates of each benchmark point were measured. Simultaneously, the optical motion capture system performs 3D reconstruction of these reference markers, and obtains the final result after first-level optimization and correction. Calculate the error vector. The average mold length is 1.85 mm and the maximum mold length is 3.42 mm, indicating that the system has obvious spatial correlation error.

[0040] Pick =1.0, then A radial basis function interpolation model is constructed, and the weights are determined by solving the following system of linear equations. : , in , It is a matrix of polynomial terms. These are the polynomial coefficients. After solving, a second-level correction is performed on all marked points: , Verification results: Five additional verification points not involved in interpolation modeling were selected within the capture area. Before correction, the average error was 1.92 mm; after correction, the average error decreased to 0.31 mm, representing an accuracy improvement of approximately 6 times. (Number of measurement points on the high-precision reference equipment) There should be no fewer than 5, and their spatial distribution should cover the entire capture area as much as possible to ensure the stability of the error field interpolation.

[0041] Example 4, Special Scene Correction (1) Coordinate calculation based on virtual image of plane mirror in occluded scene Enabling special scene correction requires first determining the triggering conditions, as follows: (a) Occlusion recovery conditions: The marker point is occluded, resulting in no effective observation for at least 5 consecutive frames, and the plane mirror is located within the capture area, with the mirror normal vector pointing towards the spatial region where the occluded marker point is located, and the plane mirror flatness is better than 0.1 mm. At the same time, the virtual image of the marker point in the plane mirror is observed by at least two cameras. When all the above conditions are met, coordinate calculation based on the virtual image of the plane mirror is enabled.

[0042] (b) Outlier detection conditions: The marker point has been continuously tracked for at least 10 frames, the Kalman filter has converged, and there are valid observation coordinates in the current frame. When all the above conditions are met, the Mahalanobis distance is calculated and outlier detection is performed.

[0043] If any of the above conditions are not met, skip the corresponding special scene correction and directly output the first or second correction coordinates.

[0044] When a marker point is occluded, traditional optical motion capture systems cannot directly observe it. This invention utilizes the principle of plane mirror reflection to calculate the true coordinates by capturing the virtual image of the marker point in the plane mirror.

[0045] Let the equation of the plane containing the plane mirror be: , in For unit normal vector, This represents the plane offset. The plane mirror reflection matrix is: , in The identity matrix. If the system captures the virtual image coordinates of the occluded marker point in the plane mirror... Then the true coordinates satisfy: , because It is an integrand matrix, that is ,and Therefore, the actual coordinates are: , The method for obtaining the plane equation of a plane mirror is as follows: A spherical reflective marker with a diameter of 10 mm is pasted at each of the four corners of the mirror surface and at the intersection of its two diagonals (a total of 5 markers, ensuring they are not collinear and cover most of the mirror surface). The three-dimensional coordinates of these 5 markers are reconstructed using a calibrated optical motion capture system. Since the mirror surface is planar, the least squares plane fitting method is used to solve for the plane equation. Specifically, let the three-dimensional coordinates of the 5 marker points be... Constructing a matrix ,right Perform singular value decomposition (SVD), and take the right singular vector corresponding to the smallest singular value as the unit normal vector of the plane. The estimated value. Then through Calculate the plane offset. The root mean square of the plane fitting residual should be less than 0.05 mm; otherwise, the points need to be re-marked or the plane mirror replaced. This method can effectively recover the 3D coordinates of marker points even when the marker points are occluded but visible in the plane mirror.

[0046] Specific implementation example of mirror calibration: A high-precision plane mirror (1.2 m × 0.8 m, flatness better than 0.05 mm) is placed on one side of the capture area. During calibration, six reflective markers are attached to the edge of the mirror. The three-dimensional coordinates of these markers are reconstructed using an optical system, and the plane equation of the mirror is obtained through least-squares plane fitting. In this embodiment, the fitted plane equation is: (World coordinate system unit: meter).

[0047] In one experiment, a marker on a rigid body was obscured by the experimenter's body, but its virtual image in a plane mirror was simultaneously observed by three cameras. The system reconstructed the coordinates of the virtual image. m. Calculate the true coordinates using the formula above. m. Compared with the Kalman filter prediction value (1.198, 0.809, 1.111)m of the trajectory of the marker point before occlusion, the deviation is only 7.2 mm, which verifies the effectiveness of the mirror restoration method.

[0048] (2) Identification and removal of dynamic anomaly markers During dynamic capture, due to occlusion, reflection, or interference with the light source, the system may generate outliers (flying points) that deviate significantly from the true trajectory. This invention employs Kalman filtering to predict and detect outliers in the trajectory of the marked points.

[0049] Let the state vector of the marker point be... ,in In three-dimensional coordinates, Let the velocity be denoted by . The state transition equation is: , in Here is the state transition matrix. To predict the covariance matrix, Let be the process noise covariance matrix. The observation equation is: , in For the observation matrix, The covariance matrix of the observation noise is: The Kalman gain is: , Status updated to: , The predicted covariance matrix is ​​updated as follows: , Calculate the actual observed coordinates With predicted coordinates Mahalanobis distance between them: , in To predict the residual covariance matrix.

[0050] like Greater than the preset threshold If the observed coordinates are not found, they are considered outliers and are either removed from the list or predicted coordinates are used. Replace it. Threshold The determination method is based on the statistical distribution characteristics of the Mahalanobis distance: when the Kalman filter is in a stable tracking state, the Mahalanobis distance... Theoretically, it follows a chi-square distribution with 3 degrees of freedom (because the coordinate vector is three-dimensional). Let the expected false detection rate (the probability of misclassifying a normal point as an outlier) be... Then the threshold Determined by the quantiles of the chi-square distribution: , in For a chi-square distribution with 3 degrees of freedom quantiles. For example, take When =0.01, ≈4.11; take When =0.001, ≈4.64. In practical applications, if the observation noise is high or the accuracy of the motion model is low, the value can be appropriately increased. To reduce the false positive rate; if strict requirements are placed on outlier removal, the cost can be reduced. In this embodiment, we take... =0.01, corresponding to ≈4.11, used in actual application =4.1.

[0051] An example of outlier removal: In a human gait capture experiment, the sampling frequency of the marker points is 120 Hz. A state transition matrix is ​​set. ,in Process noise covariance matrix Based on the maximum acceleration setting at the marker point, take ,in Observation noise covariance matrix ,Pick This corresponds to the reconstruction accuracy after system-level calibration.

[0052] In one experiment, due to another experimenter's accidental passage, a marker point experienced a significant deviation in frame 356. The Mahalanobis distance between the observed coordinates and the Kalman filter predicted coordinates in this frame is... If the value is much greater than the threshold of 3, the system automatically identifies the point as an anomaly and replaces it with predicted coordinates. The replaced trajectory is smooth and continuous with the trajectory of adjacent frames, without any jumps.

[0053] Example 5, Adaptive Weighted Fusion After the above correction steps, multiple correction results may exist simultaneously. This invention performs adaptive weighted fusion based on the uncertainty of each correction result. Let the first correction coordinate, second correction coordinate, and third correction coordinate be respectively... The corresponding uncertainties are respectively .

[0054] The formula for calculating the weight is: , Final output coordinates: , If a certain calibration coordinate is missing (e.g., no high-precision reference equipment was used or no occlusion occurred), its corresponding weight is set to 0, and the remaining weights are normalized.

[0055] The method for determining uncertainty is as follows: The root mean square error of the residuals, obtained from the first-level optimization correction, is estimated as follows: ; The nominal accuracy of high-precision reference equipment Interpolation residuals with radial basis functions Synthesis, i.e. ; The square root estimate of the trace of the prediction covariance matrix obtained by Kalman filtering, i.e. .

[0056] A concrete example of fusion: At a certain moment, a marker point simultaneously possesses three correction results. The uncertainties calculated using the above formula are respectively... =0.12mm, =0.28mm, =0.35mm. Therefore, the weights are as follows: , The final fusion coordinates are Since the uncertainty of the first calibration coordinate is the smallest, the fusion result relies more on the first-level optimization calibration. At the same time, by appropriately utilizing the information from high-precision benchmark calibration and dynamic filtering calibration, optimal accuracy is achieved.

[0057] To verify the fusion effect, a standard rigid body was placed at the center of the capture area, and the true distance between its marker points was measured using a total station. Before calibration, the root mean square error of the reconstructed distance of the optical system was 0.62 mm; after system-level calibration and rigid body constraint optimization, the error was reduced to 0.15 mm; after high-precision benchmark calibration, it was reduced to 0.09 mm; and after adaptive weighted fusion, it was reduced to 0.07 mm, which is better than any single calibration method.

[0058] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for correcting the three-dimensional coordinates of a marker point in optical motion capture, characterized in that, Includes the following steps: Step S1: Swing a T-shaped calibration rod with known marker spacing within the capture area to acquire multiple frames of multi-view images. Based on multi-view geometric constraints, use bundle adjustment to jointly optimize the camera's intrinsic and extrinsic parameters and lens distortion coefficients. Obtain the intrinsic and extrinsic parameters and lens distortion coefficients of each camera, establish an initial 3D reconstruction model, and obtain the initial 3D coordinates of the marker points. Step S2: Perform first-level optimization and correction on the initial three-dimensional coordinates using known rigid body constraints to obtain the first corrected coordinates; Step S3: Measure the true three-dimensional coordinates of at least some of the marker points using a high-precision reference device, compare the measured values ​​with the first corrected coordinates, construct a local error correction model using radial basis function interpolation, and perform a second-level correction on all marker points to obtain the second corrected coordinates; Step S4: Perform special scene correction on the first corrected coordinates to obtain the third corrected coordinates; the special scene correction includes coordinate calculation based on the virtual image of the plane mirror in occluded scenes and / or dynamic anomaly marker identification and removal; wherein, the triggering condition for occlusion recovery is that the marker is missing for at least 5 consecutive frames and the virtual image of the marker in the plane mirror is observed by at least two cameras; the triggering condition for anomaly detection is that the marker has valid observation coordinates and the Kalman filter has been continuously tracked for at least 10 frames; Step S5: Adaptively weightedly fuse the first, second, and third correction coordinates to output the final three-dimensional coordinates of the marker point; the adaptive weighted fusion is as follows: based on the uncertainty of each of the first, second, and third correction coordinates, calculate the weight of each correction coordinate in a manner that the weight is inversely proportional to the square of the uncertainty, and sum the weights to obtain the final output coordinates.

2. The method for correcting the three-dimensional coordinates of a marker point in optical motion capture according to claim 1, characterized in that, The system-level calibration in step S1 includes: By swinging a T-shaped calibration pole with a known spacing between marker points within the capture area, multiple frames of multi-view images are acquired. Extract the two-dimensional coordinates of the marked points in the multi-frame, multi-view images; Based on multi-view geometric constraints, the camera intrinsic parameters, extrinsic parameters, and lens distortion coefficients are jointly optimized using the bundle adjustment method to minimize the sum of reprojection error and calibration rod distance constraint error. The objective function is: ， In the formula, For the first The marker point at the th The observed two-dimensional coordinates in a camera image Projection function, The first The intrinsic parameter matrix, rotation matrix, and translation vector of each camera. To mark the three-dimensional coordinates of the point, The known distance between the marked points on the calibration rod. These are the distance constraint weighting coefficients.

3. The method for correcting the three-dimensional coordinates of a marker point in optical motion capture according to claim 1, characterized in that, The known rigid body constraint mentioned in step S2 is that the distance between the marked points on the rigid body is known and remains unchanged; The first-level optimization correction includes constructing a distance-constrained objective function: ， In the formula, The first one to be optimized Three-dimensional coordinates of each marker point For the first The initial three-dimensional coordinates of the marker points Mark the point and The known distance between them For a set of distance-constrained pairs, The regularization coefficient is used. The Levenberg-Marquardt algorithm is used to iteratively solve the distance constraint objective function to obtain the first corrected coordinates.

4. The method for correcting the three-dimensional coordinates of a marker point in optical motion capture according to claim 1, characterized in that, The high-precision reference equipment mentioned in step S3 includes a total station, a laser tracker, or a robotic arm; The local error correction model is constructed as follows: Calculate the measured values ​​of high-precision reference equipment With the first correction coordinate Error vector between: ， The error field is constructed using radial basis function interpolation: ， In the formula, The number of measurement points for high-precision reference equipment. Center of radial basis functions For radial basis functions, These are linear polynomial terms; The weights are determined by solving a system of linear equations. Then, the error field is applied to all marked points for a second-level correction.

5. The method for correcting the three-dimensional coordinates of a marker point in optical motion capture according to claim 4, characterized in that, The radial basis function is a Gaussian function: ， In the formula, the shape parameter Based on the average spacing between measurement points of high-precision reference equipment Confirmed, and , The value of γ is determined by leave-one-out cross-validation.

6. The method for correcting the three-dimensional coordinates of a marker point in optical motion capture according to claim 1, characterized in that, The coordinate deduction based on the virtual image of the plane mirror in the occlusion scene described in step S4 includes: Obtain the plane equation of a plane mirror ,in For unit normal vector, This is the plane offset; Calculate the reflection matrix: , Obtain the virtual image coordinates of the occluded marker point in the plane mirror. Then the true coordinates for ， The true coordinates This serves as the three-dimensional coordinate of the marker point during the occlusion period.

7. The method for correcting the three-dimensional coordinates of a marker point in optical motion capture according to claim 1, characterized in that, The dynamic anomaly marker identification and removal in step S4 includes: The predicted coordinates are obtained by predicting the trajectory of the marked points using Kalman filtering or extended Kalman filtering. and predicting covariance matrix ; Calculate the actual observed coordinates Mahalanobis distance between the predicted coordinates and the target coordinates: ， If the Mahalanobis distance is greater than the preset threshold If the actual observed coordinates are not found, they are determined to be outliers and are either removed or replaced with predicted coordinates. The preset threshold Determined by chi-square distribution quantiles: ， in For a chi-square distribution with 3 degrees of freedom Quantiles This represents the expected false positive rate.

8. The method for correcting the three-dimensional coordinates of a marker point in optical motion capture according to claim 1, characterized in that, The adaptive weighted fusion described in step S5 includes: Based on the uncertainties of the first, second, and third correction coordinates Calculate the weights: ， Final output coordinates: ， In the formula, The first calibration coordinate, The second correction coordinate. The third correction coordinate is used; if a correction coordinate is missing, its corresponding weight is set to 0, and the remaining weights are normalized.

9. A method for correcting the three-dimensional coordinates of a marker point in optical motion capture according to claim 8, characterized in that, The uncertainty The method for determining it is as follows: Estimation of the root mean square error of the residuals corrected by the first-level optimization; It is synthesized from the nominal accuracy of the high-precision reference equipment and the interpolation residual of the radial basis function; Square root estimate of the trace of the predicted covariance matrix by Kalman filtering.

10. An optical motion capture system, comprising multiple cameras, a processor, and a memory, characterized in that, The memory stores a computer program, and when the processor executes the computer program, it implements the method for correcting the three-dimensional coordinates of a marker point in optical motion capture as described in any one of claims 1 to 9.

Images