Seamless tracking system and method based on multi-site motion capture cameras

Abstract

The present invention relates to the field of motion capture technology, and specifically to a seamless tracking system and method based on multi-site motion capture cameras. The method includes the following steps: arranging at least two groups of motion capture cameras and deploying them in independent, isolated sites, each group of motion capture cameras covering the entire space of its site, and collecting motion data of a rigid body in a local coordinate system; establishing a communication relationship between a control end and each group of motion capture cameras, each group of motion capture cameras completing an independent calibration of the local coordinate system in its own site, determining the motion data collected by each group of motion capture cameras, and establishing a local coordinate system; establishing a conversion matrix from each group of local coordinate systems to a unified global coordinate system by presetting the physical position relationship between the sites or reference markers at the junction; and ensuring the continuity and accuracy of tracking through coordinate conversion and data fusion, thereby realizing seamless tracking of rigid bodies between sites without a public field of view.

Description

Technical Field

[0001] The present invention relates to the field of motion capture technology, and in particular to a seamless tracking system and method based on multi-site motion capture cameras. Background Art

[0002] Traditional motion capture systems typically rely on the overlapping fields of view of multiple cameras for joint calibration and target tracking. However, in certain application scenarios, such as when cameras need to be distributed in completely isolated physical spaces, this approach, which relies on overlapping fields of view, is no longer applicable. Specifically, existing technologies have the following shortcomings:

[0003] Some attempts to address this issue have employed independent subsystem operation, meaning that each isolated area's motion capture system operates independently. However, this approach prevents cross-area tracking continuity, leading to tracking interruptions when a rigid body moves from one location to another. Another common approach relies on additional sensors (such as inertial sensors) to assist with positioning. This requires processing data from different sensors, increasing the system's computational burden and the risk of error accumulation.

[0004] Existing multi-site motion capture systems often fail to achieve seamless integration when handling rigid body cross-site tracking, resulting in discontinuity and decreased accuracy of tracking data.

[0005] To this end, we propose a seamless tracking system and method based on multi-site motion capture cameras to solve the above problems. Summary of the Invention

[0006] The object of the present invention is to provide a seamless tracking system and method based on multi-site motion capture cameras to solve the problems raised in the above background technology.

[0007] To achieve the above-mentioned object, the present invention provides the following technical solution: a seamless tracking method based on multi-site motion capture cameras, the method comprising the following steps:

[0008] At least two groups of motion capture cameras are deployed in separate, isolated locations. Each group of motion capture cameras covers the entire space of the location and collects motion data of the rigid body in the local coordinate system.

[0009] Establish communication between the control terminal and each group of motion capture cameras. Each group of motion capture cameras completes independent calibration of the local coordinate system in its own field, determines the motion data collected by each group of motion capture cameras, and establishes the local coordinate system.

[0010] By presetting the physical position relationship between sites or reference markers at the junction, a transformation matrix from each set of local coordinate systems to a unified global coordinate system is established;

[0011] When the rigid body approaches the boundary of the venue, the timing of the rigid body entering the new venue is predicted based on the motion trajectory, triggering the target switch. The coordinate system of the rigid body is switched to obtain the local coordinates of the rigid body in the new venue. These coordinates are used as the initial tracking parameters of the rigid body in the new venue, and the tracking rights are transferred to the motion capture camera group in the new venue.

[0012] The motion state of the rigid body before switching is dynamically fused with the rigid body state collected in real time by the new motion capture camera group to obtain the motion trajectory of the rigid body.

[0013] Preferably, the step of completing independent calibration of the local coordinate system of each group of motion capture cameras in their respective locations includes:

[0014] Based on the calibration plate, multiple sets of images are taken from different angles in each site;

[0015] The coordinates of the feature points on the calibration plate are extracted at the same time, and the intrinsic and extrinsic parameters of the camera are calculated. The intrinsic parameters include the focal length and the coordinates of the principal point, and the extrinsic parameters include the rotation matrix and the translation vector.

[0016] Preferably, the step of establishing a conversion matrix from each group of local coordinate systems to a unified global coordinate system by presetting the physical position relationship between the sites includes:

[0017] Obtain the relative position relationship between each site, where the position relationship includes the translation vector of the origin of the local coordinate system in the global coordinate system The rotation angle of the local coordinate system relative to the global coordinate system;

[0018] Define the transformation matrix from the local coordinate system to the global coordinate system based on the measurement data , where the measurement data includes the rotation matrix and translation vectors , the corresponding transformation matrix is: ,in, is a 3×3 rotation matrix, is a 3×1 translation vector; for the point in the local coordinate system , its global coordinates Calculation by homogeneous coordinate transformation ;in, represents the rotation of the local coordinate system relative to the global coordinate system, Represents the offset of the origin of the local coordinate system in the global coordinate system, Represents the coordinates of a point in the global coordinate system.

[0019] Preferably, the step of establishing a conversion matrix from each set of local coordinate systems to a unified global coordinate system by using reference markers at the boundaries between preset sites includes:

[0020] Set up fixed rigid bodies or markers at the intersection of the venues that can be detected simultaneously by the motion capture cameras in adjacent venues;

[0021] Each group of cameras collects the coordinates of the reference markers in their local coordinate system: The motion capture camera group detects the coordinates of the reference marker in the local coordinate system of site A. ,site The motion capture camera group detects the coordinates of the same reference marker in the local coordinate system of site B. ;

[0022] If the true coordinates of the reference marker in the global coordinate system are known , then directly solve the transformation matrix , for the venue : , for the venue Similarly, among them, Indicates the venue The transformation matrix from the local coordinate system to the global coordinate system;

[0023] If the global coordinates are unknown, but the local coordinates of the marker in the adjacent site are known and , then the relative transformation matrix is ​​solved by the least squares method The corresponding formula is ; Associate the transformation matrix of each local coordinate system to the global coordinate system. If the site The transformation matrix is ,site for ,but arrive The conversion relationship is ,in, Indicates the venue Arrive at the venue The direct conversion matrix, Indicates the venue The transformation matrix to the global coordinate system, Indicates the venue Transformation matrix to the global coordinate system.

[0024] Preferably, when the rigid body approaches the boundary of the field, predicting the timing of the rigid body entering the new field based on the motion trajectory, triggering target switching, switching the coordinate system of the rigid body to obtain the local coordinates of the rigid body in the new field, and using the local coordinates as the initial tracking parameters of the rigid body in the new field include:

[0025] Deploy boundary sensors at the intersection of the site to monitor in real time whether the rigid body is approaching the boundary and set a unique identity for the rigid body;

[0026] When a rigid body enters the detection range of the boundary sensor, a trigger signal is generated and sent to the switching logic module of the central controller;

[0027] Predict the future position of a rigid body based on its historical motion data; if the predicted trajectory shows that the rigid body will enter an adjacent field within a threshold time, trigger switching preparation in advance;

[0028] The motion capture camera at the new location confirms the identity of the rigid body based on its unique identifier. The controller sends a stop tracking command to the motion capture camera group at the current location and a start tracking command to the motion capture camera group at the new location. The motion capture camera group at the new location immediately locks onto the target rigid body based on its unique identifier.

[0029] Switch the coordinate system of the target rigid body to obtain the local coordinates of the rigid body in the new site as the initial tracking parameters.

[0030] Preferably, the step of predicting the future position of the rigid body based on the historical motion data of the rigid body comprises:

[0031] Get the rigid body from the mocap camera group in the time window arrive The motion state of the body, including the rigid body in time Location , rigid body in time Speed , rigid body in time Acceleration , assuming constant velocity and zero acceleration, the state equation is ,in, is the time interval, is the process noise; according to the current state And the state equation, calculate the state at the next moment , the corresponding calculation formula is ,Will As the position of the rigid body at the next moment.

[0032] Preferably, the specific content of switching the coordinate system of the rigid body to obtain the local coordinates of the rigid body in the new site includes:

[0033] Before switching, get the final coordinates of the rigid body in the local coordinate system of the current site , through the preset transformation matrix , convert it to the coordinates of the rigid body in the global coordinate system , the corresponding formula is ;

[0034] The global coordinates Convert to new site local coordinates , the corresponding calculation formula is ,in, express The rotation matrix from the site local coordinate system to the global coordinate system, express The translation vector of the origin of the site local coordinate system in the global coordinate system, express Homogeneous transformation matrix from site to global coordinate system, represents the coordinates of the rigid body in the local coordinate system of the new site, express The rotation matrix from the site local coordinate system to the global coordinate system, express The translation vector of the origin of the site local coordinate system in the global coordinate system, Represents the inverse of the rotation matrix.

[0035] Preferably, the step of performing dynamic data fusion on the motion state of the rigid body before switching and the rigid body state captured in real time by the new motion capture camera group to obtain the motion trajectory of the rigid body comprises:

[0036] Obtaining the motion state information of the rigid body before it leaves the current venue through the motion capture camera group of the current venue, wherein the motion state information includes the rigid body position and rigid body velocity;

[0037] Initialize the state estimate of the filter based on the motion state before switching; , ;

[0038] The motion capture camera group at the new site collects the motion state information of the rigid body in real time, wherein the motion state information includes the position and velocity of the rigid body;

[0039] Update the state estimate of the rigid body , the fused state is estimated as the motion state of the rigid body in the new venue, and the motion capture camera group in the new venue is estimated from the fused state Start tracking the rigid body;

[0040] The state estimation after each fusion Perform associated storage to form the motion trajectory of the rigid body.

[0041] A seamless tracking system based on multi-site motion capture cameras, applied to any of the above-mentioned seamless tracking methods based on multi-site motion capture cameras, includes:

[0042] A deployment module is used to set up at least two groups of motion capture cameras, each of which is deployed in an isolated field. Each group of motion capture cameras covers the entire space of the field where it is located and collects motion data of the rigid body in the local coordinate system.

[0043] The calibration module is used to establish a communication relationship between the control terminal and each group of motion capture cameras. Each group of motion capture cameras completes independent calibration of the local coordinate system in its own field, determines the motion data collected by each group of motion capture cameras, and establishes the local coordinate system;

[0044] The conversion module is used to establish the conversion matrix from each set of local coordinate systems to the unified global coordinate system by presetting the physical position relationship between sites or reference markers at the intersection;

[0045] The switching module is used to predict the timing of the rigid body entering the new field based on the motion trajectory when the rigid body approaches the field boundary. It triggers the target switching and switches the coordinate system of the rigid body to obtain the local coordinates of the rigid body in the new field. These coordinates are used as the initial tracking parameters of the rigid body in the new field, and the tracking rights are transferred to the motion capture camera group in the new field.

[0046] The fusion module is used to dynamically fuse the motion state of the rigid body before switching with the rigid body state collected in real time by the new motion capture camera group to obtain the motion trajectory of the rigid body.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] Through independent calibration of each site and global coordinate system mapping, tracking rights can be quickly switched when a rigid body enters a new site, enabling seamless tracking of rigid bodies between sites without a public view. Precise coordinate conversion and data fusion ensure tracking continuity and accuracy, effectively reducing error accumulation caused by site switching and sensor noise, and improving the accuracy of motion trajectory capture. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 Schematic diagram of the method flow of the present invention;

[0051] Figure 2 It is a system structure block diagram of the present invention;

[0052] Figure 3 This is a position diagram of the motion capture camera of the present invention. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0054] Example

[0055] See also Figures 1 to 3 The present invention provides a technical solution for a seamless tracking system and method based on multi-site motion capture cameras: a seamless tracking method based on multi-site motion capture cameras, comprising the following steps:

[0056] S1: Set up at least two groups of motion capture cameras, each deployed in an isolated venue. Each group of motion capture cameras covers the entire space of the venue and collects motion data of the rigid body in the local coordinate system.

[0057] S2: Establishing a communication relationship between the control terminal and each group of motion capture cameras. Each group of motion capture cameras completes independent calibration of the local coordinate system in its own field, determines the motion data collected by each group of motion capture cameras, and establishes the local coordinate system.

[0058] Each set of motion capture cameras completes independent calibration of its local coordinate system within its own location. The steps include: capturing multiple sets of images from different angles within the location using a calibration plate; extracting the coordinates of feature points on the calibration plate at the same time, and calculating the camera's intrinsic and extrinsic parameters. Intrinsic parameters include focal length and principal point coordinates, while extrinsic parameters include rotation matrices and translation vectors. Each set of cameras must support time synchronization (e.g., using the PTP protocol) to ensure temporal consistency of data acquisition across different locations. This is crucial for accurately predicting the motion trajectory of rigid bodies and achieving data fusion.

[0059] Specifically, each group of motion capture cameras completes local coordinate system calibration within its respective venue. This step ensures that the cameras within each venue can accurately determine the position of the rigid body within its local area. Accurate calibration within each venue lays the foundation for subsequent global mapping and cross-domain tracking. During calibration, the camera's intrinsic parameters (such as focal length, principal point coordinates, etc.) and extrinsic parameters (such as rotation matrix and translation vector) are recorded to convert pixel coordinates in the image into three-dimensional coordinates in the local coordinate system. Based on actual needs, the motion capture cameras are divided into multiple groups, each deployed in a separate venue (e.g., ground or underwater), resulting in ground cameras and underwater cameras. The ground cameras are used for the subsequent motion capture cameras in Site A, while the underwater cameras are used for the subsequent motion capture cameras in Site B. The collected data is transmitted to a central controller via data transmission lines. The central controller processes and analyzes the data, implementing global coordinate mapping, target switching, and data fusion, thereby achieving seamless tracking of the rigid body's motion. The cameras within a group should cover the entire space of the venue to ensure comprehensive capture of the rigid body's motion within that venue. For example, in an aquatic environment, visible light cameras can be used to track buoy-type rigid markers; in an underwater environment, waterproof infrared cameras can be used to track pressure-resistant rigid markers. Each camera group undergoes independent internal calibration to determine its local coordinate system. This calibration process can use traditional camera calibration methods, such as the Zhang Zhengyou calibration method. By capturing a calibration plate of known geometry and dimensions, multiple sets of images are acquired from different angles. Image processing algorithms are then used to extract the coordinates of feature points on the plate, from which the camera's intrinsic and extrinsic parameters are calculated.

[0060] S3: By presetting the physical position relationship between sites or reference markers at the junction, a transformation matrix is ​​established from each set of local coordinate systems to a unified global coordinate system;

[0061] The steps of establishing a transformation matrix from each set of local coordinate systems to a unified global coordinate system by presetting the physical position relationship between the sites or the reference markers at the junction include: when based on the physical position relationship between the sites, obtaining the relative coordinate offset by measurement and calculation; when based on the reference markers at the junction, determining the coordinate transformation relationship by identifying the positions of the markers in the two sites; and transforming the coordinates in the local coordinate system to the global coordinate system by using a rotation and translation matrix;

[0062] The steps of establishing the transformation matrix of each set of local coordinate systems to the unified global coordinate system by presetting the physical position relationship between the sites include: obtaining the relative position relationship between the sites, wherein the position relationship includes the translation vector of the origin of the local coordinate system in the global coordinate system The rotation angle of the local coordinate system relative to the global coordinate system; based on the measurement data, define the transformation matrix from the local coordinate system to the global coordinate system , rotation matrix With translation vector , where the rotation matrix Convert to a 3×3 rotation matrix via Euler angles (such as rotation around the ZYX axis) or quaternion; translation vector : Obtained directly from measurement, the conversion matrix is ​​in the form of: ,in, is a 3×3 rotation matrix, is a 3×1 translation vector; for the point in the local coordinate system , its global coordinates Calculation by homogeneous coordinate transformation ;in, represents the rotation of the local coordinate system relative to the global coordinate system, Represents the offset of the origin of the local coordinate system in the global coordinate system, Represents the coordinates of a point in the global coordinate system;

[0063] The steps of establishing the transformation matrix of each group of local coordinate systems to the unified global coordinate system by setting reference markers at the intersection of the venues include: setting fixed rigid bodies or high-precision markers (such as chessboards, LED beacons) at the intersection of the venues to ensure that they can be detected by the motion capture cameras of adjacent venues at the same time; each group of cameras collects the coordinates of the reference markers in their local coordinate systems respectively: the venue The motion capture camera group detects the coordinates of the reference marker in the local coordinate system of site A. ,site The motion capture camera group detects the coordinates of the same reference marker in the local coordinate system of site B. ; If the true coordinates of the reference marker in the global coordinate system are known , then directly solve the transformation matrix , for the venue : ,in, Represents the rotation matrix of the local coordinate system of site A relative to the global coordinate system, which is used to describe the rotation relationship between the coordinate direction of site A and the global coordinate direction. It represents the translation vector of the origin of the local coordinate system of site A in the global coordinate system, which is used to determine the position of the origin of the coordinate system of site A in the global coordinate system. Similarly, among them, Indicates the venue The transformation matrix from the local coordinate system to the global coordinate system; if the global coordinates are unknown, but the local coordinates of the marker in the adjacent site are known and , then the relative transformation matrix is ​​solved by the least squares method The corresponding formula is ; Associate the transformation matrix of each local coordinate system to the global coordinate system. If the site The transformation matrix is ,site for ,but arrive The conversion relationship is ,in, Indicates the venue Arrive at the venue The direct conversion matrix, Indicates the venue The transformation matrix to the global coordinate system, Indicates the venue Transformation matrix to the global coordinate system;

[0064] The coordinate transformation relationship between each group is calculated through manual input or automatic detection (such as reference points at junctions). If the physical position relationship between the venues is known, the relative coordinate offset can be obtained through measurement and calculation. If reference markers are set, the coordinate transformation relationship can be determined by identifying the position of the markers in the two venues. When the rigid body approaches the boundary of the venue, the system predicts the time when it will enter the new venue based on the motion trajectory. This can be achieved by analyzing the historical motion data of the rigid body and establishing a kinematic model. Once it is predicted that the rigid body is about to enter the new venue, the target switching command is triggered, and the new camera group immediately takes over the tracking based on the rigid body identification (such as specific optical markers, RFID). At the same time, the historical motion data is integrated with the newly collected data in real time to eliminate positioning jumps during switching.

[0065] S4: When the rigid body approaches the boundary of the field, the timing of the rigid body entering the new field is predicted based on the motion trajectory, triggering the target switch. The coordinate system of the rigid body is switched to obtain the local coordinates of the rigid body in the new field. These coordinates are used as the initial tracking parameters of the rigid body in the new field, and the tracking rights are transferred to the motion capture camera group in the new field.

[0066] When the rigid body approaches the boundary of the field, the timing of the rigid body entering the new field is predicted based on the motion trajectory, the target switching is triggered, and the coordinate system of the rigid body is switched to obtain the local coordinates of the rigid body in the new field. The steps used as the initial tracking parameters of the rigid body in the new field include: deploying boundary sensors (such as gratings, ultrasonic waves, infrared sensors) at the junction of the fields to monitor in real time whether the rigid body is approaching the boundary, and setting a unique identity for the rigid body; when the rigid body enters the detection range of the boundary sensor, a trigger signal is generated and sent to the switching logic module of the central controller; based on the historical motion data (speed, acceleration, direction) of the rigid body, through the kinematic model (such as A uniform velocity model, uniform acceleration model, or high-order dynamics model is used to predict the future position of the rigid body. If the predicted trajectory shows that the rigid body will enter an adjacent venue within a threshold time (e.g., 100ms), switching preparation is triggered in advance. The motion capture camera in the new venue confirms the identity of the rigid body based on the unique identifier corresponding to the rigid body. The control end sends a stop tracking command to the motion capture camera group in the current venue and a start tracking command to the motion capture camera group in the new venue. The motion capture camera group in the new venue immediately locks onto the target rigid body based on the unique identifier of the rigid body. The coordinate system of the target rigid body is switched to obtain the local coordinates of the rigid body in the new venue as the initial tracking parameters.

[0067] Auxiliary sensors (such as gratings and ultrasonic waves) can be deployed at the intersection to detect rigid bodies crossing the boundary. These sensors can provide additional information to help more accurately determine whether a rigid body has entered a new area, thereby improving the accuracy of target switching.

[0068] Based on the historical motion data (speed, acceleration, direction) of the rigid body, the steps of predicting the future position of the rigid body through a kinematic model (such as a uniform velocity model) include: obtaining the rigid body in the time window from the motion capture camera group arrive The motion state of the body, including the rigid body in time Location , rigid body in time Speed , rigid body in time Acceleration ,Select the model according to the rigid body motion characteristics. Taking the uniform velocity model as an example, assuming that the velocity is constant and the acceleration is zero, the state equation is ,in, is the time interval, is the process noise; according to the current state And model, calculate the next moment state , taking the uniform speed model as an example: the corresponding calculation formula is ,Will As the position of the rigid body at the next moment, it can predict the time when the rigid body crosses the field boundary with high precision, providing key input for seamless switching;

[0069] The specific content of switching the coordinate system of the rigid body to obtain the local coordinates of the rigid body in the new field includes: obtaining the final coordinates of the rigid body in the local coordinate system of the current field before switching , through the preset transformation matrix , convert it to the coordinates of the rigid body in the global coordinate system , the corresponding formula is , the global coordinates Convert to new site local coordinates , the corresponding calculation formula is , ensure that the position of the rigid body in the new coordinate system is continuous and avoid jumps, where express The rotation matrix from the site local coordinate system to the global coordinate system, express The translation vector of the origin of the site local coordinate system in the global coordinate system, express Homogeneous transformation matrix from site to global coordinate system, Indicates that the rigid body is in the new location ( Coordinates in the local coordinate system of the site, express The rotation matrix from the site local coordinate system to the global coordinate system, express The translation vector of the origin of the site local coordinate system in the global coordinate system, represents the inverse of the rotation matrix (the inverse of an orthogonal matrix is ​​equal to its transpose);

[0070] Assume that the rigid body Local coordinates of the site , transformation matrix parameters: is a matrix for rotating 30° around the Z axis, , then the global coordinates The calculation is as follows: .

[0071] When a rigid body enters a new arena, a target switch is triggered by spatiotemporal logic (sensors and motion prediction algorithms). Boundary sensors detect when the rigid body leaves the current arena and approaches the boundary of another arena, while the motion prediction algorithm predicts when it will enter the new arena based on the rigid body's historical trajectory. Once the target switch is triggered, tracking authority is transferred to the corresponding camera group. To ensure the accuracy of the switch, the rigid body can be assigned a unique identifier (such as a specific optical tag or RFID) so that the new camera group can quickly identify and continue tracking. The software processor can treat multiple camera groups as a single system, simply without a common viewing area, or with a viewing area above or below the water surface, to identify the rigid body as a single system. An IMU can also be added to the rigid body. If no camera is available for positioning, inertial fusion can be used. By pre-setting the physical positional relationships between the arenas (such as relative coordinate offsets) or by using reference markers (such as static rigid bodies fixed at the intersection), a transformation matrix is ​​established from each local coordinate system to a unified world coordinate system. This transformation matrix is ​​key to cross-domain tracking, converting local coordinates within different arenas into global coordinates, making tracking data comparable across arenas.

[0072] S5: Dynamically fuse the motion state of the rigid body before switching with the rigid body state captured in real time by the new motion capture camera group to obtain the motion trajectory of the rigid body;

[0073] The steps of dynamically fusing the motion state of the rigid body before the switching with the state of the rigid body acquired in real time by the new motion capture camera group to obtain the motion trajectory of the rigid body include: obtaining the motion state information of the rigid body before leaving the current site through the motion capture camera group of the current site, wherein the motion state information includes the position and velocity of the rigid body; initializing the state estimation of the filter according to the motion state before the switching; , ;According to the new site's motion capture camera group, the motion state information of the rigid body is collected in real time, where the motion state information includes the rigid body's position and velocity; the state of the rigid body is estimated and updated , the fused state is estimated as the motion state of the rigid body in the new venue, and the motion capture camera group in the new venue is estimated from the fused state Start to continue tracking the rigid body and estimate the state after each fusion Perform associated storage to form the motion trajectory of the rigid body;

[0074] The steps of updating the state estimation of the rigid body include: according to the state transfer matrix and the state estimate at the previous moment Predict the current state, the corresponding formula is , using the process noise covariance matrix and the error covariance matrix of the previous moment Predict the error covariance at the current moment, the corresponding calculation formula is , calculate the Kalman gain , the corresponding formula is, , using the Kalman gain and observation residuals Update the state estimate, the corresponding formula is , update the error covariance matrix according to the Kalman gain and the observation matrix, and the corresponding formula is , the updated state estimate As the state of the rigid body at the current moment, Indicates the current time The estimated state of the rigid body, Represents the state transition matrix, describing the state from time At the time changes, Indicates the previous moment The estimated value of the state, is the identity matrix, For the current moment The prior error covariance matrix of , which represents the uncertainty of the estimated state, For the previous moment The posterior error covariance matrix represents the uncertainty of the updated state, is the transpose of the state transfer matrix, is the process noise covariance matrix, which represents the statistical characteristics of the process noise; Represents the observation matrix, which maps the state vector to the observation vector, is the transpose of the observation matrix, is the observation noise covariance matrix, which represents the statistical characteristics of the observation noise; Indicates the current time The state update value of Indicates the current time The observed value of represents the predicted observation value, that is, the observation value calculated based on the predicted state; The observation residual represents the difference between the actual observation value and the predicted observation value; the state equation and observation equation of the rigid body are established, and the position and velocity are optimally estimated through Kalman filtering; for nonlinear and non-Gaussian systems, the particle filter algorithm is used to approximate the rigid body state distribution through random samples; during the switching process, historical data and new data are fused in real time to eliminate positioning jumps, which can effectively integrate the motion state of the rigid body between different venues and ensure the continuity and accuracy of tracking.

[0075] By leveraging the rigid body's pre-switching state (speed, direction) and combining it with real-time data from the new camera group, a smooth transition is achieved using a Kalman filter or particle filter algorithm. The Kalman filter algorithm establishes state and observation equations to optimally estimate the rigid body's position and velocity, thereby reducing positioning jumps caused by site switching. The particle filter algorithm is suitable for nonlinear and non-Gaussian systems, approximating the state distribution of the rigid body using a set of random samples (particles), similarly achieving smooth data fusion.

[0076] A seamless tracking system based on multi-site motion capture cameras, applied to any of the above-mentioned seamless tracking methods based on multi-site motion capture cameras, includes:

[0077] A deployment module is used to set up at least two groups of motion capture cameras, each of which is deployed in an isolated field. Each group of motion capture cameras covers the entire space of the field where it is located and collects motion data of the rigid body in the local coordinate system.

[0078] The calibration module is used to establish a communication relationship between the control terminal and each group of motion capture cameras. Each group of motion capture cameras completes independent calibration of the local coordinate system in its own field, determines the motion data collected by each group of motion capture cameras, and establishes the local coordinate system;

[0079] The conversion module is used to establish the conversion matrix from each set of local coordinate systems to the unified global coordinate system by presetting the physical position relationship between sites or reference markers at the intersection;

[0080] The switching module is used to predict the timing of the rigid body entering the new field based on the motion trajectory when the rigid body approaches the field boundary. It triggers the target switching and switches the coordinate system of the rigid body to obtain the local coordinates of the rigid body in the new field. These coordinates are used as the initial tracking parameters of the rigid body in the new field, and the tracking rights are transferred to the motion capture camera group in the new field.

[0081] The fusion module is used to dynamically fuse the motion state of the rigid body before switching with the rigid body state collected in real time by the new motion capture camera group to obtain the motion trajectory of the rigid body.

[0082] To address underwater refraction effects, a media compensation algorithm is introduced during calibration to correct image distortion. Because water has a different refractive index than air, light refracts as it passes through the water surface, causing image distortion. The media compensation algorithm measures information such as the water's refractive index and the angle of incidence of light to correct the image, thereby improving underwater motion capture accuracy.

[0083] The multi-site motion capture camera system includes at least two groups of motion capture cameras, a central controller and a bus for communication between the groups of motion capture cameras. The present invention does not need to rely on high-precision inertial sensors and only uses optical motion capture to achieve cross-domain tracking. Compared with the solution of using multiple inertial sensors, the cost is greatly reduced. The optical motion capture system can use existing camera equipment to achieve accurate tracking of rigid bodies through reasonable layout and algorithm optimization; and avoid tracking interruptions through motion prediction and data fusion. When the rigid body crosses the site boundary, the prediction algorithm is used to prepare in advance, and advanced filtering algorithms are used in the data fusion process to ensure the continuity and accuracy of the tracking. This ensures that there will be no obvious interruptions or jumps in the tracking process of the rigid body in the entire multi-site environment.

[0084] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0085] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A seamless tracking method based on multi-site motion capture cameras, characterized in that: The following steps are involved: At least two groups of motion capture cameras are deployed in separate, isolated locations. Each group of motion capture cameras covers the entire space of the location and collects motion data of the rigid body in the local coordinate system. Establish communication between the control terminal and each group of motion capture cameras. Each group of motion capture cameras completes independent calibration of the local coordinate system in its own field, determines the motion data collected by each group of motion capture cameras, and establishes the local coordinate system. By presetting the physical position relationship between sites or reference markers at the junction, a transformation matrix from each set of local coordinate systems to a unified global coordinate system is established; The step of establishing a conversion matrix from each set of local coordinate systems to a unified global coordinate system by presetting the physical position relationship between the sites includes: Obtain the relative position relationship between each site, where the position relationship includes the translation vector of the origin of the local coordinate system in the global coordinate system The rotation angle of the local coordinate system relative to the global coordinate system; Define the transformation matrix from the local coordinate system to the global coordinate system based on the measurement data , where the measurement data includes the rotation matrix and translation vectors , the corresponding transformation matrix is: ,in, is a 3×3 rotation matrix, is a 3×1 translation vector; for the point in the local coordinate system , its global coordinates Calculation by homogeneous coordinate transformation ;in, represents the rotation of the local coordinate system relative to the global coordinate system, Represents the offset of the origin of the local coordinate system in the global coordinate system, Represents the coordinates of a point in the global coordinate system; The step of establishing a conversion matrix from each set of local coordinate systems to a unified global coordinate system by using reference markers at the boundaries between preset sites includes: Set up fixed rigid bodies or markers at the intersection of the venues that can be detected simultaneously by the motion capture cameras in adjacent venues; Each group of cameras collects the coordinates of the reference markers in their local coordinate system: The motion capture camera group detects the coordinates of the reference marker in the local coordinate system of site A. ,site The motion capture camera group detects the coordinates of the same reference marker in the local coordinate system of site B. ; If the true coordinates of the reference marker in the global coordinate system are known , then directly solve the transformation matrix , for the venue : , for the venue Similarly, among them, Indicates the venue The transformation matrix from the local coordinate system to the global coordinate system; If the global coordinates are unknown, but the local coordinates of the marker in the adjacent site are known and , then the relative transformation matrix is ​​solved by the least squares method The corresponding formula is ; Associate the transformation matrix of each local coordinate system to the global coordinate system. If the site The transformation matrix is ,site for ,but arrive The conversion relationship is ,in, Indicates the venue Arrive at the venue The direct conversion matrix, Indicates the venue The transformation matrix to the global coordinate system, Indicates the venue Transformation matrix to the global coordinate system; When the rigid body approaches the boundary of the venue, the timing of the rigid body entering the new venue is predicted based on the motion trajectory, triggering the target switch. The coordinate system of the rigid body is switched to obtain the local coordinates of the rigid body in the new venue. These coordinates are used as the initial tracking parameters of the rigid body in the new venue, and the tracking rights are transferred to the motion capture camera group in the new venue. The motion state of the rigid body before switching is dynamically fused with the rigid body state collected in real time by the new motion capture camera group to obtain the motion trajectory of the rigid body.

2. The seamless tracking method based on multi-location motion capture cameras according to claim 1, characterized in that: The steps of completing independent calibration of the local coordinate system of each group of motion capture cameras in their respective locations include: Based on the calibration plate, multiple sets of images are taken from different angles in each site; The coordinates of the feature points on the calibration plate are extracted at the same time, and the intrinsic and extrinsic parameters of the camera are calculated. The intrinsic parameters include the focal length and the coordinates of the principal point, and the extrinsic parameters include the rotation matrix and the translation vector.

3. The seamless tracking method based on multi-location motion capture cameras according to claim 1, characterized in that: When the rigid body approaches the boundary of the field, the timing of the rigid body entering the new field is predicted based on the motion trajectory, the target switching is triggered, and the coordinate system of the rigid body is switched to obtain the local coordinates of the rigid body in the new field as the initial tracking parameters of the rigid body in the new field. The steps include: Deploy boundary sensors at the intersection of the site to monitor in real time whether the rigid body is approaching the boundary and set a unique identity for the rigid body; When a rigid body enters the detection range of the boundary sensor, a trigger signal is generated and sent to the switching logic module of the central controller; Predict the future position of a rigid body based on its historical motion data; if the predicted trajectory shows that the rigid body will enter an adjacent field within a threshold time, trigger switching preparation in advance; The motion capture camera at the new location confirms the identity of the rigid body based on its unique identifier. The controller sends a stop tracking command to the motion capture camera group at the current location and a start tracking command to the motion capture camera group at the new location. The motion capture camera group at the new location immediately locks onto the target rigid body based on its unique identifier. Switch the coordinate system of the target rigid body to obtain the local coordinates of the rigid body in the new site as the initial tracking parameters.

4. The seamless tracking method based on multi-location motion capture cameras according to claim 3, characterized in that: The step of predicting the future position of the rigid body based on the historical motion data of the rigid body comprises: Get the rigid body from the mocap camera group in the time window arrive The motion state of the body, including the rigid body in time Location , rigid body in time Speed , rigid body in time Acceleration , assuming constant velocity and zero acceleration, the state equation is ,in, is the time interval, is the process noise; according to the current state And the state equation, calculate the state at the next moment , the corresponding calculation formula is ,Will As the position of the rigid body at the next moment.

5. The seamless tracking method based on multi-location motion capture cameras according to claim 1, characterized in that: The specific contents of switching the coordinate system of the rigid body to obtain the local coordinates of the rigid body in the new site include: Before switching, get the final coordinates of the rigid body in the local coordinate system of the current site , through the preset transformation matrix , convert it to the coordinates of the rigid body in the global coordinate system , the corresponding formula is ; The global coordinates Convert to new site local coordinates , the corresponding calculation formula is ,in, express The rotation matrix from the site local coordinate system to the global coordinate system, express The translation vector of the origin of the site local coordinate system in the global coordinate system, express Homogeneous transformation matrix from site to global coordinate system, represents the coordinates of the rigid body in the local coordinate system of the new site, express The rotation matrix from the site local coordinate system to the global coordinate system, express The translation vector of the origin of the site local coordinate system in the global coordinate system, Represents the inverse of the rotation matrix.

6. The seamless tracking method based on multi-location motion capture cameras according to claim 1, characterized in that: The step of dynamically fusing the motion state of the rigid body before switching with the rigid body state captured in real time by the new motion capture camera group to obtain the motion trajectory of the rigid body comprises: Obtaining the motion state information of the rigid body before it leaves the current venue through the motion capture camera group of the current venue, wherein the motion state information includes the rigid body position and rigid body velocity; Initialize the state estimate of the filter based on the motion state before switching; , ; The motion capture camera group at the new site collects the motion state information of the rigid body in real time, wherein the motion state information includes the position and velocity of the rigid body; Update the state estimate of the rigid body , the fused state is estimated as the motion state of the rigid body in the new venue, and the motion capture camera group in the new venue is estimated from the fused state Start tracking the rigid body; The state estimation after each fusion Perform associated storage to form the motion trajectory of the rigid body.

7. A seamless tracking system based on multi-location motion capture cameras, applied to the seamless tracking method based on multi-location motion capture cameras as claimed in any one of claims 1 to 6, characterized in that: include: A deployment module is used to set up at least two groups of motion capture cameras, each of which is deployed in an isolated field. Each group of motion capture cameras covers the entire space of the field where it is located and collects motion data of the rigid body in the local coordinate system. The calibration module is used to establish a communication relationship between the control terminal and each group of motion capture cameras. Each group of motion capture cameras completes independent calibration of the local coordinate system in its own field, determines the motion data collected by each group of motion capture cameras, and establishes the local coordinate system; The conversion module is used to establish the conversion matrix from each set of local coordinate systems to the unified global coordinate system by presetting the physical position relationship between sites or reference markers at the intersection; The switching module is used to predict the timing of the rigid body entering the new field based on the motion trajectory when the rigid body approaches the field boundary. It triggers the target switching and switches the coordinate system of the rigid body to obtain the local coordinates of the rigid body in the new field. These coordinates are used as the initial tracking parameters of the rigid body in the new field, and the tracking rights are transferred to the motion capture camera group in the new field. The fusion module is used to dynamically fuse the motion state of the rigid body before switching with the rigid body state collected in real time by the new motion capture camera group to obtain the motion trajectory of the rigid body.

Images