Motion capture camera and third party camera synchronized triggering method and timestamp alignment system

Abstract

The application relates to the technical field of multi-camera synchronous triggering, and discloses a motion capture camera and third-party camera synchronous triggering method and a timestamp alignment system, the method comprising the following steps: connecting a PTP master clock, a motion capture camera triggered autonomously by PTP software synchronization, a third-party camera triggered passively by external hardware, and a synchronization box into the same PTP domain to realize timestamp alignment; monitoring the exposure time change of the third-party camera in real time through the synchronization box, dynamically calculating a delay compensation amount, adjusting a basic triggering moment, and aligning the exposure center of the third-party camera with the exposure center of the motion capture camera; the application solves the problem of physical acquisition moment alignment under a differentiated synchronous triggering mode, the motion capture camera is triggered autonomously by PTP software synchronization, the third-party camera is triggered by the hardware of the synchronization box, and the two completely different synchronous mechanisms realize the accurate alignment of the physical acquisition moment through the dynamic delay compensation method.

Description

Technical Field

[0001] This application relates to the field of multi-camera synchronization triggering technology, and in particular to a method for synchronizing motion capture cameras and third-party cameras, as well as a timestamp alignment system. Background Technology

[0002] In optical motion capture systems, multiple infrared motion capture cameras are typically deployed to simultaneously capture reflective markers on the surface of the object from different perspectives. The spatial coordinates of these markers are then calculated using a 3D reconstruction algorithm. To ensure reconstruction accuracy, all motion capture cameras must be strictly synchronized during exposure and acquisition; any discrepancy in acquisition time between any two cameras will directly lead to 3D reconstruction errors. Current motion capture systems primarily employ two methods for camera synchronization: Method 1 is hardware-triggered synchronization, where the main control computer or synchronization box sends synchronization pulse signals to all motion capture cameras via dedicated trigger cables. The cameras immediately begin exposure upon receiving the rising (or falling) edge of the trigger signal. This method offers high synchronization accuracy (up to microsecond levels), but suffers from drawbacks such as complex wiring, limited cable length, and the requirement for cameras to support external trigger interfaces. Method 2 is PTP software synchronization, based on the IEEE 1588 Precision Time Protocol. The motion capture camera receives PTP synchronization messages over a network, calibrates its internal clock, and then autonomously triggers exposure according to a preset frame rate (Free-running mode). This method requires no additional trigger cables, simplifies wiring, and achieves sub-microsecond synchronization accuracy, making it the mainstream synchronization solution for high-end motion capture systems. However, in practical applications, it is often necessary to fuse data between the motion capture system and third-party cameras. These third-party cameras include, but are not limited to, high-resolution RGB industrial cameras, ToF depth cameras, structured light cameras, and high-speed cameras. These third-party devices typically support PTP timestamp synchronization to record precise time information of image frames and support external hardware trigger input to control exposure timing via external pulse signals. However, they do not support the proprietary synchronization protocol of the motion capture system and cannot be directly connected to the motion capture synchronization network. Furthermore, their exposure times are usually significantly different from those of the motion capture camera, resulting in fundamental differences in their synchronization triggering methods and exposure physical characteristics.

[0003] When a motion capture camera and a third-party camera coexist in the same system, existing technical solutions have the following drawbacks: Firstly, they only align the trigger time without considering differences in the exposure process. In traditional solutions, the synchronization box sends a trigger pulse to the third-party camera at the calculated trigger time, but due to the exposure time of the third-party camera… Typically much longer than the exposure time of a motion capture camera. (For example =10ms Even with the same trigger time (=0.5ms), the exposure center times of the two cameras will still differ. For high-speed motion scenarios (limb speeds up to 5m / s), a 5ms deviation will result in a 25mm spatial registration error, severely affecting the accuracy of fused data. Secondly, the PTP timestamp is inconsistent with the physical exposure time. While PTP synchronization ensures the alignment of system timestamps across devices, there is a trigger delay between the camera receiving the trigger signal and the actual start of exposure. Furthermore, the delay values ​​differ between different camera brands. If data fusion relies solely on the PTP timestamp without considering the time offset during exposure, the physical acquisition time represented by the timestamp will not be consistent. Thirdly, the phase of the frequency division trigger is not aligned. When the frame rate of the third-party camera is lower than that of the motion capture camera (e.g., motion capture 120fps, third-party 30fps), the synchronization box needs to perform frequency division triggering. Existing technologies lack a systematic solution to ensure that the trigger time after frequency division is correctly aligned in phase with the exposure window of the motion capture camera. Fourthly, there is a lack of dynamic adaptive capability. Under the action of mechanisms such as Automatic Exposure Control (AEC), the exposure time of the third-party camera will adjust in real time according to changes in ambient light. Traditional solutions use fixed compensation values, which cannot respond to such dynamic changes. When the exposure time changes, the synchronization accuracy immediately decreases.

[0004] Therefore, there is an urgent need for a technical solution that can simultaneously solve the problems of timestamp alignment and physical acquisition time alignment under the differentiated synchronization triggering methods of motion capture cameras and third-party cameras, and has dynamic adaptive capabilities. Summary of the Invention

[0005] To address the issue of physical acquisition timing discrepancies between motion capture cameras and third-party cameras due to differences in exposure time and triggering methods, this application provides a synchronization triggering method and a timestamp alignment system for motion capture cameras and third-party cameras.

[0006] Firstly, this application provides a method for synchronizing motion capture cameras and third-party cameras, employing the following technical solution: A synchronization triggering method between a motion capture camera and a third-party camera, wherein the motion capture camera autonomously triggers exposure using PTP software synchronization, and the third-party camera passively triggers exposure using a synchronization box hardware triggering method, the method comprising: S1, PTP timestamp alignment step: Connect the PTP master clock, motion capture camera, third-party camera and synchronization box to the same PTP domain, and perform master-slave clock calibration through PTP protocol to align the system timestamps of all devices to the same global reference. S2. Synchronization trigger configuration steps: Configure the motion capture camera to PTP software synchronization autonomous trigger mode, configure the third-party camera to external hardware trigger mode, and configure the frequency division ratio of the synchronization box. and trigger output parameters; S3. Exposure parameter acquisition step: Obtain the first exposure time of the motion capture camera. and the second exposure time of the third-party camera and the frame rate of the motion capture camera. ; S4. Dynamic delay compensation calculation step, determining the second exposure time. Is it greater than the first exposure time? If so, calculate the initial delay compensation; monitor the changes in the exposure parameters of the third-party camera in real time, when the second exposure time... When the dynamic change exceeds a preset threshold, the current delay compensation amount is recalculated and dynamically updated. ,in The frame number; S5. Trigger time adjustment steps: Align the reference time based on the PTP timestamp. According to the frequency division ratio Calculate the base trigger time The base trigger time is adjusted based on the dynamically updated delay compensation amount to generate the compensated trigger time. This aligns the exposure center time of the third-party camera with the exposure center time of the motion capture camera. S6. Hardware trigger output step: The synchronization box triggers at the compensated time. A trigger pulse is output to the third-party camera via a hardware trigger interface, and the third-party camera begins exposure upon receiving the trigger pulse; S7. Timestamp recording and verification steps: After the motion capture camera and the third-party camera are exposed, they respectively record the PTP timestamp of the image frame. The timestamp corresponds to the exposure center time. The host computer software compares the two timestamps to verify the synchronization accuracy.

[0007] By adopting the above technical solution, the motion capture camera uses software synchronization and autonomous triggering, while the third-party camera uses hardware triggering. Based on the unified alignment of timestamps, the changes in the exposure parameters of the third-party camera are monitored in real time and the delay compensation is dynamically updated. The triggering time is adjusted to align the exposure centers of the two cameras. The synchronization accuracy is verified by recording and checking the timestamps, thus solving the problem of physical acquisition time alignment under different triggering methods.

[0008] Optionally, in step S4, the formula for calculating the dynamically updated delay compensation amount is: in, Indicates the first The current second exposure time of the third-party camera obtained at each frame; The compensated trigger time The calculation formula is: .

[0009] By adopting the above technical solution, half of the difference between the exposure time of the current frame of the third-party camera and the exposure time of the motion capture camera is used as the delay compensation amount, and the basic trigger time is advanced by this compensation amount, thus realizing the precise mathematical calculation and dynamic adjustment of the exposure center time.

[0010] Optionally, in step S4, when the second exposure time is detected... When the dynamic change exceeds a preset threshold, a smooth transition mechanism is used to update the delay compensation amount. The smoothly updated delay compensation amount is: in, The preset smoothing coefficient, and It is used to control the step size of the delay compensation amount between adjacent frames to avoid jumps at the trigger time.

[0011] By adopting the above technical solution, a smooth transition mechanism is used to update the delay compensation amount. The step size of the delay compensation amount between adjacent frames is controlled by a preset smoothing coefficient, which avoids the triggering time jump caused by sudden changes in exposure parameters, and ensures the smoothness of the triggering output and the stability of the system.

[0012] Optionally, in step S4, the preset threshold is a threshold for the second exposure time change between adjacent frames, and the recalculation of the delay compensation amount is triggered only when the following condition is met: in, The preset tolerance threshold for changes; when Less than or equal to At that time, the current delay compensation amount will remain unchanged.

[0013] By adopting the above technical solution, a preset threshold for the change in exposure time between adjacent frames is set. The delay compensation is only recalculated when the change exceeds the threshold, which avoids frequent compensation updates caused by small fluctuations in exposure time and improves the system's anti-interference capability.

[0014] Optionally, the method further includes: S8. Closed-loop calibration step: Measure the deviation between the actual exposure center time of the third-party camera and the actual exposure center time of the motion capture camera. ; The deviation Feedback is sent to the dynamic delay compensation calculation step to correct the delay compensation amount. This enables closed-loop self-calibration of dynamic compensation parameters.

[0015] By adopting the above technical solution and adding a closed-loop calibration step, the actual exposure center deviation between the third-party camera and the motion capture camera is measured and fed back to the dynamic delay compensation calculation step to correct the delay compensation amount. This achieves closed-loop self-calibration of the dynamic compensation parameters and ensures long-term stable operation of the system.

[0016] Secondly, this application provides a timestamp alignment system between a motion capture camera and a third-party camera, employing the following technical solution: A timestamp alignment system for motion capture cameras and third-party cameras, used to execute the synchronization triggering method for motion capture cameras and third-party cameras described in any of the above-mentioned methods, the system comprising: The PTP master clock is used to provide a global reference time and broadcasts PTP synchronization messages to all devices in the system via Ethernet. The motion capture camera synchronizes its timestamp with the PTP master clock via the PTP protocol, and operates at a preset frame rate based on the synchronized internal clock. The exposure is triggered autonomously, and the exposure time of the motion capture camera is the first exposure time. The motion capture camera is synchronized and triggered autonomously via PTP software. A third-party camera synchronizes its timestamps with the PTP master clock via the PTP protocol to record the time information of image frames. The exposure time of the third-party camera is the second exposure time. The synchronization triggering method of the third-party camera is to passively trigger by receiving an external hardware trigger signal; A synchronization box, which synchronizes its timestamp with the PTP master clock via the PTP protocol and is electrically connected to the third-party camera, includes: The PTP slave clock module is used to receive PTP synchronization messages and calibrate the local clock, so that the time deviation between the local clock and the PTP master clock is kept within the preset time accuracy range. The dynamic parameter monitoring module is used to monitor the second exposure time of the third-party camera in real time. The dynamic changes are observed, and it is determined whether the amount of change exceeds a preset threshold. The frequency division trigger module is used to trigger the clock according to a preset frequency division ratio based on the calibrated local clock. frame rate of motion capture camera Perform frequency division to generate a basic trigger time sequence; The dynamic delay compensation module is used to adjust the time based on the first exposure time. And the current second exposure time as monitored in real time Calculate dynamic delay compensation amount And based on the delay compensation amount, adjust the basic trigger time sequence to generate a compensated trigger time sequence; The trigger output module is used to output a trigger pulse to the third-party camera through a hardware trigger interface at the trigger time after compensation.

[0017] By adopting the above technical solution, an alignment system consisting of a master clock, a motion capture camera, a third-party camera, and a synchronization box was constructed. The synchronization box integrates a slave clock module, a dynamic parameter monitoring module, a frequency division trigger module, a dynamic delay compensation module, and a trigger output module. The modules work together to achieve high-precision synchronization and dynamic adaptive exposure center alignment of the heterogeneous camera system under differentiated triggering methods.

[0018] Optionally, the dynamic delay compensation module includes: The difference calculation unit is used to obtain the current second exposure time fed back in real time by the dynamic parameter monitoring module. And calculate its relationship with the first exposure time. The difference; The compensation amount generation unit is used to take half of the difference as the dynamic delay compensation amount for the current frame. Output; The trigger timing synthesis unit is used to obtain the basic trigger timing generated by the frequency division trigger module. And advance the basic trigger time by the aforementioned dynamic delay compensation amount. , to generate the compensation-triggered time.

[0019] By adopting the above technical solution, the dynamic delay compensation module consists of a difference calculation unit, a compensation amount generation unit, and a trigger time synthesis unit. It sequentially completes the difference calculation, compensation amount generation, and trigger time synthesis. The three units work together in a cascaded manner to realize the real-time calculation of the delay compensation amount and the accurate synthesis of the trigger time.

[0020] Optionally, the synchronization box supports multiple synchronization trigger alignment modes, including: Center alignment mode: The exposure center time of the third-party camera is aligned with the exposure center time of the motion capture camera, and the dynamic delay compensation is... ; Lead-edge alignment mode: The exposure start time of the third-party camera is aligned with the exposure start time of the motion capture camera, and the dynamic delay compensation is... ; Trailing edge alignment mode: The exposure end time of the third-party camera is aligned with the exposure end time of the motion capture camera, and the dynamic delay compensation amount is... .

[0021] By adopting the above technical solution, the synchronization box supports three synchronization triggering alignment modes: center alignment, leading edge alignment, and trailing edge alignment, which correspond to exposure center alignment, exposure start alignment, and exposure end alignment, respectively. Users can flexibly choose according to specific application scenarios to meet diverse acquisition needs.

[0022] Optionally, the trigger output module of the synchronization box is implemented based on FPGA, and the jitter of the output trigger pulse is less than the preset jitter tolerance; The dynamic parameter monitoring module detects the second exposure time. When the change exceeds a preset threshold, an update enable signal is sent to the dynamic delay compensation module to trigger the recalculation and update of the delay compensation amount.

[0023] By adopting the above technical solution, the trigger output module of the synchronization box is implemented based on the field programmable gate array, and the jitter of the output trigger pulse is less than the preset tolerance, ensuring the accuracy of the trigger timing; the dynamic parameter monitoring module triggers the compensation amount to be recalculated when it detects that the change in exposure time exceeds the threshold, realizing hardware-level low jitter triggering and fast response to parameter changes.

[0024] Optionally, the motion capture camera can be autonomously triggered in Free-running mode based on an internal clock synchronized with PTP, without the need for an external hardware trigger line; The third-party camera receives the trigger pulse output by the synchronization box via an external trigger input interface. The PTP timestamps recorded by the motion capture camera and the third-party camera both correspond to the exposure center time of their respective exposure processes. The synchronization box, through a closed-loop calibration module, adjusts the actual exposure center deviation according to the actual exposure center deviation. The dynamic delay compensation amount is corrected.

[0025] By adopting the above technical solution, the motion capture camera is autonomously triggered based on the synchronized internal clock, eliminating the need for external hardware trigger lines and simplifying system wiring; the third-party camera receives trigger pulses through an external trigger input interface, ensuring strong compatibility; the timestamps recorded by both correspond to the exposure center time, ensuring that the timestamps are consistent with the physical acquisition time; the closed-loop calibration module corrects the compensation amount according to the actual deviation, further improving the long-term operating accuracy and stability.

[0026] In summary, this application includes at least one of the following beneficial technical effects: (1) The problem of physical acquisition time alignment under the differentiated synchronous triggering method is solved. The motion capture camera adopts PTP software synchronization autonomous triggering, while the third-party camera adopts synchronization box hardware triggering. The two completely different synchronization mechanisms achieve accurate alignment of physical acquisition time through the dynamic delay compensation method of this invention. (2) It achieves dynamic adaptive exposure center alignment instead of static trigger time alignment. By dynamically compensating for delay, the synchronization accuracy is improved from trigger time alignment to physical acquisition time alignment. It can also respond to the dynamic changes of third-party camera exposure parameters in real time, eliminating the loss of synchronization accuracy caused by factors such as automatic exposure adjustment. (3) The smooth transition and threshold triggering mechanism ensures system stability. The preset threshold ε avoids frequent compensation updates, and the smoothing coefficient α achieves a smooth transition of the delay compensation amount, preventing system instability caused by the jump in the triggering time. (4) Strong compatibility: As an independent device, the synchronization box can be used with any third-party camera that supports PTP timestamps and external triggers without modifying the camera firmware. It also supports multiple independent configurations and can be compatible with multiple third-party cameras of different models at the same time. (5) Supports closed-loop self-calibration. The system can automatically measure the actual synchronization deviation and correct the compensation amount, and has good long-term stability.

[0027] (6) Multiple alignment modes are available, supporting three modes: center alignment, leading edge alignment, and trailing edge alignment, to meet different application needs. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the overall architecture of the time stamp alignment system between the motion capture camera and a third-party camera proposed in this invention.

[0029] Figure 2 This is a schematic diagram of the dynamic adaptive exposure center alignment timing principle proposed in this invention.

[0030] Figure 3 This is a flowchart of the dynamic delay compensation algorithm proposed in this invention. Detailed Implementation

[0031] The embodiments of this application are described in detail below, and examples of the embodiments are shown in the accompanying drawings.

[0032] In the description of this specification, the references to "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples" refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0033] Example 1: System Overall Architecture and Synchronization Triggering Method like Figure 1As shown, the timestamp alignment system between the motion capture camera and a third-party camera of the present invention includes the following core components, each employing a differentiated synchronization triggering method: PTP Master Clock: Employs a Grandmaster clock device supporting IEEE 1588v2, broadcasting PTP synchronization messages (Sync, Follow_Up, Delay_Req, Delay_Resp) to all devices in the system via Ethernet to establish a globally unified time base. The PTP master clock is the basis for the timestamp alignment of the entire system. The PTP master clock can be an industrial-grade PTP server with a built-in high-precision oven-controlled crystal oscillator (OCXO), whose time retention accuracy is better than ±50ppb, and can still maintain a high-precision time base when the PTP synchronization signal is briefly interrupted.

[0034] PTP Switch: A PTP switch that supports hardware timestamps ensures that the delay of PTP packets during forwarding is measurable and compensable. It is a key network device for achieving sub-microsecond time synchronization. All devices access the same PTP domain through the PTP switch. The PTP switch uses either a boundary clock or a transparent clock mode to correct the residence time during packet forwarding, thereby eliminating synchronization errors introduced by network switching.

[0035] Motion capture camera: This refers to the infrared camera in an optical motion capture system (such as a motion capture camera equipped with an 850nm infrared LED light source). The synchronization triggering method of the motion capture camera is PTP software synchronization autonomous triggering (Free-running mode): The camera has a built-in PTP slave clock function, receives PTP synchronization messages via Ethernet, calibrates its internal clock, and then, based on the synchronized clock, executes the synchronization at a preset frame rate. Exposure is automatically triggered, requiring no external hardware trigger wire; the exposure time of the motion capture camera is the first exposure time. Its value range is usually from 100μs to 2ms, depending on the camera model and shooting scene configuration. Multiple motion capture cameras are synchronized through PTP, and the deviation of their exposure start time is less than 1μs.

[0036] Third-party cameras: These are heterogeneous cameras that need to be integrated with the motion capture system, such as high-resolution RGB industrial cameras, ToF depth cameras, structured light cameras, or high-speed cameras. The synchronization triggering method for third-party cameras is external hardware triggering (passive triggering): The camera synchronizes with the system timestamp (used to record image frame time information) via the PTP protocol, and simultaneously receives the hardware trigger pulse output by the synchronization box through the external TriggerIn interface. Exposure begins after receiving the trigger pulse, and the exposure time of the third-party camera is the second exposure time. Its value is usually greater than The typical range is 5ms to 30ms, depending on the camera type, resolution, and operating mode. For example, the effective exposure time of a ToF depth camera in multi-frame integration mode can be more than 10ms.

[0037] Synchronization Box: This is the core component of the present invention. It accesses the PTP domain via Ethernet and synchronizes its timestamp with the PTP master clock. The synchronization box includes a PTP slave clock module, a dynamic parameter monitoring module, a frequency division triggering module, a dynamic delay compensation module, and a trigger output module. The core function of the synchronization box is to achieve precise alignment between the triggering time of the third-party camera hardware and the autonomous triggering time of the motion capture camera based on PTP timestamp alignment, through frequency division triggering and dynamic delay compensation, while responding in real time to the dynamic changes in the exposure parameters of the third-party camera.

[0038] Example 2: Timing Comparison of Two Synchronous Triggering Methods In this invention, the motion capture camera and the third-party camera employ two different synchronization triggering methods, and their timing characteristics are compared as follows: PTP software synchronization autonomous triggering timing sequence for motion capture cameras: (1) PTP synchronization stage: The motion capture camera receives the Sync message of the PTP master clock via Ethernet, uses the precise timestamp in the Follow_Up message and the Delay_Req / Delay_Resp message to measure the network delay, and adjusts the local clock frequency and phase through the clock servo algorithm (such as PI controller) so that the local clock deviates from the master clock by less than 1μs.

[0039] (2) Autonomous Triggering Phase: After PTP synchronization is completed, the frame rate counter inside the motion capture camera starts counting based on the aligned PTP clock. When the count reaches the frame period... At that time, the camera automatically triggers exposure, and the exposure duration is... The trigger time is strictly aligned to the PTP time base (such as the whole second boundary) to ensure that the exposure time of multiple motion capture cameras is highly consistent.

[0040] (3) Timestamp recording: After the exposure is completed, the motion capture camera records the PTP timestamp of the image frame. This timestamp usually corresponds to the exposure start time or the exposure center time, depending on the camera firmware configuration. In this invention, the timestamp is preferably configured to correspond to the exposure center time so as to facilitate direct comparison with the exposure center time of a third-party camera.

[0041] Hardware trigger timings for the sync box of a third-party camera: (1) PTP timestamp synchronization: Third-party cameras also synchronize with the master clock through the PTP protocol, but PTP synchronization is only used for timestamp recording and does not directly control exposure triggering.

[0042] (2) Hardware trigger reception: The third-party camera waits for an external trigger pulse through the Trigger In interface. The polarity of the trigger pulse is configurable (rising edge trigger or falling edge trigger), and the pulse width is configured by the synchronization box, typically from 2μs to 10μs.

[0043] (3) Exposure execution: Upon receiving the trigger pulse, the third-party camera begins exposure, and the exposure duration is... After the exposure is completed, a PTP timestamp is recorded, which corresponds to the exposure center time.

[0044] (4) Trigger source: The trigger pulse is generated by the synchronization box, which precisely controls the trigger time based on the local clock synchronized by PTP.

[0045] The key difference between the two synchronous triggering methods is that the triggering time of the motion capture camera is directly controlled by the internal PTP clock (software synchronization), while the triggering time of the third-party camera is controlled by the hardware pulse output by the synchronization box (hardware triggering). This invention aligns the two different synchronous triggering methods at the physical acquisition time level through the dynamic delay compensation mechanism of the synchronization box, and this alignment can adapt to the real-time changes in the exposure parameters of the third-party camera.

[0046] Example 3: Principle of Dynamic Delay Compensation and Trigger Timing Adjustment like Figure 2 and Figure 3 As shown, the core innovation of this invention lies in achieving precise adjustment of the triggering time through dynamic delay compensation, aligning the exposure center of the third-party camera with the exposure center of the motion capture camera, and enabling real-time response to dynamic changes in the exposure parameters of the third-party camera. Figure 2 This demonstrates the timing principle behind exposure center alignment after delay compensation. Figure 3 The specific logic of delay compensation calculation and trigger time adjustment is demonstrated.

[0047] Basic trigger time calculation: In traditional solutions, the synchronization box is based on the basic trigger time. Output trigger pulse: in, To align the PTP timestamp with the reference time, a whole-second boundary or a whole-frame boundary is typically chosen. For the frame rate of the motion capture camera, It is the frequency division ratio. A positive integer, representing each A frame capture camera triggers a frame from a third-party camera. This is the frame number.

[0048] Static delay compensation: Due to the exposure time of the third-party camera Larger than the exposure time of the motion capture camera If triggered at the base trigger time, the exposure centers of the two cameras will differ: This discrepancy means that although the trigger times of the two cameras are aligned on the PTP timeline, the actual physical times captured are not consistent.

[0049] Therefore, the dynamic delay compensation module of the synchronization box calculates the initial delay compensation amount: And adjust the trigger time as follows: By advancing the trigger time The third-party camera's exposure start time is earlier, and its exposure center time coincides exactly with the motion capture camera's exposure center time, such as... Figure 2 As shown, after delay compensation, the exposure centers of the motion capture camera (with a narrower exposure window) and the third-party camera (with a wider exposure window) are strictly aligned on the PTP time axis.

[0050] Dynamic adaptive delay compensation: In practical applications, the exposure time of third-party cameras The system may dynamically adjust to changes in ambient lighting conditions (e.g., automatic exposure control (AEC) function). To address this, the present invention introduces a dynamic adaptive delay compensation mechanism.

[0051] The sync box's built-in dynamic parameter monitoring module monitors the current exposure time of the third-party camera in real time. ,in Indicates the current frame number; the dynamic delay compensation module is based on the latest... Calculate the dynamic delay compensation: The corresponding trigger time after compensation is: When the exposure time of a third-party camera changes, for example from Change to The dynamic delay compensation module updates automatically. This ensures that the exposure center moment of each frame is precisely aligned.

[0052] Example (static scenario): Set the frame rate of the motion capture camera Exposure time Third-party camera frame rate = 30fps (ratio N = 4), initial exposure time =10ms.

[0053] Basic trigger interval: Initial delay compensation amount: Triggering time after compensation: At this point, the exposure center time of the third-party camera is strictly aligned with the exposure center time of the motion capture camera.

[0054] Example (dynamically changing scenario): Continuing the example, suppose the ambient light dims at a certain moment, and the third-party camera's automatic exposure control adjusts the exposure time from 10ms to 15ms. If a static compensation scheme is used, the compensation amount remains 4.75ms, resulting in: The exposure center deviation will result in an additional registration error of 12.5mm for a moving target at a speed of 5m / s.

[0055] Using the dynamic adaptive compensation scheme of this invention, the dynamic parameter monitoring module detects... After the change, a recalculation is triggered: After compensation, the trigger time is automatically adjusted to: base trigger time - 7.25ms. The exposure center time of the third-party camera is re-aligned with that of the motion capture camera, eliminating the synchronization deviation caused by changes in exposure parameters.

[0056] Example 4: Smooth Transition Mechanism and Threshold Triggering In the synchronous triggering method of the present invention, the dynamic delay compensation calculation step (step S4) further includes a smooth transition mechanism and a threshold triggering mechanism to avoid the triggering time jumping due to drastic or frequent changes in exposure parameters, which would affect the stability of the system.

[0057] Threshold triggering mechanism: The dynamic parameter monitoring module calculates the change in exposure time of the third-party camera between adjacent frames in real time: Only when the change exceeds a preset threshold Only when this happens will the recalculation of the delay compensation amount be triggered; when Less than or equal to At that time, the current delay compensation amount remains unchanged, that is: like ,but This mechanism avoids frequent compensation updates caused by minor fluctuations in exposure time (such as quantization noise or measurement errors), improving the system's anti-interference capability and stability. A preset threshold is also included. The value can be configured according to the exposure time resolution of the third-party camera and the accuracy requirements of the application scenario. For example, it can be set to twice the exposure time resolution, or... A certain percentage.

[0058] Smooth transition mechanism: When the detected change in exposure time exceeds a preset threshold When the delay compensation amount needs to be updated, in order to avoid the triggering time from jumping between adjacent frames (which may cause third-party cameras to trigger jitter or exposure instability), this invention adopts a smooth transition mechanism: in, The preset smoothing coefficient has a range of values. It is used to control the step size of the delay compensation amount between adjacent frames.

[0059] when At this time, the smooth transition mechanism degenerates into a direct switch, and the delay compensation amount is updated to the target value all at once; when The delay compensation gradually approaches the target value over multiple frame periods, achieving a smooth change in the trigger moment.

[0060] The specific value of the smoothing coefficient needs to strike a balance between response speed and stability. For example, in dynamic scenes, α=0.5 can be used to make the delay compensation amount transition within 2 to 3 frames; in static scenes, a smaller α value (such as α=0.2) can be used to obtain a smoother change curve.

[0061] The smooth transition mechanism is particularly suitable for the following scenarios: (1) When the exposure time of a third-party camera changes abruptly, it avoids the sudden change in the trigger time that could cause the PLL inside the camera to lose lock or the trigger circuit to malfunction; (2) When multiple third-party cameras share the same synchronization box, the smooth transition can avoid the power ripple and electromagnetic interference problems caused by the simultaneous change of multiple trigger pulses.

[0062] Comprehensive example: Continuing with the example in Example 3, the exposure time of the third-party camera changes abruptly from 10ms to 15ms, with a preset threshold. =200μs, smoothing coefficient α=0.5.

[0063] The following was detected in frame k: Triggering compensation update: At frame k+1: At frame k+2: After about 6 frames, the delay compensation gradually approaches the target value of 7.25ms, achieving a smooth transition.

[0064] Example 5: Complete Flow of Synchronous Triggering Method like Figure 3 As shown, the synchronization triggering method between the motion capture camera and a third-party camera of the present invention includes the following steps: Step S1: PTP timestamp alignment. After power-on, each component performs self-tests and loads default configuration parameters. The synchronization box loads FPGA firmware, initializes the PTP slave clock module, configures the default threshold and smoothing coefficient of the dynamic parameter monitoring module, and connects the PTP master clock, motion capture camera, third-party camera, and synchronization box to the same PTP domain. The synchronization box acts as a PTP slave device, exchanging Sync, Follow_Up, Delay_Req, and Delay_Resp messages with the master clock, measuring network path delay and compensating for clock offset, so that the deviation between the local clock and the master clock is kept within the preset time accuracy range (preferably, the deviation is less than 1μs). The motion capture camera and third-party camera also complete PTP synchronization. This step ensures that the system timestamps of all devices are aligned to the same global reference.

[0065] Step S2: Configure the synchronization trigger mode. Configure the motion capture camera to PTP software synchronization autonomous trigger mode (Free-running), configure the frame rate F, configure the third-party camera to external hardware trigger mode, set the trigger polarity (rising edge / falling edge) and trigger delay (usually configured to 0), and configure the frequency division ratio N, trigger pulse width, output channel, dynamic compensation parameters (preset threshold ε, smoothing coefficient α) of the synchronization box.

[0066] Step S3: Exposure parameter acquisition and dynamic monitoring start. Obtain the first exposure time of the motion capture camera via camera API or manual configuration. Initial and second exposure times of third-party cameras The exposure parameters can be obtained in the following ways: (a) by reading directly from the camera register; (b) by broadcasting via PTP Action Command; and (c) by manual input by the user, followed by starting the dynamic parameter monitoring module to monitor the exposure time changes of the third-party camera in real time.

[0067] Step S4: Determine If not, then no delay compensation is needed; the base trigger time will be used directly as the output time. If so, then read the current exposure time of the third-party camera. Calculate the dynamic delay compensation amount: If a smooth transition mechanism is used, the update will be performed as follows: like: but: otherwise: Step S5: Adjust the trigger time and calculate the compensated trigger time: Step 6: Hardware trigger output. The FPGA trigger pulse generator of the synchronization box outputs a hardware trigger pulse at a precisely calculated moment. The pulse is output to the Trigger In port of the third-party camera through the BNC or SMA interface. The FPGA implements the jitter of the trigger output to be less than the preset jitter tolerance, preferably less than 100ns.

[0068] Step S7: Third-party camera exposure and timestamp recording. The third-party camera begins exposure after receiving the trigger pulse, and the exposure continues. The motion capture camera and the third-party camera record the PTP timestamp of the image frame after the exposure is completed. The timestamp corresponds to the exposure center time.

[0069] Step S8: Closed-loop calibration and dynamic monitoring loop. Measure the actual exposure center deviation Δt between the third-party camera and the motion capture camera. If |Δt| exceeds a predetermined threshold (e.g., 10μs), then feed Δt back to the dynamic delay compensation module for correction. The closed-loop calibration can be performed periodically or once at system startup, and then the process returns to step S4 to repeat the dynamic delay compensation calculation for the next frame (k+1) to achieve continuous dynamic adaptive synchronization.

[0070] Example 6: Synchronization Box Hardware Structure The hardware structure of the synchronization box specifically includes the following modules: Input interface area: PTP network port (supports IEEE 1588v2, 10 / 100 / 1000Base-T auto-negotiation), configuration interface (USB / RS232, for parameter configuration and firmware upgrade), external trigger input (optional, for external synchronization signal cascading), power input (DC 5V-24V wide voltage input, with reverse connection protection).

[0071] The PTP slave clock module includes a PTP protocol parser (hardware implementation, deployed inside the FPGA), a clock servo phase-locked loop (PLL, based on digital dual-loop control, adjusting frequency and phase respectively), and a high-precision local clock (TCXO / OCXO, frequency stability better than ±50ppb), which is responsible for synchronizing with the PTP master clock and outputting a timestamp with nanosecond precision (64-bit PTP timestamp, resolution of 1ns).

[0072] Dynamic parameter monitoring module: This is a key new module in this invention, including an exposure parameter receiving unit (receiving real-time exposure time from a third-party camera via Ethernet or camera control bus) and a change calculation unit (calculating...). The threshold comparator (compares with the preset threshold ε and outputs an update enable signal) notifies the dynamic delay compensation module to update the compensation amount via an interrupt, with a response delay of less than 10μs.

[0073] Frequency division trigger module: includes a frame rate counter (aligned with PTP clock, counting accuracy 1ns), a programmable frequency divider (division ratio N:1, N is configurable from 1 to 65535) and a trigger pulse generator (FPGA implementation, output pulse width is programmable, jitter is less than 100ns).

[0074] Dynamic delay compensation module (core): includes exposure parameter register (storage) and current ), difference calculation unit (calculation) ), division / displacement unit (divides the difference by 2, which can be implemented by right shifting by one bit in hardware), and smooth transition operation unit (implements...). The system includes a calculation unit and a trigger timing adjuster (which subtracts the dynamic delay compensation amount from the basic trigger timing). The difference calculation unit, the compensation amount generation unit, and the trigger timing synthesis unit work together to achieve the dynamic delay compensation function. All of the above calculation units can be implemented through FPGA hardware logic to ensure that the calculation delay is less than 100ns and that no additional trigger jitter is introduced.

[0075] Trigger output module: multi-channel trigger output (supports multiple third-party cameras, each channel can be independently configured with delay compensation parameters), Strobe output (exposure indication, used for external devices to monitor exposure status), PTP status LED (indicating PTP synchronization lock status and accuracy level), the trigger output adopts LVTTL or RS-422 level, supports rising edge / falling edge configuration, and the output impedance is 50Ω.

[0076] Main control unit: ARM Cortex-M7 or FPGA SoC (such as Xilinx Zynq series), running a lightweight real-time operating system, coordinating the work of various modules, supporting remote configuration and status query through HTTP / JSON interface, and supporting online firmware upgrade.

[0077] Example 7: Synergistic Operation of Closed-Loop Calibration and Dynamic Compensation In this invention, the closed-loop calibration module and the dynamic delay compensation module work together to form a dual protection mechanism.

[0078] Closed-loop calibration principle: The closed-loop calibration module obtains the actual exposure center deviation Δt between the third-party camera and the motion capture camera through external measurement methods. These measurement methods include, but are not limited to: (a) Use a high-speed photodetector to detect the changes in light signals during the exposure process of the two cameras, and measure the time difference using an oscilloscope; (b) High-frequency LED markers are placed in the scene, and the exposure center deviation is calculated by analyzing the brightness phase difference of the LEDs in the images of the two cameras; (c) Δt is obtained by measuring the time interval using the Strobe output of the sync box and the exposure indication output of a third-party camera (such as a flash sync signal).

[0079] Collaborative working mechanism: (1) When the system starts up, the closed-loop calibration module performs the initial measurement to obtain the initial deviation. This deviation is used to correct the initial delay compensation amount: (2) During system operation, the dynamic parameter monitoring module continuously monitors The dynamic delay compensation module updates in real time to reflect changes. The closed-loop calibration module can perform calibration measurements periodically (e.g., every 1000 frames). If a new deviation is measured... Then the delay compensation amount is corrected: (3) When the dynamic parameter monitoring module detects When a significant change occurs (exceeding 3 times the preset threshold ε), the closed-loop calibration module can be actively triggered to perform a measurement, verify the compensation effect, and make corrections if necessary.

[0080] Through the combined action of dynamic delay compensation and closed-loop calibration, the system can adaptively cope with various changing factors (including exposure time variations, temperature drift, equipment aging, etc.) during long-term operation, always maintaining high-precision time synchronization and exposure center alignment.

[0081] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for synchronizing a motion capture camera with a third-party camera, characterized in that, The motion capture camera uses PTP software synchronization to autonomously trigger exposure, while the third-party camera uses a synchronization box hardware trigger to passively trigger exposure. The method includes: S1, PTP timestamp alignment step: connect the PTP master clock, motion capture camera, third-party camera and synchronization box to the same PTP domain, and perform master-slave clock calibration through PTP protocol to align the system timestamps of all devices to the same global reference. S2. Synchronization trigger configuration steps: Configure the motion capture camera to PTP software synchronization autonomous trigger mode, configure the third-party camera to external hardware trigger mode, and configure the frequency division ratio of the synchronization box. and trigger output parameters; S3. Exposure parameter acquisition step: Obtain the first exposure time of the motion capture camera. and the second exposure time of the third-party camera and the frame rate of the motion capture camera. ; S4. Dynamic delay compensation calculation step, determining the second exposure time. Is it greater than the first exposure time? If so, calculate the initial delay compensation; monitor the changes in the exposure parameters of the third-party camera in real time, when the second exposure time... When the dynamic change exceeds a preset threshold, the current delay compensation amount is recalculated and dynamically updated. ,in The frame number; S5. Trigger time adjustment steps: Align the reference time based on the PTP timestamp. According to the frequency division ratio Calculate the base trigger time The base trigger time is adjusted based on the dynamically updated delay compensation amount to generate the compensated trigger time. This aligns the exposure center time of the third-party camera with the exposure center time of the motion capture camera. S6. Hardware trigger output step: The synchronization box triggers at the compensated time. A trigger pulse is output to the third-party camera via a hardware trigger interface, and the third-party camera begins exposure upon receiving the trigger pulse; S7. Timestamp recording and verification steps: After exposure, the motion capture camera and the third-party camera respectively record the PTP timestamp of the image frame. The timestamp corresponds to the exposure center time. The host computer software compares the two timestamps to verify the synchronization accuracy.

2. The method for synchronizing the motion capture camera and a third-party camera according to claim 1, characterized in that, In step S4, the formula for calculating the dynamically updated delay compensation is: in, Indicates the first The current second exposure time of the third-party camera obtained at each frame; The compensated trigger time The calculation formula is: 。 3. The method for synchronous triggering of motion capture camera and third-party camera according to claim 2, characterized in that, In step S4, when the second exposure time is detected... When the dynamic change exceeds a preset threshold, a smooth transition mechanism is used to update the delay compensation amount. The smoothly updated delay compensation amount is: in, The preset smoothing coefficient, and It is used to control the step size of the delay compensation amount between adjacent frames to avoid jumps at the trigger time.

4. The method for synchronous triggering of motion capture camera and third-party camera according to claim 3, characterized in that, In step S4, the preset threshold is the threshold for the change in second exposure time between adjacent frames, and the recalculation of the delay compensation amount is triggered only when the following condition is met: in, The preset tolerance threshold for changes; when Less than or equal to At that time, the current delay compensation amount will remain unchanged.

5. The method for synchronous triggering of a motion capture camera and a third-party camera according to claim 4, characterized in that, The method further includes: S8. Closed-loop calibration step: Measure the deviation between the actual exposure center time of the third-party camera and the actual exposure center time of the motion capture camera. ; The deviation Feedback is sent to the dynamic delay compensation calculation step to correct the delay compensation amount. This enables closed-loop self-calibration of dynamic compensation parameters.

6. A timestamp alignment system between a motion capture camera and a third-party camera, characterized in that, The system is used to execute the synchronization triggering method between the motion capture camera and a third-party camera as described in any one of claims 1 to 5, the system comprising: The PTP master clock is used to provide a global reference time and broadcasts PTP synchronization messages to all devices in the system via Ethernet. The motion capture camera synchronizes its timestamp with the PTP master clock via the PTP protocol, and operates at a preset frame rate based on the synchronized internal clock. The exposure is triggered autonomously, and the exposure time of the motion capture camera is the first exposure time. The motion capture camera is synchronized and triggered autonomously via PTP software. A third-party camera synchronizes its timestamps with the PTP master clock via the PTP protocol to record the time information of image frames. The exposure time of the third-party camera is the second exposure time. The synchronization triggering method of the third-party camera is to passively trigger by receiving an external hardware trigger signal; A synchronization box, which synchronizes its timestamp with the PTP master clock via the PTP protocol and is electrically connected to the third-party camera, includes: The PTP slave clock module is used to receive PTP synchronization messages and calibrate the local clock, so that the time deviation between the local clock and the PTP master clock is kept within the preset time accuracy range. The dynamic parameter monitoring module is used to monitor the second exposure time of the third-party camera in real time. The dynamic changes are observed, and it is determined whether the amount of change exceeds a preset threshold. The frequency division trigger module is used to trigger the clock according to a preset frequency division ratio based on the calibrated local clock. frame rate of motion capture camera Perform frequency division to generate a basic trigger time sequence; The dynamic delay compensation module is used to adjust the time based on the first exposure time. And the current second exposure time as monitored in real time Calculate dynamic delay compensation amount And based on the delay compensation amount, adjust the basic trigger time sequence to generate a compensated trigger time sequence; The trigger output module is used to output a trigger pulse to the third-party camera through a hardware trigger interface at the trigger time after compensation.

7. The timestamp alignment system between the motion capture camera and a third-party camera according to claim 6, characterized in that, The dynamic delay compensation module includes: The difference calculation unit is used to obtain the current second exposure time fed back in real time by the dynamic parameter monitoring module. And calculate its relationship with the first exposure time. The difference; The compensation amount generation unit is used to take half of the difference as the dynamic delay compensation amount for the current frame. Output; The trigger timing synthesis unit is used to obtain the basic trigger timing generated by the frequency division trigger module. And advance the basic trigger time by the aforementioned dynamic delay compensation amount. , to generate the compensation-triggered time.

8. The timestamp alignment system between the motion capture camera and a third-party camera according to claim 7, characterized in that, The synchronization box supports multiple synchronization trigger alignment modes, including: Center alignment mode: The exposure center time of the third-party camera is aligned with the exposure center time of the motion capture camera, and the dynamic delay compensation is... ; Lead-edge alignment mode: The exposure start time of the third-party camera is aligned with the exposure start time of the motion capture camera, and the dynamic delay compensation is... ; Trailing edge alignment mode: The exposure end time of the third-party camera is aligned with the exposure end time of the motion capture camera, and the dynamic delay compensation amount is... .

9. The timestamp alignment system between the motion capture camera and a third-party camera according to claim 8, characterized in that, The trigger output module of the synchronization box is implemented based on FPGA, and the jitter of the output trigger pulse is less than the preset jitter tolerance; The dynamic parameter monitoring module detects the second exposure time. When the change exceeds a preset threshold, an update enable signal is sent to the dynamic delay compensation module to trigger the recalculation and update of the delay compensation amount.

10. The timestamp alignment system between the motion capture camera and a third-party camera according to claim 9, characterized in that, The motion capture camera is autonomously triggered in Free-running mode based on the internal clock synchronized by PTP, without the need for external hardware trigger lines; The third-party camera receives the trigger pulse output by the synchronization box via an external trigger input interface. The PTP timestamps recorded by the motion capture camera and the third-party camera both correspond to the exposure center time of their respective exposure processes. The synchronization box uses a closed-loop calibration module to adjust the actual exposure center deviation. The dynamic delay compensation amount is corrected.

Images