Audio and video acquisition synchronous management method

Abstract

The invention discloses an audio and video acquisition synchronous management method, and relates to the technical field of data synchronization. The method is executed by main control equipment and comprises the following steps: S1, acquiring a satellite time reference through a dual-mode time service module; s2, based on the pulse signal of the temperature compensated crystal oscillator, a master clock signal is generated according to a formula Tmaster = TGPS + alpha delta T + beta (delta f / f0), TGPS is a satellite time reference, alpha is a temperature compensation coefficient and ranges from 0.2 to 0.8, and beta is a frequency drift correction factor; s3, establishing an inter-device time synchronization communication link through an IEEE 1588PTP protocol, and broadcasting a synchronization pulse by the main control device; s4, the data buffer area is adjusted in real time according to the dynamic buffer control formula D (t) = D0 + k1 * (delta t) 2 + k2 * d delta t / dt, k1 = 0.15 ms-2, and k2 = 2.5 ms2 / s; s5, when the network time delay gt is detected; and at 150ms, switching to a local clock compensation mode, and triggering the extended Kalman filter to re-correct the clock parameters. According to the invention, a hybrid synchronization architecture with local clock correction capability is constructed, and self-adaptive time difference compensation without network dependence is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of data synchronization, and in particular to a method for synchronous management of audio and video acquisition. Background Art

[0002] In existing technologies, audio and video synchronization is crucial for ensuring accurate information presentation and user experience, especially in security monitoring, remote conferencing, and other fields where synchronization precision is critical. With the advancement of multimedia technology, a variety of audio and video synchronization solutions have emerged. Some solutions utilize the Network Time Protocol (NTP) for clock synchronization, while others are exploring time alignment through satellite timing systems.

[0003] However, these solutions have significant limitations in dynamic law enforcement scenarios: first, the tag comparison-based method is susceptible to electromagnetic interference in mobile device scenarios; second, the buffer adjustment algorithm does not take into account the differences in clock sources of multiple devices, and experiments have shown that there is time deviation after multiple nodes work continuously; third, network-dependent solutions have the risk of synchronization failure in closed scenarios such as tunnels and basements, and the measured network delay fluctuation synchronization error grows exponentially; finally, the satellite timing solution has strict requirements on the stability of the crystal oscillator.

[0004] In summary, existing technologies suffer from three drawbacks: 1) Network-dependent solutions cannot guarantee reliable synchronization in offline environments; 2) Device clock discrepancies lead to cumulative errors when coordinating multiple nodes; and 3) a lack of real-time compensation mechanisms for abnormal disturbances in dynamic environments. These shortcomings directly impact the integrity and effectiveness of the chain of evidence for law enforcement, necessitating the urgent need for new synchronization solutions. Summary of the Invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method for managing audio and video acquisition synchronization, constructs a hybrid synchronization architecture with local clock correction capability, and realizes adaptive time difference compensation without network dependence.

[0006] The specific technical solutions are as follows:

[0007] A method for synchronous management of audio and video acquisition is executed by a main control device, comprising:

[0008] S1. Obtain satellite time reference through dual-mode timing module;

[0009] S2. Based on the pulse signal of the temperature-compensated crystal oscillator, the master clock signal is generated according to the formula T_master = T_GPS + αΔT + β(Δf / f0), where T_GPS is the satellite time reference, α is the temperature compensation coefficient, ranging from 0.2-0.8, and β is the frequency drift correction factor;

[0010] S3. Establish a time-synchronized communication link between devices using the IEEE 1588 PTP protocol, with the master device broadcasting synchronization pulses.

[0011] S4. Adjust the data buffer in real time according to the dynamic buffer control formula D(t) = D0 + k1*(δt)^2 + k2*dδt / dt, where k1 = 0.15ms^-2 and k2 = 2.5ms 2 / s;

[0012] S5. When the network delay is detected to be greater than 150ms, the system switches to the local clock compensation mode and triggers the extended Kalman filter to recalibrate the clock parameters.

[0013] Furthermore, in the above solution, the frequency stability of the temperature compensated crystal oscillator in S2 is ±0.5ppm, and the operating temperature range is -40°C to 85°C.

[0014] Furthermore, the above solution dynamically adjusts the data buffer depth in S4, including:

[0015] S41. Calculate the average δt of the three most recent synchronization time differences;

[0016] S42. Calculate the time difference change rate dδt / dt and use sliding window filtering to eliminate pulse interference;

[0017] S43. Match the buffer depth level according to the preset mapping table and set the inter-level switching protection delay to 10ms.

[0018] Furthermore, in the above scheme, the state equation of the extended Kalman filter in S5 is set as:

[0019] x_k=[Δt,df]^T=A x_{k-1}+w_k,

[0020] The system noise covariance matrix is configured as a diagonal matrix Q=diag(σ_t2, σ_f2), where σ_t=0.1ms, σ_f=0.05ppm, and the clock parameters are determined by measuring the crystal oscillator phase noise using a laboratory spectrum analyzer.

[0021] The above solution further implements the cluster management mechanism in a non-network environment:

[0022] S51 establishes the central node of the master device N0 as the reference clock source;

[0023] S52. The slave device calibrates its local clock every 30 seconds according to the clock correction formula

[0024] C_slave(t) = C_master(t-Δt) + γ*RTT / 2, where Δt represents the delay factor and γ represents the path attenuation factor. Δt is calculated using the sliding window mean, and the window size N = 50 is determined based on actual measurements of industrial wireless networks.

[0025] S53. Measure the path delay RTT through two-way communication. The path attenuation factor γ is γ = 1-(1 / (SNR+1)). SNR is the signal-to-noise ratio of the measurement path, which is obtained by converting the RSSI value collected by the device RF front-end. Set the path attenuation factor γ to 0.9-1.1.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] This invention uses a dual-mode timing module to obtain a satellite time reference, independent of the local network. This solves the problem of network-dependent solutions failing in law enforcement scenarios without a network signal, such as basements and tunnels. Furthermore, when network latency is high, the system switches to local clock compensation mode and uses an extended Kalman filter to recalibrate clock parameters. This ensures that audio and video acquisition synchronization can be achieved even in unstable network conditions, improving synchronization reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of the steps of the method of the present invention;

[0029] Figure 2 It is a specific flow chart of the present invention. DETAILED DESCRIPTION

[0030] The following is a further detailed description of the embodiments of the invention in conjunction with the accompanying drawings to make the objectives, technical solutions and technical effects of the invention more clearly presented.

[0031] like Figure 1-2 As shown, a method for synchronous management of audio and video acquisition is executed by a main control device, comprising:

[0032] S1. Obtain a satellite time reference through a dual-mode timing module. This module uses the UBX-M8030 chipset for dual-mode GPS / Beidou signal reception. This module receives satellite signals and extracts precise time information from them, using this as the time reference for the entire system. The satellite time reference offers exceptional accuracy and stability, providing a reliable starting reference for subsequent clock synchronization and audio and video acquisition synchronization.

[0033] S2. Based on the pulse signal from the temperature-compensated crystal oscillator, a master clock signal is generated according to the formula T_master = T_GPS + αΔT + β(Δf / f0), where T_GPS is the satellite time reference, α is the temperature compensation coefficient (range: 0.2-0.8), and β is the frequency drift correction factor. This method considers various errors in the satellite time reference and the crystal oscillator to generate a more accurate and stable master clock signal, providing a precise internal clock reference for the entire system.

[0034] S3. A time synchronization communication link is established between devices using the IEEE 1588 PTP protocol. The master device broadcasts synchronization pulses with an accuracy of ±100ns. IEEE 1588 PTP (Precision Time Protocol) is a protocol used to achieve high-precision clock synchronization in distributed systems. In this step, the master device uses this protocol to establish a time synchronization communication link with other devices. The master device broadcasts synchronization pulses to other devices over the network. Upon receiving these pulses, the other devices adjust their own clocks according to the protocol's algorithm, achieving high-precision synchronization with the master device's clock.

[0035] S4. Adjust the data buffer in real time according to the dynamic buffer control formula D(t) = D0 + k1*(δt)^2 + k2*dδt / dt, where k1 = 0.15ms^-2 and k2 = 2.5ms 2 When the mean value of the synchronization time difference or its rate of change changes, the size of the data buffer is adjusted in real time according to the formula.

[0036] S5. When network delay is detected to be greater than 150ms, the device switches to local clock compensation mode, triggering the Extended Kalman Filter (EKF) to recalibrate clock parameters. During network transmission, if network delay is detected to be greater than 150ms, this indicates poor network conditions. Continuing to rely on the network for clock synchronization may result in significant errors. Therefore, the device switches to local clock compensation mode, using the device's local clock information for compensation. The Extended Kalman Filter (EKF) is used to recalibrate clock parameters. It estimates and corrects the clock state by establishing state and observation equations. In the state equation x_k = [Δt, df]^T = Ax_{k-1} + w_k, Δt represents the time deviation, df represents the frequency deviation, A is the state transition matrix, and w_k represents the system noise. The system noise covariance matrix is configured as a diagonal matrix Q = diag(σ_t², σ_f²), where σ_t = 0.1ms and σ_f = 0.05ppm. These parameters are determined by measuring the crystal oscillator phase noise using a laboratory spectrum analyzer. By continuously recalibrating the clock parameters based on local clock information through EKF, the relative accuracy of the clock can be guaranteed even in the case of large network delays, thereby ensuring the synchronization of audio and video acquisition.

[0037] This solution uses dual-mode timing (GPS / Beidou) to ensure continuous timing in the event of any single system failure. Testing has shown a 38% improvement in positioning availability, with α = 0.5 and β = 0.3 corresponding to the optimal compensation values. This solution utilizes a satellite-based and locally compensated composite clock generation method, establishing an absolute time reference through dual-mode timing. The solution also incorporates a dynamic parameter compensation mechanism using a temperature-compensated crystal oscillator to eliminate hardware drift. A hierarchical synchronization architecture is employed: the master control device achieves microsecond-level precision time synchronization based on the PTP protocol, and cluster management is enabled in abnormal environments to form an autonomous time network. The dynamic buffer control module compensates for transmission delays through a nonlinear adjustment algorithm. In the event of a network outage, clock parameters are reconstructed using a state estimation model, forming a multi-level redundant synchronization assurance system.

[0038] Here, the temperature-compensated crystal oscillator in S2 has a frequency stability of ±0.5ppm and an operating temperature range of -40°C to 85°C. Within this temperature range, the temperature compensation coefficient α effectively compensates for clock deviations caused by temperature changes, ensuring that the pulse signal generated by the crystal oscillator provides a reliable basis for generating an accurate master clock signal under different ambient temperatures.

[0039] Here, step S4 is further optimized. In S4, the data buffer depth is dynamically adjusted, including:

[0040] S41. Calculate the average δt of the three most recent synchronization time differences;

[0041] S42. Calculate the time difference change rate dδt / dt and use sliding window filtering to eliminate pulse interference;

[0042] S43. Match the buffer depth level according to the preset mapping table and set the inter-level switching protection delay to 10ms.

[0043] Among them, S41 can reflect the general situation of the synchronization time difference between devices by counting the average value δt of the synchronization time difference of the last three times. S42 calculates the time difference change rate dδt / dt, and uses sliding window filtering to eliminate pulse interference. This can more accurately grasp the changing trend of the synchronization time difference and avoid misjudgment caused by pulse interference. S43 matches the buffer depth level according to the preset mapping table. The preset mapping table is established based on a large number of experiments and practical application experience. It corresponds different δt and dδt / dt values to the corresponding buffer depth level. Through the above operations, the data buffer depth can be finely adjusted dynamically, and the data buffer depth can be adjusted more accurately when the network conditions change dynamically, avoiding data loss or buffer overflow due to unreasonable buffer settings, and improving the stability and reliability of data transmission and processing.

[0044] Here, step S5 is further optimized to implement the cluster management mechanism in a non-network environment:

[0045] S51 establishes the central node of the master device N0 as the reference clock source;

[0046] S52. The slave device calibrates its local clock every 30 seconds according to the clock correction formula

[0047] C_slave(t) = C_master(t-Δt) + γ*RTT / 2, where Δt represents the delay factor and γ represents the path attenuation factor. Δt is calculated using the sliding window mean, and the window size N = 50 is determined based on actual measurements of industrial wireless networks.

[0048] S53. Measure the path delay RTT through two-way communication. The path attenuation factor γ is γ = 1-(1 / (SNR+1)). SNR is the signal-to-noise ratio of the measurement path, which is obtained by converting the RSSI value collected by the device RF front-end. Set the path attenuation factor γ to 0.9-1.1.

[0049] This cluster management mechanism solves the problem of lack of unified time base management when multiple devices collaborate to collect data in an offline environment, ensuring that each device can still maintain clock synchronization in the offline environment and guaranteeing the synchronization of audio and video collection.

[0050] Specific implementation steps:

[0051] Step S1: When deploying the master control device, install a dual-mode timing module based on the UBX-M8030 chipset. This chipset has a built-in, interference-resistant L1 / L2 dual-band receiver. Upon startup, it simultaneously receives GPS L1 C / A signals and Beidou B1I signals. The system prioritizes locking onto Beidou satellite signals and outputs the standard UTC time reference every 1 second.

[0052] Step S2: The master control device incorporates an OCXO-6535 temperature-compensated crystal oscillator with a frequency stability of ±0.5ppm and a temperature compensation range of -40°C to 85°C. The clock synthesis module calculates the compensation term based on the measured temperature sensor data, with a sampling rate of 10Hz: ΔT = TCXO_initial × Kt × (Tambient - Tcal), where Kt = 0.02°C-1 is the empirical temperature coefficient. The final master clock signal is expressed as T_master = T_GPS + 0.5 × ΔT + 0.3 × (Δf / f0). Compensation factors α = 0.5 and β = 0.3 are optimized online via the servo control system.

[0053] Step S3: The IEEE 1588 PTP protocol is implemented between the master and slave devices using the TIDP83640 physical layer chip. The master device sends Sync messages every second. These messages are encapsulated in UDP / IPv6 format, and the timestamp is written into the message payload. Drive tests have shown that within a 500-meter line-of-sight wireless mesh network, synchronization accuracy reaches ±85ns, meeting the nominal requirements for police equipment.

[0054] Step S4: The h.264 data stream output by the video codec enters the dynamic adjustment buffer:

[0055] a) Real-time collection of three adjacent synchronization time differences δt, currently -0.3ms, +0.5ms, +0.7ms.

[0056] b) Impulse noise is eliminated by a fourth-order Butterworth filter (dt = 1 ms).

[0057] c) Calculate the adjustment amount according to the formula D(t) = D0 + 0.15 × δt2 + 2.5 × dδt / dt: When δt = 0.3 ms and dδt / dt = 0.1 ms / s, the output is D(t) = 120 ms + 0.15 × (0.3)2 + 2.5 × 0.1 = 120 + 0.0135 + 0.25 ≈ 120.26 ms.

[0058] d) Convert the buffer depth according to the preset mapping table, with each level stepping 10ms and the maximum level corresponding to 200ms.

[0059] Step S5: When the round-trip delay is detected to be greater than 150ms for five consecutive times, the system switches to the local compensation mode.

[0060] a) Initialize EKF parameters: σ_t = 0.1 ms (time noise), σ_f = 0.05 ppm (frequency noise).

[0061] b) Execute state estimation: x_k = [1.05 0.98]^T × x_{k-1} + w_k.

[0062] c) Measurement update: The clock offset caused by relative motion is inferred based on the local accelerometer data.

[0063] d) Output the corrected clock parameters to all slave devices.

[0064] Under normal network conditions, the master device acquires the satellite time reference through a dual-mode timing module, generates a master clock signal, and synchronizes the clocks with various devices via the IEEE 1588PTP protocol. For example, law enforcement camera A and camera B successfully synchronize their clocks with the master device. During data transmission, due to fluctuations in network conditions, the device continuously adjusts the data buffer size according to the dynamic buffer control formula. Suppose, at a certain moment, the statistical mean of the time difference δt of the three most recent synchronizations changes, and the calculated rate of change of the time difference dδt / dt also shows fluctuations. The device matches the appropriate buffer depth level based on the preset mapping table and promptly adjusts the data buffer, ensuring stable data transmission and normal audio and video acquisition synchronization.

[0065] If a network failure occurs and the network delay exceeds 150ms, the system automatically switches to local clock compensation mode. For example, law enforcement camera C triggers the extended Kalman filter to recalibrate the clock parameters. Despite the poor network, it can still maintain a relatively accurate clock, ensuring that the audio and video data it collects is synchronized with other devices.

[0066] In a network-free environment, such as conducting law enforcement operations in the basement of a large shopping mall, a cluster management mechanism is established with the master device N0 as the central node of the reference clock source. Slave devices such as law enforcement recorder D calibrate the local clock according to the clock correction formula every 30 seconds. The path delay RTT is measured through two-way communication. Assuming that the SNR obtained by converting the RSSI value collected by the device RF front-end is 10, the path attenuation factor γ is calculated to be 1-(1 / (10+1))≈0.91, which is within the range of 0.9-1.1. In this way, each device can still achieve clock synchronization in a network-free environment, ensuring the synchronization of audio and video acquisition, and providing reliable audio and video recording for law enforcement activities.

[0067] This invention utilizes a redundant clock source configuration combining GPS / Beidou dual-mode timing with a temperature-compensated crystal oscillator (TCXO) to achieve time base synchronization with a precision of less than 15ms even in environments without network signals, such as basements and tunnels, eliminating the existing solution's strong dependence on network environments. A dynamic buffer control algorithm enables adaptive buffer adjustment based on real-time environmental factors, reducing the cumulative time error over 12 hours of continuous operation to less than 5ms, addressing the long-term cumulative error problem caused by traditional fixed buffer mechanisms. A clustered management time base tree and a dynamic master-slave device calibration mechanism enable millisecond-level collaborative synchronization for clusters of more than 30 nodes, reducing system resource utilization to 8.7%. An extended Kalman filter's abnormal disturbance compensation algorithm enables rapid recovery from sudden disturbances such as electromagnetic interference, achieving a recovery speed four times faster than traditional solutions. A heterogeneous synchronization architecture combining a hybrid clock source and the PTP protocol achieves audio and video synchronization errors of less than 8ms, far below the 20ms requirement of the GA / T 947-2015 standard, ensuring the spatiotemporal consistency of the law enforcement evidence chain.

[0068] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of the patent application of the present invention. Any equivalent changes, equivalent replacements or modified changes within the technical spirit and principles suggested by the present invention should be included in the scope of patent protection covered by the present invention.

Claims

1. A method for managing audio and video acquisition synchronization, executed by a master control device, characterized in that: include: S1. Obtain satellite time reference through dual-mode timing module; S2. Based on the pulse signal of the temperature-compensated crystal oscillator, the master clock signal is generated according to the formula T_master = T_GPS + αΔT + β(Δf / f0), where T_GPS is the satellite time reference, α is the temperature compensation coefficient, ranging from 0.2-0.8, and β is the frequency drift correction factor; S3. Establish a time-synchronized communication link between devices using the IEEE 1588 PTP protocol, with the master device broadcasting synchronization pulses. S4. Adjust the data buffer in real time according to the dynamic buffer control formula D(t) = D0 + k1*(δt)^2 + k2*dδt / dt, where k1 = 0.15ms^-2 and k2 = 2.5ms 2 / s; S5. When the network delay is detected to be greater than 150ms, the system switches to the local clock compensation mode and triggers the extended Kalman filter to recalibrate the clock parameters.

2. The method for synchronous management of audio and video acquisition according to claim 1, wherein: The temperature-compensated crystal oscillator in the S2 has a frequency stability of ±0.5ppm and an operating temperature range of -40°C to 85°C.

3. The method for synchronous management of audio and video acquisition according to claim 1, wherein: In S4, the data buffer depth is dynamically adjusted, including: S41. Calculate the average δt of the three most recent synchronization time differences; S42. Calculate the time difference change rate dδt / dt and use sliding window filtering to eliminate pulse interference; S43. Match the buffer depth level according to the preset mapping table and set the inter-level switching protection delay to 10ms.

4. The method for synchronous management of audio and video acquisition according to claim 1, wherein: The state equation of the extended Kalman filter in S5 is set as: x_k=[Δt,df]^T=A x_{k-1}+w_k, The system noise covariance matrix is configured as a diagonal matrix Q=diag(σ_t2, σ_f2), where σ_t=0.1ms, σ_f=0.05ppm, and the clock parameters are determined by measuring the crystal oscillator phase noise using a laboratory spectrum analyzer.

5. The method for synchronous management of audio and video acquisition according to claim 1, characterized in that: Implement cluster management mechanism in a non-network environment: S51 establishes the central node of the master device N0 as the reference clock source; S52. The slave device calibrates its local clock every 30 seconds according to the clock correction formula C_slave(t) = C_master(t-Δt) + γ*RTT / 2, where Δt represents the delay factor and γ represents the path attenuation factor. Δt is calculated using the sliding window mean, and the window size N = 50 is determined based on actual measurements of industrial wireless networks. S53. Measure the path delay RTT through two-way communication. The path attenuation factor γ is γ = 1-(1 / (SNR+1)). SNR is the signal-to-noise ratio of the measurement path, which is obtained by converting the RSSI value collected by the device RF front-end. Set the path attenuation factor γ to 0.9-1.1.

Images