Multipoint PTS correction method and system for audio synchronization, storage medium and equipment

Abstract

The invention relates to the technical field of digital audio signal processing, and discloses an audio synchronization multipoint PTS correction method and system, a storage medium and a device, and the method comprises the steps: setting PTS calibration points at a plurality of key nodes on an audio processing link of a master device and a slave device, and carrying out the systematic initialization of an initial PTS value of each calibration point; a clock synchronization mechanism between the master device and the slave device is established, and dynamic calibration of a slave device clock is achieved by periodically interacting time synchronization signals and calculating clock offset; performing multi-stage dynamic adjustment and compensation on the PTS value of the corresponding calibration point according to the network transmission delay fluctuation of the audio frame and the received actual coding duration and time deviation; the filling state and the PTS deviation value of the playing buffer area are monitored in real time, audio output synchronization between the master device and the slave device is maintained through a playing speed fine tuning strategy, optimization is carried out aiming at a lossy coding scene, and the limitation of a traditional single-point synchronization method is effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital audio signal processing, and in particular to an audio synchronization multi-point PTS correction method and system, a storage medium and equipment. BACKGROUND

[0002] With the rapid development and popularization of wireless audio technology, multi-device collaborative playback has become one of the core functions to improve user experience, and is widely used in multi-room intelligent sound system, true wireless stereo (TWS) earphone group, and distributed home theater scenes. However, under this application trend, the problem of audio synchronization is increasingly prominent, especially in the real environment where the master and slave devices use heterogeneous coding schemes, the traditional synchronization technology shows significant limitations.

[0003] Current mainstream audio synchronization methods mostly rely on the network time protocol (NTP) or a linear calculation mechanism based on single-point presentation time stamp (PTS). Under ideal network conditions, such methods can still maintain basic synchronization effects. However, the actual wireless environment is complex and variable, and there are differences in device resources and processing capabilities, and the traditional method exposes many shortcomings. First, the coding heterogeneity introduces a fundamental challenge: the master device usually uses pulse code modulation (PCM) uncompressed format to ensure sound quality, while the slave device often uses lossy coding formats such as LC3, SBC or AAC to reduce power consumption and transmission bandwidth. The frame length of lossy coding is variable, and the processing delay has a large uncertainty, resulting in cumulative errors in the PTS calculation mechanism based on fixed time length. Second, the inherent instability of the wireless channel, such as radio frequency interference, multipath attenuation, and dynamic load changes, causes random jitter in network transmission delay. The traditional linear delay prediction model cannot effectively adapt to this dynamic change, further destroying the consistency of the audio playback timing. In addition, although the frequency deviation of the local clock source between devices is small, it will cause a considerable drift effect in long-time playback. If an effective clock synchronization and compensation mechanism is not established, the playback gap between the master and slave devices will gradually expand over time. More importantly, existing synchronization strategies mostly focus on single-point correction at the end of playback, lacking the ability to systematically observe and regulate the whole link of audio data processing. In the entire process of audio frames from acquisition, encoding, transmission, reception to decoding and rendering, timing distortion at any link may cause the final output to be out of tune. This local correction method cannot cope with the coupling effects of coding fluctuations and transmission jitter, and it is also difficult to achieve sub-millisecond high-precision synchronization requirements at the global level.

[0004] Therefore, under the comprehensive challenge of multi-device, heterogeneous coding and dynamic network, the prior art generally causes perceptible delay difference of audio output, sound image positioning disorder, echo interference and other problems, which seriously affects the user's immersive experience. A new synchronization scheme is urgently needed that can run through the whole link of audio processing, adapt to coding and network fluctuations, and has the ability of dynamic coordination between devices to support the development of the next generation of high-reliability wireless audio applications. SUMMARY

[0005] The purpose of the present application is to solve the above-mentioned shortcomings in the prior art, and to provide a multi-point PTS correction method, system, storage medium and device for audio synchronization, which sets multiple PTS calibration points on the audio processing link of the master and slave devices, combines dynamic timestamp correction, adaptive buffer control and inter-device clock synchronization, realizes high-precision audio synchronization under lossy coding, and significantly improves the multi-device playback experience.

[0006] In one aspect, a multi-point PTS correction method for audio synchronization is provided, comprising the following steps: S1: setting PTS calibration points for timestamp calibration at multiple key nodes on the audio processing link of the master and slave devices, and systematically initializing the initial PTS values of each calibration point based on the capability parameter information of device interaction; S2: establishing a clock synchronization mechanism between the master and slave devices with the master device as the clock reference source, and realizing dynamic calibration of the slave device clock by periodically interacting time synchronization signals and calculating the clock offset; S3: dynamically adjusting and compensating the PTS values of the corresponding calibration points according to the network transmission delay fluctuation of the audio frame and the actual encoding time length and time deviation received; S4: Real-time monitoring of the filling state of the playback buffer and combining the accumulated PTS deviation value of each calibration point, maintaining the audio output synchronization between the master and slave devices through the playback speed fine-tuning strategy.

[0007] Further, in step S1, the key nodes include at least five nodes: an encoding pre-node after audio acquisition, an encoding post-node after encoding is completed, a transmission pre-node before network packet is sent, a decoding pre-node after data is received, and a decoding post-node before audio rendering.

[0008] Preferably, in step S1, the capability parameter information of the device includes the audio encoding format type supported by the master and slave devices, the size configuration range of the audio buffer, the estimated interval of the audio processing delay, and the precision index of the local clock of the device.

[0009] Further, in step S2, the clock synchronization mechanism includes: Collecting bidirectional timestamp interaction data between the master and slave devices, calculating bidirectional interaction round-trip delay, and calculating clock offset according to the round-trip delay and timestamp difference to dynamically compensate and calibrate the local time of the slave device.

[0010] Further, in step S3, the multi-stage dynamic adjustment and compensation includes: In the encoding processing stage, a correction coefficient is introduced for adaptive adjustment of the PTS value according to the difference between the actual encoding time consumption and the expected encoding time consumption of the current frame; In the network transmission stage, a smooth prediction algorithm is used to estimate the current transmission delay based on historical transmission delay statistical data and to correct the PTS value of the node before transmission accordingly; In the data receiving stage, the PTS value of the node before decoding is compensated and corrected by a set proportion according to the deviation between the actual receiving time of the message and the expected receiving time calculated according to the previous PTS.

[0011] Preferably, configurable correction coefficients are introduced in the PTS correction process in the encoding processing stage and the receiving stage and the receiving correction factor The value range of the correction coefficient is The receiving correction factor is used to control the smoothness and stability of the PTS adjustment amplitude.

[0012] Further, in step S4, the playback speed fine-tuning strategy includes: Based on the buffer level and the PTS deviation value, a non-linear function model is used to calculate the playback rate adjustment amount, and the adjustment range is controlled within a range that cannot be perceived by the human ear.

[0013] Further, step S2 further includes master-slave device synchronization coordination, specifically including: Periodically exchanging synchronization signals between the master and slave devices, including timestamps, PTS information and buffer status, and dynamically adjusting the synchronization signal interaction period for synchronization state alignment between the master and slave devices through real-time monitoring of network jitter indicators; By comparing the PTS values of the master and slave devices, the synchronization error is detected, and when the error exceeds a preset threshold, a fast synchronization mechanism is triggered; When a large synchronization error is detected, a fast recovery mechanism is used for fast synchronization recovery.

[0014] Further, step S3 further includes encoding format adaptive processing, specifically including: For the variable-length frame characteristics of the lossy coding format, the time-frequency domain features of the audio signal are extracted and the coding frame length is estimated based on a pre-trained frame length prediction model, and then the coding delay is fed forward compensated according to the predicted frame length to optimize the PTS reference; According to the network condition and synchronization requirement, the lossy coding quality parameter is dynamically adjusted, and under the premise of ensuring the synchronization accuracy, the sound quality and transmission efficiency are optimized.

[0015] Preferably, the time-frequency domain features of the audio signal at least include spectral entropy, zero-crossing rate and short-time energy, and the frame length prediction model outputs the frame length time prediction result by weighting and combining these feature values and adding a bias term.

[0016] Further, the method further includes system performance optimization, specifically including: The processor running frequency is dynamically adjusted, the buffer memory allocation strategy is optimized, and unnecessary periodic synchronization interaction is reduced, so as to realize the optimization of the overall power consumption of the system under the premise of meeting the preset synchronization accuracy threshold.

[0017] Preferably, the method further includes establishing an exception handling mechanism, monitoring the system running state in real time, and automatically triggering the corresponding recovery compensation process when various faults are detected, including but not limited to: When a network connection exception is detected, a network reconnection sequence is started to restore the communication link; When a device failure is detected, device state reset and resynchronization are performed to eliminate the impact of the failure; Empty the audio buffer and re-inject the reference timestamp to restore the playback continuity; Or enable a degraded synchronization mode to maintain basic playback functions at limited accuracy.

[0018] At the same time, when the buffer is in an abnormal state, the corresponding optimization strategy is implemented: If the buffer is in a low water level state, increase the pre-reading amount and appropriately reduce the playback speed; If the buffer is in a high water level state, reduce the pre-reading amount and appropriately increase the playback speed.

[0019] On the other hand, an audio synchronization multi-point PTS correction system is provided, including: A PTS initialization and configuration module for setting PTS calibration points for timestamp calibration on multiple key nodes of the audio processing link of the master device and the slave device, and systematically initializing the initial PTS value of each calibration point based on the capability parameter information of device interaction; A clock synchronization and management module for establishing a master-slave device clock synchronization mechanism with the master device as the clock reference source, achieving dynamic calibration of the slave device clock by periodically interacting time synchronization signals and calculating the clock offset. The PTS dynamic adjustment module is configured to perform multi-stage dynamic adjustment and compensation on the PTS value of the corresponding calibration point according to the network transmission delay fluctuation of the audio frame and the actual encoding time length and time deviation received; The adaptive buffer management module is configured to monitor the filling state of the playing buffer in real time and combine the accumulated PTS deviation value of each calibration point to maintain the audio output synchronization between the master device and the slave device through a playing speed fine-tuning strategy.

[0020] In addition, a computer readable storage medium is provided, and a computer program is stored on the computer readable storage medium.

[0021] Meanwhile, an electronic device is provided, which includes: one or more processors; a storage device configured to store one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the multi-point PTS correction method for audio synchronization according to any one of the preceding embodiments.

[0022] Compared with the prior art, the present application has the following beneficial effects: The present application effectively overcomes the accumulated error caused by the length fluctuation of lossy encoding frames and network transmission jitter by setting multiple PTS calibration points on the audio link between the master device and the slave device and monitoring and dynamically compensating the timestamps of each node, and achieves high-precision audio synchronization at the sub-millisecond level. The present application effectively compensates for the inherent clock drift between devices by establishing a master-slave device clock synchronization mechanism based on round-trip delay measurement and introducing periodic clock calibration signal interaction, and ensures that the system maintains stable synchronization during long-time playing. The present application can adaptively absorb network jitter and processing delay fluctuation and maintain smooth playing by monitoring the playing buffer level in real time and fine-tuning the playing speed based on the PTS deviation value using a nonlinear function, while avoiding pitch distortion (ensuring that the adjustment amplitude is within the range of human ear insensitivity). The present application improves the initial PTS calculation accuracy in the variable frame length encoding scenario by extracting the spectral entropy, zero-crossing rate and other characteristics of the audio signal and constructing a frame length prediction model to predict the frame length change of LC3 and other lossy encodings in advance, and then feedforward compensates the encoding delay. The present application constructs a perfect exception handling mechanism, so that the system can automatically and quickly restore the basic synchronization playing function when encountering network interruption, slave device failure or serious synchronization out-of-step and other abnormal conditions, greatly enhancing the reliability and user experience continuity of the system. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of the specification, illustrate embodiments of the application and are used to explain the application, but are not intended to limit the application. In the drawings: Figure 1 A flow chart of a multi-point PTS correction method for audio synchronization according to the present application; Figure 2 A block diagram of a multi-point PTS correction system structure for audio synchronization according to the present application; Figure 3 An embodiment of an electronic device according to the present application. DETAILED DESCRIPTION

[0024] To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0025] The present application sets PTS calibration points at key nodes (before encoding, after encoding, before transmission, before decoding, after decoding) of an audio processing link, monitors and corrects timestamp deviation in real time, introduces an adaptive synchronization algorithm based on buffer state, dynamically adjusts PTS values in combination with actual encoding duration of an audio frame and network transmission delay, and further establishes a clock synchronization mechanism between master and slave devices, eliminates the influence of clock drift between devices through periodic synchronization signal interaction. The method is particularly optimized for lossy encoding scenarios and effectively solves the limitations of traditional single-point synchronization methods.

[0026] The specific implementation of the present application will be described below in connection with the drawings and embodiments.

[0027] Embodiment 1 Please refer to Figure 1 A technical solution of a multi-point PTS correction method for audio synchronization provided by the present embodiment includes the following steps: S1: setting PTS calibration points for timestamp calibration at multiple key nodes on an audio processing link of a master device and a slave device, and systematically initializing initial PTS values of the calibration points based on capability parameter information of device interaction; S2: establishing a clock synchronization mechanism between the master and slave devices with the master device as a clock reference source, and dynamically calibrating the clock of the slave device by periodically interacting time synchronization signals and calculating clock offset; S3: According to the network transmission delay fluctuation of the audio frame and the actual encoding time length and time deviation received, the PTS value of the corresponding calibration point is dynamically adjusted and compensated in multiple stages; S4: Real-time monitoring of the filling state of the playback buffer and combining the accumulated PTS deviation value of each calibration point, the audio output synchronization between the master and slave devices is maintained through the playback speed fine-tuning strategy.

[0028] At system startup, first, the device role recognition is performed. In this embodiment, the master device (Master) adopts the PCM encoding format, and the slave device (Slave) adopts the LC3 48Kbps encoding format, then through the device discovery protocol, the communication connection between the master and slave devices is established, and the audio processing capability parameters of each device are exchanged.

[0029] Among them, the capability parameter information of the device includes the audio encoding format type supported by the master and slave devices, the size configuration range of the audio buffer, the estimated interval of the audio processing delay, and the precision index of the local clock of the device. In this embodiment, it specifically includes: Supported encoding format: E = {PCM, LC3, SBC, AAC}; Buffer size: B_size (unit: byte); Processing delay range: D_range = [D_min, D_max] (unit: millisecond); Clock accuracy: C_accuracy (unit: ppm).

[0030] Next, 5 key PTS calibration points are set in the audio processing link, which at least include: P1: the pre-encoding node after audio acquisition (audio input point), P2: the post-encoding node after encoding is completed (encoding completion point), P3: the pre-transmission node before network packet sending (network transmission point), P4: the pre-decoding node after data receiving (receiving decoding point), and P4: the post-decoding node before audio rendering (audio output point).

[0031] Among them, each calibration point maintains a PTS value: PTS_i (i = 1,2,3,4,5), and all PTS values are set to 0 at initialization. The initialization algorithm formula is as follows: Among them, is the system startup timestamp, represents the theoretical processing interval (calculated according to the audio sampling rate), is the calibration point number. Through this algorithm formula, a unified initial time reference is established for each calibration point in the audio processing pipeline.

[0032] Then the master-slave synchronization mechanism of step S2 is implemented, using the improved network time protocol. The master device serves as a clock reference source, and the slave device periodically performs clock calibration with the master device. Specifically, the clock synchronization mechanism includes: Bidirectional timestamp interaction data between the master-slave devices is collected, the bidirectional interaction round-trip delay is calculated, and the clock offset is calculated according to the round-trip delay and the timestamp difference, so as to dynamically compensate and calibrate the local time of the slave device.

[0033] Clock offset The calculation formula is as follows: Wherein, is the master device timestamp, is the slave device timestamp, indicates the round-trip delay (Round Trip Time). Through the calculation formula, the clock deviation between the master device and the slave device is accurately measured and quantified, providing a calibration basis for clock synchronization.

[0034] In this embodiment, it is assumed that the master device uses PCM encoding with a sampling rate of 48 kHz, and the slave device uses LC3 48 Kbps encoding. When the system starts, the master device timestamp is 1000 ms, the slave device timestamp is 1002 ms, and the measured RTT is 8 ms. Therefore, the clock offset is: , which indicates that the slave device needs to subtract 6 ms from the local timestamp for calibration.

[0035] Then, dynamic PTS calculation and correction of step S3 are performed, wherein the multi-stage dynamic adjustment and compensation of step S3 further includes: In the encoding processing stage, a correction coefficient is introduced for adaptive adjustment of the PTS value according to the difference between the actual encoding time consumption and the expected encoding time consumption. Specifically, in the P1 to P2 stage, the PTS value needs to be dynamically adjusted according to the actual encoding time. For PCM encoding of the master device, the processing delay is relatively fixed. For LC3 encoding of the slave device, since it is lossy encoding, the frame length will change according to the complexity of the audio content. The PTS correction formula in the encoding stage is as follows: Wherein, is the P1 node PTS value, is the P2 node PTS value, is the actual encoding time consumption, is the expected encoding time consumption, is the correction coefficient (a typical value is 0.3). Through the correction formula, the timestamp deviation introduced by the fluctuation of the encoding processing time (especially for lossy encoding) is dynamically compensated; In the network transmission stage, a smooth prediction algorithm is used to predict the current transmission delay based on historical transmission delay statistics and to correct the PTS value of the node before transmission. Specifically, in the P2 to P3 stage, the network transmission delay needs to be predicted, and the PTS value is adjusted accordingly. An adaptive prediction algorithm based on historical data is used, wherein the transmission delay prediction formula is as follows: wherein, is the historical average transmission delay, is the transmission delay of the previous frame, is a smoothing factor (typical value 0.7), and the obtained represents the prediction. The prediction formula adaptively predicts the current network transmission delay based on historical data, and provides forward-looking adjustment for the PTS; In the data receiving stage, the deviation between the actual receiving time of the packet and the expected receiving time calculated according to the previous PTS is used to compensate and correct the PTS value of the node before decoding according to the set proportion. Specifically, at the P4 point, the receiving end corrects the PTS according to the difference between the actual receiving time and the expected receiving time, wherein the PTS value correction formula of the receiving end P4 node is as follows: wherein, is the PTS_3 value received from the sending end, is the actual receiving time, is the expected receiving time. Through the correction formula, the final a posteriori correction is made at the receiving end based on the actual arrival time of the data packet, and the influence of prediction residual and random jitter is eliminated.

[0036] In this embodiment, the PTS of a certain audio frame at the P1 point of the main device is 1000 ms, and after PCM encoding, the actual time consumption is 2 ms (expected 1.5 ms), then: The frame is transmitted through the network, and the predicted transmission delay is 15 ms (based on historical data), then: s.

[0037] The step algorithm respectively introduces a configurable correction coefficient and a receiving correction factor in the PTS correction process in the encoding processing stage and the receiving stage, so as to control the smoothness and stability of the PTS adjustment amplitude.

[0038] Then, adaptive buffer management of step S4 is performed. First, the filling state of the playback buffer is monitored in real time, and the synchronization strategy is dynamically adjusted according to the buffer level. The buffer state is divided into three intervals: Low water level area: Buffer_level < 25%; Normal area: 25% ≤ Buffer_level ≤ 75%; High water level area: Buffer_level > 75%.

[0039] Wherein, the water level of the buffer The calculation formula is as follows: Wherein, The current buffer size, The maximum limit buffer size. Through the calculation formula, the filling state of the playing buffer is quantified in real time, and a core judgment basis is provided for the decision of the synchronization strategy.

[0040] On this basis, the playing speed is fine-tuned, including: Based on the buffer water level and the PTS deviation value, a non-linear function model is used to calculate the playing rate adjustment amount, and the adjustment range is controlled in a range that cannot be perceived by the human ear. The adjustment amount The calculation formula is as follows: Wherein, The maximum adjustment amplitude (typical value 0.02, i.e. 2%), The PTS deviation value, The adjustment sensitivity parameter (typical value 50ms), Indicates the hyperbolic tangent function, which is used to limit the adjustment amplitude. The adjustment formula intelligently adjusts the playing rate in a smooth, limited and inaudible way to the human ear according to the synchronization deviation (PTS_error), so as to absorb the delay jitter and maintain synchronization.

[0041] In this embodiment, at a certain moment, the filling degree of the device buffer is 30%, and the PTS deviation is-20ms (playing lag), then: Indicates that the playing speed is reduced by about 0.76%, which helps to alleviate the playing lag problem.

[0042] The method of the application further includes master-slave device synchronization coordination, specifically including: Periodically exchange synchronization signals between master and slave devices, including time stamp, PTS information and buffer state, and dynamically adjust the synchronization signal interaction period between the master and slave devices for synchronization state alignment through real-time monitoring of network jitter indicators; By comparing the PTS values of the master and slave devices, the synchronization error is detected, and when the error exceeds the threshold, the fast synchronization mechanism is triggered; When a large synchronization error is detected, a fast recovery mechanism is used for fast synchronization recovery.

[0043] Wherein, the synchronization period adjustment amount The calculation formula is as follows: wherein, is the base synchronization period (typical value 100ms), is the adjustment factor (typical value 0.5), is the network jitter value. This calculation formula adaptively adjusts the interaction frequency of the synchronization signal between the master and slave devices according to the current network quality (jitter degree), realizing the optimal balance between synchronization overhead and stability.

[0044] Then, by comparing the PTS values of the master and slave devices, the synchronization error is detected. When the error exceeds the threshold value, the fast synchronization mechanism is triggered. The synchronization error The calculation formula is as follows: wherein, is the expected offset due to encoding differences. Through this synchronization error calculation formula, the real synchronization deviation between the master and slave devices is accurately quantified, providing accurate decision basis for judging the system state and triggering the recovery mechanism.

[0045] When a large synchronization error is detected, the fast recovery mechanism is adopted, including: pausing audio playback, emptying the buffer, re-establishing the PTS reference, and resuming playback.

[0046] In this embodiment, if the network jitter value is 10ms, the synchronization period is adjusted to: Increase the synchronization frequency to cope with network instability.

[0047] When performing adaptive adjustment of the PTS value in step S3, we also perform adaptive processing of the encoding format, which specifically includes: For the variable-length frame characteristics of lossy encoding formats, the time-frequency domain features of the audio signal are extracted and the encoding frame length is estimated based on a pre-trained frame length prediction model, and then the encoding delay is fed forward compensated according to the predicted frame length to optimize the PTS reference; According to the network status and synchronization requirements, the quality parameters of lossy encoding are dynamically adjusted to optimize the sound quality and transmission efficiency on the premise of ensuring synchronization accuracy.

[0048] Wherein, the time-frequency domain features of the audio signal at least include spectral entropy, zero-crossing rate and short-time energy, and the frame length prediction model combines these feature values by linear weighting and adds a bias term to output the frame length prediction result.

[0049] Specifically, the LC3 encoding characteristics are that its frame length will vary according to the complexity of the audio content, and a frame length prediction model needs to be established. Machine learning method based on audio features is adopted.

[0050] The LC3 frame length prediction algorithm formula is as follows: wherein, is the feature weight, is the audio feature value (including spectral entropy, zero-crossing rate, energy, etc.), is the bias term, denotes the predicted next frame length of the output. The prediction algorithm formula establishes a prediction model based on the audio signal features to estimate the dynamic frame processing time of LC3 lossy encoding in advance, overcoming its inherent uncertainty.

[0051] Then according to the predicted frame length, the encoding delay is calculated in advance and PTS compensation is performed. The delay compensation amount is calculated as follows: wherein, is the average frame length, is the compensation coefficient (typical value 0.8).

[0052] Finally, according to the network condition and synchronization requirement, the LC3 encoding quality parameter is dynamically adjusted. Under the premise of ensuring the synchronization accuracy, the sound quality and transmission efficiency are optimized. The compensation amount calculation formula uses the frame length prediction result to perform forward-looking compensation on the original time stamp (PTS), and actively offsets the timing deviation caused by the change of encoding complexity.

[0053] In this embodiment, for a complex music segment, after audio feature extraction, the spectral entropy f1 = 0.8, the zero-crossing rate f2 = 0.3, and the energy f3 = 0.6 are obtained. Assuming that the weight is , , the bias is: .

[0054] The method further includes system performance optimization, specifically including: dynamically adjusting the processor running frequency, optimizing the buffer memory allocation strategy, and reducing unnecessary periodic synchronization interaction, so as to realize the optimization of the overall power consumption of the system under the premise of meeting the preset synchronization accuracy threshold.

[0055] Specifically, we reduce the system power consumption through intelligent scheduling and resource management. Including: dynamically adjusting the processor frequency, optimizing the buffer size, and reducing unnecessary synchronization interaction. Among them, the power consumption optimization objective function is expressed as follows: , wherein, is the processor power consumption, is the transmission power consumption, is the buffer power consumption, , and are weight coefficients of the above three power consumptions, respectively, the synchronization accuracy needs to be greater than a preset threshold . The optimization objective function establishes a global, quantitative energy efficiency optimization objective for the entire audio synchronization system, and intelligently coordinates the power consumption distribution of different modules under the hard constraint of ensuring synchronization accuracy, to achieve the best energy efficiency performance at the system level.

[0056] During the process, we need to monitor performance in real time, and establish a performance index monitoring system, including: synchronization accuracy: ±1ms, delay jitter: <5ms, buffer utilization rate: 30-70%, and power efficiency: reduced by 15% compared with traditional methods.

[0057] On this basis, we introduce an exception handling mechanism to monitor the system running state in real time, and automatically trigger the corresponding recovery compensation process when various faults are detected, including but not limited to: When network connection anomalies are detected, start the network reconnection sequence to restore the communication link; When device failures are detected, perform device state reset and resynchronization to eliminate the impact of failures; Empty the audio buffer and re-inject the reference timestamp to restore the continuity of playback; Or enable a degraded synchronization mode to maintain basic playback functions at limited precision; At the same time, when the buffer is in an abnormal state, implement the corresponding optimization strategy, wherein if the buffer is in a low water level state, increase the pre-reading amount and appropriately reduce the playback speed; if the buffer is in a high water level state, vice versa.

[0058] In the optimization process of the present embodiment, by adjusting the synchronization period from 100ms to 200ms, the transmission power consumption can be reduced by about 20%, while maintaining the synchronization accuracy within ±2ms, meeting the system requirements.

[0059] In summary, through the above method, specifically through the multi-point correction mechanism, the audio synchronization accuracy can reach ±1ms, which is more than 10 times higher than traditional methods, and can adapt to different encoding formats, network environments and device performance differences, maintaining stable synchronization effect. At the same time, through intelligent buffer management and adaptive synchronization strategy, the system power consumption is reduced by 15%, improving the device endurance, supporting mainstream audio encoding formats such as PCM, LC3, SBC, AAC, etc., suitable for various wireless audio devices. In addition, under the interference factors such as network jitter and device load change, it can still maintain high-quality audio synchronization effect, eliminating problems such as channel misalignment and echo, and realizing high-quality multi-device collaborative playback experience.

[0060] In another aspect, the embodiment also provides an audio-synchronized multi-point PTS correction system, as shown in Figure 2 comprising: a PTS initialization and configuration module 10, configured to set PTS calibration points for timestamp calibration at a plurality of key nodes on an audio processing link between a master device and a slave device, and to systematically initialize initial PTS values of the calibration points based on capability parameter information of device interaction; a clock synchronization and management module 20, configured to establish a clock synchronization mechanism between the master device and the slave device with the master device as a clock reference source, and to realize dynamic calibration of the slave device clock by periodically interacting time synchronization signals and calculating clock offset; a PTS dynamic adjustment module 30, configured to perform multi-stage dynamic adjustment and compensation on PTS values of the corresponding calibration points according to network transmission delay fluctuations of audio frames and actual encoding time length and time deviation received; an adaptive buffer management module 40, configured to monitor filling states of a playing buffer in real time and combine accumulated PTS deviation values of the calibration points, and to maintain audio output synchronization between the master device and the slave device through a playing speed fine-tuning strategy.

[0061] It should be noted that the steps in the low-latency multi-room audio playing optimization method provided in the embodiment can be implemented based on corresponding modules in the low-latency multi-room audio playing optimization system, and those skilled in the art can refer to the technical solutions of the system to implement the step flow of the method, that is, the embodiments in the system can be understood as preferred examples of implementing the method, and will not be described here.

[0062] The embodiment also provides an electronic device, as shown in Figure 3 The electronic device includes a processor 14 and a memory 13, and the memory 13 stores machine executable instructions executable by the processor 14, and the processor 14 executes the machine executable instructions to implement the above-mentioned audio control method.

[0063] Further, Figure 3 The electronic device shown in the figure also includes a bus 12 and a communication interface 11, and the processor 14, the communication interface 11 and the memory 13 are connected through the bus 12.

[0064] The memory 13 can include a high-speed random access memory (RAM), and can also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 11 (which can be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used. The bus 12 can be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used to represent the system network element and at least one other network element, but it does not mean that there is only one bus or one type of bus.

[0065] The processor 14 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 14 or the instructions in the form of software. The processor 14 described above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Each method, step and logic block disclosed in the embodiment can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiment can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 1001, and the processor 1000 reads the information in the memory 1001, and combines the hardware to complete the steps of the audio control method.

[0066] The present disclosure also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium or a volatile computer readable storage medium, and the computer readable storage medium stores a computer program, and when the computer program runs on a computer, the computer program causes the computer to execute the steps of the audio control method.

[0067] Finally, it should be noted that the above only describes the preferred embodiments of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solutions falling within the concept of the present application shall be considered as the protection scope of the present application. It should be pointed out that, for ordinary skilled in the art, some improvements and refinements without departing from the principles of the present application are also considered as the protection scope of the present application.

[0068] The technical features of the above-described embodiments can be combined arbitrarily, and in order to make the description concise, all possible combinations of the technical features in the above-described embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present disclosure.

Claims

1. A multi-point PTS correction method for audio synchronization, characterized in that: The steps include: S1: Set PTS calibration points for timestamp calibration at multiple key nodes in the audio processing chain of the master and slave devices, and systematically initialize the initial PTS value of each calibration point based on the capability parameter information of the device interaction; S2: Establish a clock synchronization mechanism between master and slave devices with the master device as the clock reference source. Dynamic calibration of the slave device clock is achieved by periodically exchanging time synchronization signals and calculating clock offsets. S3: Based on the fluctuation of the network transmission delay of the audio frame and the actual received encoding duration and time deviation, the PTS value of the corresponding calibration point is dynamically adjusted and compensated in multiple stages; S4: Real-time monitoring of the filling status of the playback buffer and the accumulated PTS deviation value of each calibration point are combined to maintain the audio output synchronization between the master and slave devices through the playback speed fine-tuning strategy.

2. The multi-point PTS correction method for audio synchronization according to claim 1, wherein In step S1, the key nodes include at least five nodes: a pre-encoding node after audio acquisition, a post-encoding node after encoding is completed, a pre-transmission node before network packet sending, a pre-decoding node after data reception, and a post-decoding node before audio rendering.

3. The multi-point PTS correction method for audio synchronization according to claim 1, wherein: In step S1, the capability parameter information of the device includes the audio encoding format types supported by the master and slave devices, the size configuration range of the audio buffer, the estimated interval of the audio processing delay, and the accuracy index of the local clock of the device.

4. The multi-point PTS correction method for audio synchronization according to claim 1, wherein: In step S2, the clock synchronization mechanism includes: The two-way timestamp interaction data between the master and slave devices is collected, the round-trip delay of the two-way interaction is calculated, and the clock offset is calculated based on the round-trip delay and the timestamp difference to dynamically compensate and calibrate the local time of the slave device.

5. The multi-point PTS correction method for audio synchronization according to claim 1, wherein: In step S3, the multi-stage dynamic adjustment and compensation further includes: During the encoding process, a correction coefficient is introduced to adaptively adjust the PTS value based on the difference between the actual encoding time and the expected encoding time of the current frame; During the network transmission phase, a smooth prediction algorithm is used based on historical transmission delay statistics to estimate the current transmission delay and to modify the PTS value of the node before transmission accordingly; During the data reception phase, the PTS value of the node before decoding is compensated and corrected according to a set ratio based on the deviation between the actual message reception time and the expected reception time calculated based on the previous PTS.

6. The multi-point PTS correction method for audio synchronization according to claim 5, wherein: Configurable correction coefficients are introduced in the PTS correction process of the encoding processing stage and the receiving stage respectively and receiving correction factor , the correction coefficient The value range is , the receiving correction factor Used to control the smoothness and stability of the PTS adjustment range.

7. The multi-point PTS correction method for audio synchronization according to claim 1, wherein: In step S4, the play speed fine-tuning strategy includes: Based on the buffer level and the PTS deviation value, a nonlinear function model is used to calculate the playback rate adjustment amount, and the adjustment range is controlled within a range that is imperceptible to the human ear.

8. The multi-point PTS correction method for audio synchronization according to claim 4, wherein: Step S2 also includes synchronization coordination between the master and slave devices, specifically including: The master and slave devices periodically exchange synchronization signals, including timestamps, PTS information, and buffer status. The synchronization signal exchange period for aligning the synchronization status between the master and slave devices is dynamically adjusted based on the real-time monitoring of network jitter indicators. By comparing the PTS values ​​of the master and slave devices, the synchronization error is detected. When the error exceeds the preset threshold, the fast synchronization mechanism is triggered; When a large synchronization error is detected, a fast recovery mechanism is used to quickly restore synchronization.

9. The multi-point PTS correction method for audio synchronization according to claim 1, wherein: Step S3 also includes coding format adaptive processing, specifically including: In view of the variable-length frame characteristics of lossy coding formats, the time-frequency domain features of the audio signal are extracted and the coding frame length is estimated based on a pre-trained frame length prediction model. Then, the coding delay is feed-forward compensated according to the predicted frame length to optimize the PTS benchmark. Dynamically adjust lossy encoding quality parameters based on network conditions and synchronization requirements to optimize sound quality and transmission efficiency while ensuring synchronization accuracy.

10. The multi-point PTS correction method for audio synchronization according to claim 9, characterized in that: The time-frequency domain features of the audio signal include at least spectral entropy, zero-crossing rate and short-time energy. The frame length prediction model outputs a frame length prediction result by weighted combination of these feature values ​​and adding a bias term.

11. The multi-point PTS correction method for audio synchronization according to claim 1, wherein: The method also includes system performance optimization, specifically including: Dynamically adjust the processor operating frequency, optimize the buffer memory allocation strategy and reduce unnecessary periodic synchronization interactions to optimize the overall system power consumption while meeting the preset synchronization accuracy threshold.

12. The multi-point PTS correction method for audio synchronization according to claim 1, wherein: The method also includes establishing an exception handling mechanism to monitor the system operation status in real time and automatically trigger corresponding recovery and compensation processes when various faults are detected. The processes include but are not limited to: When a network connection anomaly is detected, a network reconnection sequence is initiated to restore the communication link; When a device failure is detected, the device status is reset and resynchronized to eliminate the impact of the failure; Clear the audio buffer and reinject the base timestamp to restore playback continuity; Or enable degraded sync mode to maintain basic playback functionality with limited accuracy.

13. The multi-point PTS correction method for audio synchronization according to claim 12, characterized in that: When the buffer is in an abnormal state, implement the corresponding optimization strategy: If the buffer is at a low water level, increase the pre-read amount and reduce the playback speed appropriately; If the buffer is at a high water level, reduce the pre-read amount and increase the playback speed appropriately.

14. A multi-point PTS correction system for audio synchronization, characterized in that: include: The PTS initialization and configuration module is used to set PTS calibration points for timestamp calibration at multiple key nodes in the audio processing chain of the master and slave devices, and systematically initialize the initial PTS value of each calibration point based on the capability parameter information of the device interaction; The clock synchronization and management module is used to establish a clock synchronization mechanism between master and slave devices with the master device as the clock reference source. It achieves dynamic calibration of the slave device clock by periodically exchanging time synchronization signals and calculating clock offsets. The PTS dynamic adjustment module is used to dynamically adjust and compensate the PTS value of the corresponding calibration point in multiple stages according to the fluctuation of the network transmission delay of the audio frame and the actual encoding duration and time deviation received; The adaptive buffer management module is used to monitor the filling status of the playback buffer in real time and, combined with the PTS deviation value accumulated at each calibration point, maintain the audio output synchronization between the master and slave devices through a playback speed fine-tuning strategy.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the multi-point PTS correction method for audio synchronization according to any one of claims 1 to 13 is implemented.

16. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the multi-point PTS correction method for audio synchronization as described in any one of claims 1-13.

Images