A media broadcast master-slave node switching method, system and medium

Abstract

The application discloses a media broadcast master-slave node switching method and system and a medium, relates to the technical field of media broadcast, and realizes real-time collection of master node playing state update information, running information and master node fault information on the premise of keeping the time synchronization of a slave node and a master node; the master node playing state update information is analyzed, the local playing state cache of the slave node is updated according to the analysis information, and an AI monitoring model performs state evaluation on the master node; when the master node fault information is collected or the master node state is evaluated to be abnormal, a switching instruction is triggered; the switching instruction is executed based on an application layer switching protocol; the scheme is improved in the method based on the prior art, high-precision time synchronization technology, an application layer protocol and artificial intelligence monitoring are fused, the audio stream can be rapidly and seamlessly switched to the slave node when the master node is down or the network is abnormal, the continuity of client playing is ensured, and the reliability of audio broadcast is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of media broadcast, in particular to a media broadcast master-slave node switching method, system and medium. BACKGROUND

[0002] In order to improve the reliability of the media broadcast system, the master-slave (or master-backup) hot backup architecture is widely used, in which the master server processes and outputs the audio stream, and the slave server keeps state synchronization with the master server in real time or quasi-real time, and takes over the work when the master server fails.

[0003] In recent years, AoIP technologies such as Dante standard, AES67 standard and SMPTE ST 2022-7 (seamless protection switching) standard at network level have been widely used in professional audio field; the AES67 standard specifies the basis of audio interoperability, and relies on IEEE 1588 PTP (Precision Time Protocol) protocol for accurate clock synchronization between devices; the Dante standard provides a more perfect device discovery, routing configuration and clock management scheme based on the AES67 standard, and supports the redundancy of master and backup network ports; the SMPTE ST 2022-7 standard focuses on realizing seamless protection switching at the network transmission layer by transmitting the same IP packets (usually two identical AoIP streams) on two independent network paths, and the receiving end reconstructs the lossless data stream by comparing the packet sequence number and arrival time, effectively resisting network packet loss or single path failure.

[0004] The Dante standard, AES67 standard and SMPTE ST 2022-7 standard and the like provide powerful network layer and transmission layer, but in the aspect of realizing truly non-perceptual smooth switching between master-slave audio processing application units, there are still the following shortcomings:

[0005] 1. The synchronization accuracy of the application layer audio content is insufficient, and it is difficult to avoid misalignment or stuttering depending on the standard itself:

[0006] The AES67 / PTP standard provides accurate clock synchronization at the device level, but does not specify the frame / sampling level synchronization mechanism of the audio stream content between master-slave application instances. Even if the two independent audio processing units (application software of the master server and the slave server) are clock-synchronized, their respective internal audio processing pipelines, playback pointers or content generation logic may still have a slight deviation in audio content at the time of switching, resulting in misalignment or stuttering.

[0007] The redundancy of Dante standard is mainly reflected in the network interface level. When the application software or hardware of the main audio processing unit fails, rather than just the network connection problem, the Dante standard itself does not guarantee that the audio content before the main unit interruption can be seamlessly continued from the audio processing unit with sampling precision. The control and timing of switching the audio routing (such as switching from the main Dante sending device to the standby Dante sending device) depend on the higher layer application, and if there is no accurate content alignment, the switching will still be perceived.

[0008] The SMPTE ST 2022-7 standard provides "seamless packet merging" under network path redundancy, that is, it ensures that a single AoIP stream can obtain packets from another path when the network transmission fails (such as a network cable being disconnected) so as not to be interrupted. However, it does not solve the switching problem when the main audio source device / application itself fails; if the main audio processing unit fails, causing the AoIP stream output by it to be interrupted or the content to be incorrect, ST 2022-7 cannot "create" the missing main path content from the stream of a healthy standby audio processing unit to achieve seamless switching. It protects the "transmission path of the stream", rather than the "source of the stream".

[0009] 2. The application layer lacks switching logic and content alignment, which may introduce "pauses" or switching traces: the above standards mainly focus on clock synchronization and data transmission reliability, but the switching decision logic (when to switch), switching execution mechanism (how to switch), and most importantly, the accurate connection of audio content at the switching point between the main and standby audio processing applications, usually need to be designed by the user or system integrator at the application level. The lack of a standardized application layer switching protocol that is closely integrated with content generation / processing may cause delays in switching decisions and execution, resulting in perceptible audio pauses or switching noise.

[0010] Even if the clock is synchronized and the network is reliable, if the slave server cannot accurately predict the audio output state of the master server (for example, accurately to the identification of the next audio sample / frame to be played) before switching through the application layer mechanism, and prepare completely aligned audio data in the internal buffer, the transition cannot be guaranteed to be "zero gap" or "phase continuous" when switching.

[0011] 3. Complexity of state and parameter synchronization: In addition to the audio stream itself, complex audio processing applications (such as mixing consoles, effectors, automated playback systems) also contain a large number of real-time state parameters (such as volume, sound equalization parameters, dynamics, routing, playlist pointers, etc.). The accurate and low-latency synchronization of these application layer parameters is crucial for achieving truly imperceptible switching, and the existing AoIP standards do not cover the synchronization of these application layer states.

[0012] These techniques provide a solid foundation for building reliable audio systems, but there are still challenges in achieving truly seamless and non-perceptible application-level smooth switching between primary and backup audio processing units, not just network paths. SUMMARY

[0013] The technical problem to be solved by the present application is that the traditional audio switching technology has the problems of insufficient audio content synchronization accuracy, difficulty in covering application layer state synchronization due to dependence on standards, and obvious misalignment or stuttering. The present application aims to provide a media broadcast master-slave node switching method, system and medium, which improves the method based on the existing technology, and through the integration of high-precision time synchronization technology, application layer protocol and artificial intelligence monitoring, can quickly and seamlessly switch the audio stream to the slave node when the host machine is down or the network is abnormal, ensuring continuous, uninterrupted and lossless audio quality for the client.

[0014] The present application is implemented by the following technical solutions:

[0015] The present application provides a media broadcast master-slave node switching method, which includes:

[0016] Under the premise of maintaining time synchronization between the slave node and the master node:

[0017] Real-time collection of master node playback state update information, running information and master node fault information;

[0018] Analyzing the master node playback state update information to obtain analysis information, and updating the local playback state cache of the slave node according to the analysis information, while the AI monitoring model performs state evaluation on the master node based on the analysis information and the running information;

[0019] When the master node fault information is collected or the AI monitoring model evaluates that the master node state is abnormal, a switching instruction is triggered;

[0020] Based on the application layer switching protocol, the switching instruction is executed.

[0021] Further optimization scheme is that the method for time synchronization between the slave node and the master node includes:

[0022] Initialization of the time reference of the master node and the slave node: the master node runs a master clock and periodically broadcasts a first message to the slave node, the first message containing a master node timestamp; the slave node runs a slave clock and receives the first message; the slave node records a local timestamp and embeds it in the audio frame header;

[0023] Obtaining the round-trip delay and physical distance between the master node and the slave node;

[0024] Based on the round-trip delay and the physical distance, the local timestamp of the slave node is updated in state based on an improved Kalman filter and a state transition method; in the process, the master node timestamp is used as an observation value, the linear scaling of the local timestamp is realized based on the drift coefficient between the master node and the slave node, and the process noise covariance and the observation noise covariance are dynamically adjusted according to the observation index; the observation index includes: observation residual, round-trip delay jitter and packet loss rate;

[0025] The local timestamp of the slave node is updated based on the state update result.

[0026] Further optimization scheme is that the overall process model of the state update includes:

[0027]

[0028] Wherein, represents the local timestamp of the slave node corresponding to the kth update; represents the local timestamp of the slave node corresponding to the k-1th update; represents the round-trip delay between the master node and the slave node corresponding to the kth update; represents the signal propagation delay when measuring the physical distance; represents the drift coefficient between the master node and the slave node corresponding to the kth update; represents the system noise.

[0029] Further optimization scheme is that the method supports limited deployment and wireless deployment, and the physical distance is obtained based on a UWB ranging module;

[0030] In wired deployment, =0;

[0031] In wireless deployment, =d / c; wherein, d represents the physical distance between the master node and the slave node; c represents the propagation speed of electromagnetic wave in air.

[0032] Further optimization scheme is that the drift coefficient between the master node and the slave node is calculated according to the following formula:

[0033]

[0034] Wherein, represents the drift coefficient between the master node and the slave node corresponding to the kth update; represents the initial local timestamp recorded by the slave node; represents the local timestamp of the slave node corresponding to the kth update; represents the clock frequency deviation of the slave node relative to the master node corresponding to the kth update; represents the local timestamp of the slave node obtained by the kth update; represents the initial offset between the master node timestamp and the initial local timestamp of the slave node.

[0035] A further optimization scheme is that the updating of the local playback state cache of the slave node according to the parsed information comprises the following method:

[0036] The master node encapsulates the playback state update information into ALSP state information and sends it to the slave node and the client, and inserts the master node timestamp into the audio frame to form an audio stream and sends it to the slave node;

[0037] The slave node listens to and parses the ALSP state information and the audio stream in real time, and parses the frame ID, the master node timestamp, the playback pointer position and the audio processing parameters of the current playback frame of the master node based on the ALSP state information;

[0038] The slave node updates the playback pointer position and the audio processing parameters to the local playback state cache area, sorts the audio frames in the audio stream according to the master node timestamp in sequence, and updates the audio frames to the local playback content cache area; the local playback content cache area maintains at least N audio frames.

[0039] A further optimization scheme is that the AI monitoring model comprises a trained LSTM prediction model or a Transformer model; and the running information comprises CPU usage trend, memory occupation trend, network bandwidth, network quality, heartbeat signal and master node timestamp continuity. The LSTM prediction model is an improved structure of a recurrent neural network (RNN) and has long-term memory capability, is suitable for processing time series data, and is widely used in fields such as speech recognition, music generation, text-to-speech synthesis, etc.

[0040] A further optimization scheme is that the method of executing the switching instruction based on the application layer switching protocol comprises:

[0041] For the switching instruction triggered by the AI monitoring model evaluation of the master node state exception, directly switch to the slave node based on the application layer switching protocol;

[0042] For the switching instruction triggered by the master node fault information, it is judged whether the slave node has a local audio source file corresponding to the audio stream, if yes, a double-layer cache switching strategy is enabled; otherwise, a PLC filling cache switching strategy is enabled.

[0043] The double-layer cache switching strategy comprises: determining a playback starting point of the slave node based on a drift coefficient between the master node and the slave node; finding an audio frame i corresponding to a playback starting point timestamp from a local audio source file, loading N audio frames subsequent to the audio frame i to a second layer cache area, reading the audio frame by the first layer cache area from the second layer cache area, and issuing the audio frame read by the first layer cache area after being labeled with a local timestamp by the slave node;

[0044] The PLC filling cache switching strategy comprises: extracting an audio stream segment of a current timestamp context of the master node, inputting an AI prediction model to generate a filling audio frame loaded to the first layer cache area, and issuing the audio frame in the first layer cache area after being labeled with a local timestamp by the slave node. The PLC in the PLC filling cache switching strategy represents an enhanced PLC algorithm.

[0045] The scheme also provides a media broadcast master-slave node switching system for implementing the media broadcast master-slave node switching method.

[0046] The synchronization module is configured to keep the slave node in time synchronization with the master node.

[0047] The acquisition module is configured to acquire master node playback state update information, running information and master node fault information in real time.

[0048] The analysis and evaluation module is configured to analyze the master node playback state update information to obtain analysis information, and update the local playback state cache of the slave node according to the analysis information, and the AI monitoring model performs state evaluation on the master node based on the analysis information and the running information.

[0049] The trigger module is configured to trigger a switching instruction when the master node fault information is acquired or the AI monitoring model evaluates that the state of the master node is abnormal.

[0050] The execution module is configured to execute the switching instruction based on an application layer switching protocol.

[0051] The scheme also provides a computer readable medium having a computer program stored thereon, and the computer program is executed by a processor to implement the media broadcast master-slave node switching method.

[0052] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0053] 1. The present application provides a master-slave node switching method, system and medium for media broadcast. Based on the prior art, the method is improved by fusing high-precision time synchronization technology, application layer protocol and artificial intelligence monitoring. When the master node is down or the network is abnormal, the audio stream can be quickly and seamlessly switched to the slave node, ensuring continuous, uninterrupted and lossless audio quality for the client playback.

[0054] 2. The application provides a master-slave node switching method, system and medium for media broadcasting, which realizes timestamp synchronization of audio frames of the master node and the slave node by combining an improved Kalman filter, a state transition method and UWB ranging technology, and the synchronization error is less than ±1 microsecond.

[0055] 3. The application provides a master-slave node switching method, system and medium for media broadcasting, which continuously synchronizes audio processing parameters (such as volume and equalizer) to ensure consistency of the master node and the slave node; the application layer switching protocol (ALSP) is applied to coordinate switching time and playing state to ensure zero gap and phase continuous transition; the application ensures fast, seamless and user-perception-free master-slave switching, and significantly improves the reliability of audio broadcasting.

[0056] 4. The application provides a master-slave node switching method, system and medium for media broadcasting, which uses an LSTM or Transformer model to monitor the health status of the master node, predict faults and actively trigger a switching instruction; periodic checking ensures alignment of the master node and the slave node, and instant switching is realized.

[0057] 5. The application provides a master-slave node switching method, system and medium for media broadcasting, which uses a double-layer buffer switching strategy, loads audio frames to a second layer buffer area, the first layer buffer area reads audio frames from the second layer buffer area, and the slave node labels local timestamps on the audio frames read by the first layer buffer area and then sends out the audio frames, so that seamless switching is realized and listeners have no perception. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the application, and therefore should not be regarded as a limitation on the scope. Other related drawings can also be obtained by those skilled in the art without creative labor. In the drawings:

[0059] Figure 1 Fig. 1 is a flowchart of a media broadcasting master-slave node switching method.

[0060] Figure 2 Fig. 2 is a schematic diagram of a media broadcasting master-slave node switching system structure. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the application more clear and apparent, the following will further describe the application in combination with embodiments and drawings. The exemplary embodiments of the application and their descriptions are only used to explain the application, and should not be regarded as a limitation on the application.

[0062] The traditional audio switching technology has the problems of insufficient audio content synchronization accuracy, difficulty in covering the application layer state synchronization due to the dependence on the standard itself, and obvious misalignment or stuttering. In view of this, the present scheme provides the following embodiments to solve the above technical problems.

[0063] Embodiment 1: The present embodiment provides a media broadcast master-slave node switching method, as shown in the following formula (1), which comprises the following steps: Figure 1

[0064] The subsequent steps 1 to 4 are implemented under the premise of maintaining the time synchronization of the slave node with the master node.

[0065] In the audio broadcast master node slave node switching process, high-precision timestamp synchronization between the master node and the slave node is the core prerequisite to ensure seamless switching. If there is a time deviation between the master node and the slave node, even if it is only a few milliseconds, it may cause audio frame misalignment, playback interruption or sudden change in sound quality. In the traditional audio broadcast technology, the time synchronization between the master node and the slave node usually depends on the IEEE 1588 PTP protocol. However, in the case of complex network topology or asymmetric delay, it is difficult to achieve microsecond-level synchronization accuracy of audio data frames by relying only on the PTP protocol, which may easily lead to audio frame misalignment or playback interruption. To solve this problem, the present scheme proposes a new timestamp synchronization mechanism that combines PTP protocol, UWB ranging and FPGA hardware acceleration, and implements dynamic delay compensation combined with an improved Kalman filter, finally achieving sub-microsecond time synchronization accuracy (<1 μs) to meet the high reliability requirements of professional audio broadcast systems.

[0066] The core of the present technical solution is to build a unified time synchronization mechanism, so that the slave node can dynamically adjust the current timestamp based on the historical timestamp, network delay, UWB ranging result and local clock drift trend, to keep the timestamp of the slave node consistent with that of the master node.

[0067] The specific method for time synchronization between the node and the master node comprises:

[0068] S01, initializing the time reference of the master node and the slave node: the master node runs a master clock and periodically broadcasts a first message to the slave node, the first message containing the master node timestamp; the slave node runs a slave clock and receives the first message; the slave node records the local timestamp and embeds it into the audio frame header; specifically, the first message can be an Announce message, and a high-precision FPGA module is pre-configured in the slave node, which is used to accurately record the local timestamp and embed it into the audio frame header;

[0069] ​S02, obtaining the round-trip delay and the physical distance between the master node and the slave node; specifically, the round-trip delay between the master node and the slave node is obtained according to the following method: the slave node sends a Delay Request packet to the master node, and the master node returns a Delay Response packet to the slave node after receiving the Delay Request packet; the slave node obtains the round-trip delay (Round-Trip Time, RTT) between the master node and the slave node by calculating the time difference between the Delay Request and the Delay Response; this delay reflects the average communication time consumption in the network transmission process between the master node and the slave node.

[0070] The physical distance between the master node and the slave node can be obtained according to the following method: in the case of supporting UWB wireless ranging deployment, UWB communication is started between the master node and the slave node, and the physical distance between the master node and the slave node is measured by UWB bidirectional ranging technology; the physical distance is converted into signal propagation delay according to the speed of electromagnetic wave propagation; the propagation delay is used as auxiliary information independent of network delay to further improve the timestamp synchronization accuracy of the master node and the slave node.

[0071] S03, based on the round-trip delay and the physical distance, the local timestamp of the slave node is updated by combining the improved Kalman filter and the state transition method; in the process, the master node timestamp is used as the observation value, the linear scaling of the local timestamp is realized based on the drift coefficient between the master node and the slave node, and the process noise covariance and the observation noise covariance are dynamically adjusted according to the observation index; the observation index includes: observation residual, round-trip delay jitter and packet loss rate;

[0072] Specifically, the following steps are included: a reasonable state estimation value is initially set for the improved Kalman filter: the initial local timestamp is taken from the local time recorded when the master node and the slave node first handshake; the initial drift rate is assumed to be zero, assuming that the master node and the slave node have the same frequency; the local timestamp and the drift rate of the slave node at the next time are predicted by combining the state transition method and the improved Kalman filter; this predicted value is the prior estimate, which represents the state prediction before new observation data is received.

[0073] The slave node continuously receives the timestamp information sent by the master node (such as from PTP Announce or Response message); these information are used as observation values, which are input into the improved Kalman filter, and the observation values reflect the deviation between the master node timestamp and the predicted value of the slave node; the improved Kalman filter automatically adjusts the state estimation value according to the observation error, and outputs the posterior estimate, i.e. the local timestamp and the drift rate closer to the true state.

[0074] The observation residual, network jitter level and packet loss rate are continuously monitored. If it is found that the current observation data is unstable (e.g., network packet loss is frequent and RTT fluctuates greatly), the observation noise covariance is appropriately increased. If the system is stable, the noise covariance is reduced to make the Kalman filter converge faster. At the same time, the process noise covariance is dynamically adjusted to adapt to the changes in the local clock drift, so that the system can maintain good timestamp estimation performance under different network conditions.

[0075] In each update, the drift coefficient between the master and slave nodes is calculated. The drift coefficient reflects the trend of the frequency difference between the master and slave nodes and is used for linear scaling of the timestamps of subsequent audio frames, thereby eliminating long-term frequency offset caused by crystal aging, temperature change or network delay. The FPGA module in the slave node updates the internal timestamp register in real time according to the estimated value output by the Kalman filter. When the next audio frame enters the FPGA module, it is immediately marked with a corrected timestamp. The FPGA module appends the corrected timestamp to the audio frame header or the RTP extension header and sends the data frame to the terminal. This mechanism guarantees the nanosecond-level precision of timestamp insertion and avoids the delay uncertainty introduced by the software layer.

[0076] To ensure the consistency of the audio frame timestamps, the embodiment adopts a unified format 64-bit timestamp, which has the structure of [32-bit seconds][32-bit nanosecond offset]. An example is 0x5A5A5A5A 0x12345678, which represents the time of January 1, 1516, 00:00:00.305175846. The timestamp has the following characteristics: (1) global uniqueness: each audio frame has a unique identifier in the entire system; (2) fixed format: convenient for parsing and comparison; (3) strong compatibility: supports mainstream standards such as IEEE 1588, AES67, SMPTE ST 2110, etc.; (4) scalability: more metadata can be supported by expanding the field.

[0077] The current timestamp of the slave node is determined by the last timestamp, network delay compensation and UWB ranging correction, and is updated by the drift coefficient The long-term clock offset is linearly corrected to maintain the continuous synchronization between the master and slave nodes. The overall process model of state update includes:

[0078]

[0079] wherein, Tk represents the local timestamp of the slave node corresponding to the kth update; Tk-1 represents the local timestamp of the slave node corresponding to the (k-1)th update; RTT(k) represents the round-trip delay between the master and slave nodes corresponding to the kth update; represents the signal propagation delay when the kth updated corresponding measurement physical distance; represents the drift coefficient between the master node and the slave node corresponding to the kth update; represents the system noise.

[0080] The method supports limited deployment and wireless deployment, and the physical distance is obtained based on a UWB ranging module;

[0081] In wired deployment, = 0; in wireless deployment, = d / c; where d represents the physical distance between the master node and the slave node; c represents the propagation speed of electromagnetic waves in air.

[0082] To realize the dynamic adjustment capability of the above formula, the scheme adopts an improved Kalman filter to perform state estimation on the local timestamp; assuming that there is a small frequency difference between the master and slave nodes, the following model can be established:

[0083]

[0084] wherein, represents the timestamp stamped by the slave node when the kth audio frame is encoded; represents the timestamp stamped by the master node when the kth audio frame is encoded; represents the time deviation, which is composed of an initial offset and an accumulated offset; the time deviation is modeled as follows:

[0085]

[0086] wherein, represents the frequency difference per unit time; t 0 represents the initial local timestamp recorded by the slave node; t k represents the local timestamp of the slave node corresponding to the kth update;

[0087] The initial offset The calculation method includes: the master and slave nodes perform first time synchronization through the PTP protocol; at this time, the master node sends an Announce message or a Delay Response message; the slave node records the local timestamp at this moment as , and records the timestamp corresponding to the master node as ; thereby obtaining the initial offset:

[0088] ;

[0089] The initial offset is denoted as The time difference offset model is obtained as follows:

[0090]

[0091] wherein, represents the master node timestamp corresponding to the kth update;

[0092] The drift coefficient between the kth update master node and the slave node local is

[0093]

[0094] The drift coefficient calculation formula between the master node and the slave node is simplified as follows: wherein, represents the clock frequency deviation (drift rate) of the kth update corresponding slave node relative to the master node; if > 0, it means that the slave node is faster than the master node; if < 0, it means that the slave node is slower than the master node; if = 0, it means that the frequencies are consistent.

[0095] The clock frequency deviation is not directly measured, but is obtained by an improved Kalman filter estimation, and the model is as follows: wherein, represents the state vector output by trend prediction at the kth update; wherein the state vector is defined as: and the initial value thereof is estimated as: wherein, represents the estimated initial state vector; represents the time stamp when the slave node first handshakes, which is actually impossible to know very accurately at the moment of handshake; : It is assumed that the initial frequency difference is zero, because there is no historical data to calculate, and it can only be estimated.

[0096] It is assumed that the change of the slave node local clock follows a simple physical law: the time stamp increases linearly with time, and the drift rate remains stable in the short term, but there is a slow change in the long term (crystal aging, temperature influence, etc.). Therefore, the state transition matrix F is used to describe how the system state evolves over time, which is a constant velocity model (CV Model), one of the most commonly used motion models in target tracking and navigation systems.

[0097]

[0098] ​​​Where Δt represents the time interval between two updates (e.g., once per second), the first row indicates that the timestamp increases linearly, and the second row indicates that the drift rate remains constant.

[0099] The physical meaning of the state transition matrix F is as follows:

[0100] The base portion of the timestamp indicates that the timestamp retains the value of the previous frame;

[0101] : The timestamp increment item, representing the value by which the timestamp increases over time;

[0102] : This indicates that the drift rate will not change abruptly;

[0103] This indicates that the local clock drift rate (clock frequency deviation) remains stable for a short period of time.

[0104] Assuming the drift rate remains constant during the prediction phase (nanosecond level) (i.e.) Then, the state at the previous time step can be predicted using the state transition matrix F, thus obtaining... That is, prior estimation:

[0105] ;

[0106] Assuming the timestamp of the previous frame seconds, drift rate That is, error per second Frame interval =0.002 seconds, or 2ms, so according to the above formula, the state of the next frame can be predicted as follows:

[0107]

[0108] We can obtain that the next frame timestamp is 1000.002000 seconds, while the drift rate remains at [value missing]. .

[0109] Specifically, methods for predicting clock drift rate include: receiving the master node timestamp as an observation. : ;in, This represents the signal propagation delay when measuring physical distance; This represents the master node timestamp corresponding to the k-th update;

[0110] The predicted state is mapped to the observation space using the observation matrix H = [1, 0] (only local timestamps are observed). And based on the Kalman gain coefficient Make dynamic adjustments accordingly. The posteriori estimation is obtained, and the clock drift rate is obtained The drift coefficient is obtained And further .

[0111]

[0112] The Kalman gain coefficient model is:

[0113]

[0114]

[0115] Wherein, The state estimation error covariance in the prediction stage is represented, and the initial value is set to a small constant × unit matrix, such as: ; T represents the transpose of the matrix; Q represents the process noise covariance, and the initial value is a small constant × unit matrix, which represents the clock disturbance error, and is dynamically adjusted subsequently; R represents the observation noise covariance (which represents the uncertainty of the measurement error), and the initial value is a small constant × unit matrix, which is generally the same as , increasing K k decreases, and the filter relies less on the current observation value, and vice versa, which will be automatically adjusted subsequently.

[0116] S04, updating the local timestamp of the slave node based on the state update result.

[0117] The process noise covariance Q and the observation noise covariance R in the scheme are not fixedly set, but are adjusted online according to the network jitter level (round-trip delay jitter), the packet loss rate, the UWB signal strength (if enabled), the current observation residual , the system running state of the current observation residual, the network jitter, the packet loss rate and the like is collected in real time, if the above indexes exceed the preset threshold, Q and R are increased, the trust in the model prediction and the observation value is reduced, and vice versa. Thus, the improved Kalman filter can automatically adapt to the timestamp drift between the master and the slave node, the network jitter and the observation uncertainty, so as to maintain sub-microsecond synchronization precision in various deployment environments. Through this dynamic noise covariance adjustment mechanism based on the system running state, the robustness of the filter in the complex network environment is effectively improved. Thus, the scheme makes up for the frame-level timestamp synchronization mechanism not provided by the existing AoIP standards (such as AES67 and Dante), and the time synchronization precision reaches sub-microsecond level (<1 μs), which meets the high reliability requirement of the professional audio broadcast system.

[0118] ​​​Under the premise of keeping the slave node time synchronization with the master node, step one is performed: real-time collection of master node playing state update information, running information and master node fault information; specifically, the master node runs an audio processing service, and periodically sends a playing state update message, which includes: current frame ID (current frame unique identifier), timestamp (current frame timestamp), playing pointer position (such as seconds and frames) and audio processing parameters (volume, EQ, routing, etc.); the data packet of the playing state update message of the embodiment is shown in the following example, including current frame ID (current frame unique identifier), current frame timestamp, playing pointer position (corresponding to each sampling point), audio processing parameters, buffer state and switching instruction fields.

[0119] {

[0120] "protocol_version": "1.0", / / protocol version number, ensure compatibility

[0121] "timestamp": { / / current frame timestamp (seconds + nanoseconds)

[0122] "seconds": 1620000000,

[0123] "nanoseconds": 123456789

[0124] },

[0125] "frame_id": "FRAME_001234", / / current frame unique identifier

[0126] "play_position": 12345, / / current playing position (unit: sampling point)

[0127] "audio_parameters": { / / audio processing parameters

[0128] "volume": 0.85,

[0129] "eq": [ -2.0, 0.0, +1.5,... ],

[0130] "compressor_threshold": -6.0,

[0131] "limiter_ceiling": 0.0,

[0132] "input_routing": "INPUT_1",

[0133] "output_routing": "OUTPUT_A

[0134] },

[0135] "buffer_status": { / / Buffer status

[0136] "available_frames": 5,

[0137] "next_frame_id": "FRAME_001235"

[0138] },

[0139] "switch_command": { / / Switch command field

[0140] "initiate_switch": false,

[0141] "target_slave": "SLAVE_02"

[0142] }

[0143] }

[0144] Step 2: Parse the playback status update information of the master node to obtain parsed information, and update the local playback status cache of the slave node according to the parsed information. At the same time, the AI ​​monitoring model evaluates the status of the master node based on the parsed information and the running information.

[0145] In this step, updating the local playback status cache of the slave node according to the parsed information includes the following method: the master node encapsulates the playback status update information into ALSP status information, sends it to the slave node and the client through a dedicated channel or RTP extension header, and inserts the master node timestamp into the audio frame to form an audio stream to be sent to the slave node;

[0146] The slave node listens to and parses ALSP status information and audio stream in real time, and parses the frame ID of the current playback frame of the master node, the master node timestamp, the playback pointer position and audio processing parameters based on the ALSP status information;

[0147] The slave node updates the playback pointer position and audio processing parameters to the local playback status buffer, and sorts the audio frames in the audio stream according to the timestamp order of the master node, updating the audio frames to the local playback content buffer; the local playback content buffer maintains at least N audio frames. If a discontinuous frame ID or a timestamp deviation exceeds the limit is found, a reload of the audio stream is triggered; this ensures that the slave node has the next frame of audio content when switching, achieving seamless transition.

[0148] Step three: trigger the switching instruction when the master node failure information is collected or the AI monitoring model evaluates that the master node state is abnormal; in this step, the AI monitoring model includes a trained LSTM prediction model or a Transformer model; the running information includes: CPU usage trend, memory occupation trend, network bandwidth, network quality, heartbeat signal and master node timestamp continuity.

[0149] The AI monitoring model gives a health score to the master node, and if the score is lower than the set threshold, a switching instruction is generated; the switching instruction is broadcast to all slave nodes and clients through the ALSP protocol; the AI monitoring model actively predicts the switching opportunity to avoid passive waiting for the master node to go down.

[0150] Step four: execute the switching instruction based on the application layer switching protocol; this step specifically includes the following methods:

[0151] For the switching instruction triggered by the AI monitoring model evaluating that the master node state is abnormal, directly switch to the slave node based on the application layer switching protocol;

[0152] For the switching instruction triggered by the master node failure information, (at this time, the master node suddenly goes down, and the slave node passively enters the switching process, usually without completing the last frame reception before disconnecting from the master node, that is, the slave node receives the current frame FRAME_X, but has not received the next frame FRAME_{X+1} before going down, at this time, the slave node has only the data of the current frame FRAME_X in the first-level cache, and the data of the next frame FRAME_{X+1} and subsequent frames has not arrived, then it is divided into two cases:

[0153] Determine whether the slave node has a local audio source file corresponding to the audio stream, if yes, use the double-layer cache switching strategy (such as in the scenario of real-time music broadcasting and concert relay); otherwise, use the PLC filling cache switching strategy (such as in the scenario of real-time voice program, network live broadcast and real-time broadcast without manual intervention);

[0154] The double-layer cache switching strategy includes: determining the playback starting point of the slave node based on the drift coefficient between the master node and the slave node; the specific playback starting point is: ; Wherein, △t represents the interval from the time when the slave node discovers the master node disconnection to the current time; represents the estimated value of the local timestamp of the slave node when the master node is disconnected, which comes from the prediction of the above Kalman filter; then find the audio frame i corresponding to the playback starting point timestamp from the local audio source file, load the subsequent N audio frames of audio frame i into the second layer cache area, the first layer cache area reads the audio frame from the second layer cache area, and the slave node labels the local timestamp of the audio frame read by the first layer cache area and sends it out; the client plays to realize seamless switching and listeners have no perception.

[0155] The PLC filling cache switching strategy includes: extracting the audio stream segment of the current timestamp context of the master node, inputting an AI prediction model to generate a filling audio frame loaded to a first layer cache area, and issuing by the slave node after labeling the local timestamp of the audio frame in the first layer cache area. The AI prediction model adopts LSTM autoregressive prediction: x fill =f LST M (x K-M , …, x K ); wherein x K-M represents audio data of the Mth sampling frame (such as 10 ms), f LSTM represents a trained local waveform prediction function; x fill represents the audio frame content filled by the filling model output; K represents the total number of sampled frames from x K-M sampling time to the current frame sampling time; and x k represents the current frame content.

[0156] The specific process is that the slave node receives the "switching instruction" sent by the ALSP protocol, enters the switching preparation stage according to the instruction, queries whether the local buffer contains the next frame of audio, the slave node checks whether the next frame of audio of the master node has been cached in the local buffer, if so, the frame is directly enabled for continuous playing, if not (such as sudden failure of the master node), the PLC filling cache switching strategy fills the missing frame; finally, the inherited playing state is enabled to continue outputting, the slave node inherits the playing position, volume, EQ, routing and other parameters of the master node, so as to ensure the consistency of the audio style before and after switching and avoid the listening feeling interruption caused by parameter mutation.

[0157] The client receives the "switching instruction" sent by the ALSP protocol, immediately queries the slave node audio stream state, queries the timestamp of the head frame of the slave node buffer, the client acquires the timestamp of the current playing frame of the slave node, judges whether it is continuous with the last frame timestamp of the master node, if so, immediately switches to the slave node, if not, the PLC filling cache switching strategy; after switching, the client continuously monitors the playing state of the slave node, after switching is completed, the client continuously monitors the playing state of the slave node, if the master node recovers subsequently, the AI module can decide whether to switch back, and the switching back process is also coordinated through the ALSP protocol.

[0158] The slave node regularly feeds back the playing state to the master node, in the duplex deployment scene, the slave node also feeds back the playing state of itself to the master node, for the master node to monitor whether the slave node keeps synchronization; the master node dynamically adjusts the audio stream to adapt to the state of the slave node, if the slave node appears delay or deviation, the master node can inform it to reload through the ALSP protocol, or insert redundant frames in advance to help the slave node to catch up with the progress, form a closed loop control, and improve the overall stability of the system.

[0159] The ALSP protocol solves the key problems of uncertain switching timing and misaligned playback states in the existing AoIP standard. Through ALSP's frame-level state synchronization and playback pointer coordination mechanism, it achieves truly seamless switching. It is not only suitable for hardware device switching scenarios, but also for new audio broadcasting architectures such as software service migration and virtualization deployment.

[0160] During the switchover between master and slave nodes in professional audio broadcasting, audio frame loss or interruption can easily occur in the event of a sudden master node failure or network outage. Therefore, ensuring playback continuity is crucial.

[0161] Example 2: This example provides a media broadcast master-slave node switching system to implement the media broadcast master-slave node switching method described in Example 1; as follows: Figure 2 As shown, the system includes:

[0162] The synchronization module is used to keep the time of the slave node synchronized with that of the master node;

[0163] The acquisition module is used to collect real-time playback status updates, operation information, and fault information of the main node;

[0164] The parsing and evaluation module is used to parse the playback status update information of the master node to obtain parsing information, and update the local playback status cache of the slave node according to the parsing information. At the same time, the AI ​​monitoring model evaluates the status of the master node based on the parsing information and the running information.

[0165] The trigger module is used to trigger a switchover command when the master node fault information is collected or the AI ​​monitoring model evaluates that the master node status is abnormal.

[0166] The execution module is used to execute the switching instructions based on the application layer switching protocol.

[0167] Specifically, this system includes: Master node: the main audio processing unit responsible for generating and broadcasting audio streams, running the master clock service, and responsible for audio stream distribution; Slave node: a backup unit that mirrors the operation of the master node, can take over at any time, synchronizes the master node's timestamp in real time, and caches the same audio data; Encoder: processes the encoding of audio data, encodes the audio signal into a unified format, adds digital fingerprint information to the data frame, and timestamps it; The parsing and evaluation module includes an AI monitoring model unit, used to monitor the node status in real time, predict faults, and trigger switching;

[0168] The client receives and plays the audio stream, supports passive trigger switching and autonomous active switching; the acquisition module includes a UWB ranging module: provides accurate distance measurement to enhance time synchronization, assists the master and slave nodes in physical distance measurement and time compensation; an application layer synchronization engine: manages the synchronization of audio content and parameters; a state parameter synchronization module: ensures that all related parameters are synchronized between the master and slave nodes. The system adopts a dual-active hot backup architecture, and the master and slave nodes continuously process the audio stream and maintain synchronization status; UWB (Ultra-Wideband) is an ultra-wideband wireless communication technology.

[0169] Embodiment 3: The embodiment provides a computer readable medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the media broadcast master-slave node switching method according to embodiment 1; the following steps are specifically implemented:

[0170] Under the premise of maintaining time synchronization between the slave node and the master node:

[0171] Step one: real-time acquisition of master node playing state update information, running information and master node fault information;

[0172] Step two: parsing the master node playing state update information to obtain parsed information, and updating the local playing state cache of the slave node according to the parsed information, while the AI monitoring model performs state evaluation on the master node based on the parsed information and the running information;

[0173] Step three: when the master node fault information is collected or the AI monitoring model evaluates that the master node state is abnormal, trigger the switching instruction;

[0174] Step four: execute the switching instruction based on the application layer switching protocol.

[0175] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for switching master and slave nodes in media broadcasting, characterized in that, include: While maintaining time synchronization between the slave node and the master node: Real-time collection of playback status updates, operational information, and fault information from the main node; The master node playback status update information is parsed to obtain parsed information, and the local playback status cache of the slave node is updated according to the parsed information. At the same time, the AI ​​monitoring model evaluates the status of the master node based on the parsed information and the running information. When a master node fault information is collected or the AI ​​monitoring model determines that the master node is in an abnormal state, a switchover command is triggered. The handover command is executed based on the application layer handover protocol; the method includes: For switching commands triggered by abnormal master node status as assessed by the AI ​​monitoring model, the node is directly switched to a slave node based on the application layer switching protocol. For switching instructions triggered by master node failure information, determine whether the slave node has a local audio source file corresponding to the audio stream. If so, enable the dual-layer buffer switching strategy; otherwise, enable the PLC buffer filling switching strategy. The dual-layer buffer switching strategy includes: determining the playback start point of the slave node based on the drift coefficient between the master node and the slave node; finding the audio frame i corresponding to the playback start point timestamp from the local audio source file; loading the N audio frames following audio frame i into the second-layer buffer; the first-layer buffer reading audio frames from the second-layer buffer; and having the slave node annotate the audio frames read from the first-layer buffer with the local timestamp before sending them out. The PLC filling buffer switching strategy includes: extracting the audio stream segment of the current timestamp context of the master node, inputting it into the AI ​​prediction model to generate filling audio frames and loading them into the first-level buffer, and having the slave node mark the audio frames in the first-level buffer with local timestamps before sending them out.

2. The media broadcast master-slave node switching method according to claim 1, characterized in that, The method for synchronizing time between the slave node and the master node includes: Initialize the time base of the master node and slave nodes: The master node runs the master clock and periodically broadcasts a first message to the slave nodes, the first message containing the master node's timestamp; the slave nodes run the slave clock and receive the first message; the slave nodes record their local timestamps and embed them into the audio frame header; Obtain the round-trip delay and physical distance between the master node and the slave node; Based on round-trip delay and physical distance, an improved Kalman filter and state transition method are used to update the state of the local timestamp of the slave node. During the process, the master node timestamp is used as the observation value, and the local timestamp is linearly scaled based on the drift coefficient between the master node and the slave node. The process noise covariance and observation noise covariance are dynamically adjusted according to the observation indicators. The observation indicators include: observation residual, round-trip delay jitter and packet loss rate. Update the local timestamp of the slave node based on the state update result.

3. The media broadcast master-slave node switching method according to claim 2, characterized in that, The overall process model for state updates includes: ； in, This represents the local timestamp of the slave node corresponding to the k-th update; This indicates the local timestamp of the slave node corresponding to the (k-1)th update; This represents the round-trip delay between the master node and the slave node corresponding to the k-th update; This represents the signal propagation delay when measuring physical distance; This represents the drift coefficient between the master node and the slave node corresponding to the k-th update; This indicates system noise.

4. The media broadcast master-slave node switching method according to claim 3, characterized in that, The method supports both limited and wireless deployments, and the physical distance is obtained based on a UWB ranging module. In wired deployments, =0; In wireless deployments, =d / c; where d represents the physical distance between the master node and the slave node; and c represents the speed of electromagnetic wave propagation in the air.

5. A method for switching master and slave nodes in media broadcasting according to claim 2 or 3, characterized in that, The drift coefficient between the master node and the slave node is calculated according to the following formula: ； in, This represents the drift coefficient between the master node and the slave node during the k-th update. This represents the initial local timestamp recorded from the node; This represents the local timestamp of the corresponding slave node during the k-th update; This represents the clock frequency deviation of the slave node relative to the master node corresponding to the k-th update; This represents the local timestamp of the slave node obtained after the k-th update; This represents the initial offset between the master node's timestamp and the slave node's initial local timestamp.

6. The media broadcast master-slave node switching method according to claim 1, characterized in that, The method for updating the local playback state cache of the slave node based on the parsed information includes: The master node encapsulates the playback status update information into ALSP status information and sends it to the slave nodes and the client. It also inserts the master node's timestamp into the audio frame to form an audio stream and sends it to the slave nodes. The slave node listens to and parses ALSP status information and audio stream in real time, and parses the frame ID of the current playback frame of the master node, the master node timestamp, the playback pointer position and audio processing parameters based on the ALSP status information; The slave node updates the playback pointer position and audio processing parameters to the local playback status buffer, and sorts the audio frames in the audio stream according to the master node's timestamp order, updating the audio frames to the local playback content buffer; the local playback content buffer maintains at least N audio frames.

7. A method for switching master and slave nodes in media broadcasting according to claim 1, characterized in that, The AI ​​monitoring model includes a pre-trained LSTM prediction model or a Transformer model; the operational information includes: CPU utilization trend, memory usage trend, network bandwidth, network quality, heartbeat signal, and master node timestamp continuity.

8. A media broadcast master-slave node switching system, characterized in that, A system for implementing a media broadcast master-slave node switching method according to any one of claims 1-7; the system includes: The synchronization module is used to keep the time of the slave node synchronized with that of the master node; The acquisition module is used to collect real-time playback status updates, operation information, and fault information of the main node; The parsing and evaluation module is used to parse the playback status update information of the master node to obtain parsing information, and update the local playback status cache of the slave node according to the parsing information. At the same time, the AI ​​monitoring model evaluates the status of the master node based on the parsing information and the running information. The trigger module is used to trigger a switchover command when the master node fault information is collected or the AI ​​monitoring model evaluates that the master node status is abnormal. The execution module is used to execute the switching instructions based on the application layer switching protocol.

9. A computer-readable medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, can implement a media broadcast master-slave node switching method as described in any one of claims 1-7.

Images