Method, system and storage medium for synchronizing multimedia presentation content

Abstract

The application relates to the technical field of multimedia communication, and discloses a multimedia display content synchronization method and system and a storage medium. The method comprises the following steps: collecting a play history record and real-time state data of a target terminal device, and generating a behavior feature vector representing a user operation mode based on time sequence feature extraction technology analysis; using the vector, calculating a triggering probability of each play control instruction, and based on a network state evaluation process, obtaining a current link quality evaluation index value to optimize the sending time of the play control instruction; through a load balancing strategy, combining bandwidth utilization and network jitter feedback results, determining an optimal transmission time window and an instruction pre-dispatching time; sending an emergency control instruction through a multi-path transmission channel, and adjusting a play progress jump prediction according to execution state feedback data, and finally ensuring the segmented time synchronization of multimedia content through a clock synchronization protocol. The application improves the stability and response speed of multimedia content playing.

Description

Technical Field

[0001] This application relates to the field of multimedia communication technology, and in particular to a method, system and storage medium for synchronizing multimedia display content. Background Technology

[0002] With the widespread adoption of multimedia applications, especially in scenarios such as video streaming, online education, and live broadcasting, users have increasingly higher demands for the immediacy and smoothness of content. However, in practical applications, factors such as unstable network bandwidth, differences in device performance, and network congestion often lead to desynchronization of multimedia content across multiple terminal devices, affecting the user's viewing experience. Particularly in environments with significant network fluctuations, the playback progress may not be synchronized with the actual content, causing stuttering and skipping during viewing, severely impacting the user's continuous experience and satisfaction.

[0003] In existing technologies, multimedia playback synchronization mainly relies on static network bandwidth allocation and limited device resource scheduling, making it difficult to adapt to complex and dynamically changing network environments. Traditional synchronization methods typically ignore real-time changes in user operation modes and dynamic assessments of network status, resulting in delays and untimely playback commands, thus failing to meet the real-time playback control requirements of multiple terminal devices.

[0004] With technological advancements, real-time video streaming, network jitter monitoring, and load balancing algorithms have been increasingly applied to multimedia synchronization in recent years. However, most methods rely on preset bandwidth and fixed synchronization mechanisms, lacking the ability to respond in real-time to changes in user behavior. This is especially true in complex network environments, where it is difficult to dynamically adjust playback control strategies. Therefore, how to intelligently optimize the timing of command transmission based on user behavior, network status, and real-time feedback data to ensure the synchronization of multimedia content has become a significant technological challenge. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a method, system, and storage medium for synchronizing multimedia display content, which improves the stability and response speed of multimedia content playback.

[0006] Firstly, this application provides a method for synchronizing multimedia display content, the method comprising:

[0007] Step S1: Collect the playback history and real-time status data of the target terminal device, and analyze them based on time series feature extraction technology to obtain behavioral feature vectors that represent user operation patterns;

[0008] Step S2: Based on the behavior feature vector, calculate the instruction trigger probability value of each playback control instruction within the future time window; based on the instruction trigger probability value, determine the high-probability playback control instruction sequence; and start the network status assessment process to obtain the link quality assessment index value of the current network environment.

[0009] Step S3: Process the link quality evaluation index values ​​through the load balancing allocation strategy, and combine the bandwidth utilization calculation results and network jitter monitoring feedback results to determine the optimal transmission time window and instruction pre-dispatch time.

[0010] Step S4: Based on the constraints of the optimal transmission time window, adjust the sending priority of the high-probability playback control instruction sequence, generate an instruction dispatch queue, extract emergency playback control instructions, send them to the target terminal device through a pre-established multi-path transmission channel, and obtain execution status feedback data.

[0011] Step S5: Based on the execution status feedback data, adjust the playback progress jump prediction and update the behavior feature vector. When the updated behavior feature vector indicates that there is a content segmentation transmission control requirement, activate the inter-device clock synchronization protocol and synchronize the multimedia content segments in time through the clock synchronization protocol.

[0012] Secondly, this application provides a multimedia content synchronization system, the system comprising:

[0013] The feature vector acquisition unit is used to collect the playback history and real-time status data of the target terminal device, and analyze them based on time series feature extraction technology to obtain behavioral feature vectors that characterize the user's operation mode.

[0014] The link quality assessment unit is used to calculate the instruction trigger probability value of each playback control instruction within a future time window based on the behavior feature vector, determine the high-probability playback control instruction sequence based on the instruction trigger probability value, and start the network status assessment process to obtain the link quality assessment index value of the current network environment.

[0015] The optimal transmission determination unit is used to process the link quality evaluation index values ​​through a load balancing allocation strategy, and combine the bandwidth utilization calculation results and network jitter monitoring feedback results to determine the optimal transmission time window and instruction pre-dispatch time.

[0016] The execution status feedback unit is used to adjust the sending priority of the high-probability playback control instruction sequence based on the constraints of the optimal transmission time window, generate an instruction dispatch queue, extract emergency playback control instructions, send them to the target terminal device through a pre-established multi-path transmission channel, and obtain execution status feedback data.

[0017] The content time synchronization unit is used to adjust the playback progress jump prediction based on the execution status feedback data and update the behavior feature vector. When the updated behavior feature vector indicates that there is a content segmentation transmission control requirement, the inter-device clock synchronization protocol is activated, and the multimedia content segments are time-synchronized through the clock synchronization protocol.

[0018] A third aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the aforementioned method for synchronizing multimedia display content.

[0019] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0020] 1. The technical solution provided in this application effectively reduces playback asynchrony problems caused by network fluctuations or delays by adjusting the timing, priority and path selection of instruction transmission in real time, thus ensuring the consistency of multimedia content across multiple terminal devices.

[0021] 2. By introducing network status assessment and load balancing mechanisms, the command sending strategy can be dynamically optimized according to different network environments, especially adapting to low bandwidth or high latency network environments, ensuring that the system can operate stably under various network conditions.

[0022] 3. By adjusting playback progress jump prediction and using a clock synchronization protocol to synchronize the device clock, the continuity of users watching multimedia content is improved, interruptions caused by instruction delays or synchronization problems are avoided, and the user's viewing experience is optimized.

[0023] 4. Employing a multi-path transmission mechanism ensures that emergency control commands can still be reliably transmitted even under network congestion or poor link quality, thereby improving overall transmission efficiency and the consistency of multimedia display content. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of an embodiment of the method for synchronizing multimedia display content in this application.

[0026] Figure 2 This is a flowchart illustrating the multipath transmission process for emergency commands in an embodiment of this application.

[0027] Figure 3 This is a schematic diagram of one embodiment of the multimedia display content synchronization system in this application. Detailed Implementation

[0028] This application provides a method, system, and storage medium for synchronizing multimedia display content. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Example 1:

[0030] This application provides a method for synchronizing multimedia display content, primarily aimed at addressing the poor user experience caused by instruction response delays and unstable network transmission in traditional playback control systems. In particular, existing systems lack analysis and prediction of the timing characteristics of user operations and fail to effectively optimize instruction sending timing based on network conditions, thus hindering more efficient and accurate multimedia content synchronization. The technical solution provided in this application can predict user actions in advance, rationally schedule instruction sending, and optimize various operation responses during multimedia playback.

[0031] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the method for synchronizing multimedia display content in this application includes:

[0032] Step S1: Collect the playback history and real-time status data of the target terminal device, and analyze them based on time series feature extraction technology to obtain behavioral feature vectors that represent user operation patterns.

[0033] The above step S1 further includes: acquiring playback history data and real-time status data from the target terminal device, and organizing them into time series data according to timestamps. The playback history data includes historical playback progress jump records and pause / resume timing information, while the real-time status data includes the current playback progress, playback speed, volume level, and play / pause operation time points. Time series feature extraction technology is used to extract time series features from the playback history data and real-time status data to identify user operation pattern patterns. Based on the user operation pattern patterns, a behavioral feature vector is generated, which at least includes playback progress jump prediction, pause / resume timing judgment, volume adjustment frequency analysis results, and playback speed change trend data.

[0034] Specifically, playback history data is obtained from the terminal device, including at least past playback progress jump records and pause / resume timing information, reflecting how the user jumps to specific time points or pauses and resumes playback during viewing. Secondly, real-time device behavior data is obtained by monitoring the device's current status data, such as current playback progress, playback speed, volume level, and the time points of playback or pause operations. This data is then sorted by timestamps to form an original, chronologically ordered sequence of operation events, providing a foundation for the next step of behavior feature extraction. Time series feature extraction technology is then used to process the collected time series data; the key to this processing flow lies in identifying… By analyzing user operation patterns and extracting their behavioral characteristics, this process involves organizing playback history and real-time status data into time series data, and then applying an autoregressive moving average model to process this time series. The training data for the autoregressive moving average model includes user operation patterns extracted from the time series data, playback history, and real-time status data, used to identify and model periodic features in user behavior. By calculating the autocorrelation function of the sequence, periodic features are extracted to identify the user's tendency to pause during content highlights. For example, in video streaming applications, the model can reveal users' skipping habits during ad breaks and predict their next possible action.

[0035] Regarding the processing of collected time series data using time series feature extraction technology, the key to the process lies in identifying user operation patterns and extracting behavioral features. This includes: preprocessing and feature engineering the aforementioned operation event sequences to prepare input for the time series model. Preprocessing includes data cleaning, missing value handling, and data normalization; preprocessing is an existing technology and will not be elaborated here. Subsequently, the preprocessed data is organized into a multivariate time series according to a preset time granularity, such as one data point per second. Then, for the preprocessed multivariate time series, models are built for sequences such as "jump events" and "pause events." Taking the "jump event sequence" as an example, the value at each time point is either "1" (indicating a jump occurred) or "0" (indicating no jump occurred). The autoregressive integrated moving average model identifies user operation patterns through the following steps: model order determination; by calculating the autocorrelation function and partial autocorrelation function of the sequence, the parameter range of the ARIMA model is initially determined; then, the Akaike information criterion is used for further analysis. Precise search automatically determines the optimal parameter combination based on minimizing the AIC value. Model training and fitting include training the ARIMA model using the determined optimal parameters and the maximum likelihood estimation method to fit the jump event sequence. After training, the model can capture potential periodic fluctuations and trend changes in the sequence. For example, the model may reveal a potential jump peak every 5 minutes after the video starts, or a significant increase in the probability of jumping 30 seconds after the start of an advertisement. Feature quantification and pattern recognition include extracting key features from the trained ARIMA model to quantify the operational pattern, including periodic intensity, which is calculated by spectral analysis of the model residual sequence to determine the energy proportion at a specific frequency, such as the corresponding 5-minute period, as an indicator of the strength of the periodic pattern; trend slope, which is extracted from the deterministic trend term of the model to determine whether user jump behavior tends to be more frequent or less frequent; and autoregressive coefficients, which directly reflect the dependence of historical operations on current operations, i.e., the inertia or anti-inertia pattern of operations. Finally, based on the extracted quantitative features, a behavioral feature vector is generated. This behavioral feature vector is a multi-dimensional numerical vector, whose dimensions include at least: the jump probability value predicted by the ARIMA model in the next few seconds as the playback progress jump prediction; the confidence of the pause and resume timing calculated by the statistical model based on the historical pause duration and resume interval; the main adjustment frequency obtained by performing a Fourier transform on the volume operation sequence as the volume adjustment frequency analysis result; and the slope of the change trend obtained by fitting the recent playback speed sequence through linear regression as the playback speed change trend data. This series of quantitative indicators derived from the specific time series model output constitutes a comprehensive and computable behavioral feature vector, providing a data foundation for subsequent instruction prediction.

[0036] Based on the identified user operation patterns, a behavioral feature vector representing these patterns is generated. This vector contains features across multiple dimensions. First, it predicts playback progress jumps, indicating when the user will jump to other parts of the video in the future. Second, it determines pause / resume timing, identifying under what circumstances the user pauses or resumes playback. Furthermore, the behavioral feature vector incorporates volume adjustment frequency and playback speed change trends, reflecting user preferences during viewing. In particular, frequent volume adjustments may indicate a response to the difficulty of the content being explained, or changes in playback speed may show interest in or dissatisfaction with the content. By integrating these features into the behavioral feature vector, this application can comprehensively and accurately predict future user behavior and provide a reliable basis for subsequent playback control command assignment.

[0037] This application further optimizes the accuracy of pattern recognition by comparing behavioral feature vectors with historical data. It calculates the similarity between feature vectors and historical vectors; if the similarity is below a preset threshold, the dimensional weights of the vectors are adjusted to make the prediction of current user behavior more accurate. Furthermore, the method uses clustering methods when necessary to refine the behavioral feature vectors, ensuring that the final generated behavioral feature vectors more accurately reflect the user's current operating habits and future tendencies. These optimization measures help improve the method's sensitivity to changes in user behavior, thereby avoiding misjudgments and reducing unnecessary command triggering. Through the above technical solutions, this application can effectively improve the accuracy and response speed of playback control commands, especially during the synchronous display of multimedia content. By combining network status and user behavior prediction, it ensures that commands are promptly assigned according to the user's actual needs, improving user experience and enhancing the overall performance of the multimedia playback system.

[0038] Step S2: Based on the behavioral feature vector, calculate the instruction trigger probability value of each playback control instruction within the future time window. Based on the instruction trigger probability value, determine the high-probability playback control instruction sequence and start the network status assessment process to obtain the link quality assessment index value of the current network environment.

[0039] The process of determining the high-probability playback control instruction sequence includes: extracting volume adjustment frequency analysis results and playback speed change trend data from behavioral feature vectors, and using a long short-term memory network to process the volume adjustment frequency analysis results and playback speed change trend data to calculate the trigger probability value of each instruction within a future time window; sorting potential instructions based on the instruction trigger probability values, and adding instructions whose trigger probability values ​​exceed a preset probability threshold to the high-probability playback control instruction sequence, which includes pause, jump, playback speed adjustment, and volume adjustment instructions.

[0040] Specifically, in order to solve the timing delay problem in traditional playback control systems, especially in multi-terminal and multi-network environments, how to accurately predict user behavior and achieve real-time synchronous control, existing technologies usually have difficulty handling the complex changes in user behavior and network status, resulting in synchronization deviations between playback content and user operations. This application can predict the user's operation mode in advance and combine it with real-time network conditions to dynamically optimize instruction dispatch, improve the consistency of content display, reduce latency and stuttering, and ultimately optimize the user experience.

[0041] Specifically, a Long Short-Term Memory (LSTM) network is used to process the trends in volume adjustment frequency and playback speed using time-series data. LSTM is a recurrent neural network capable of capturing long-term dependencies, suitable for analyzing sequence data, and able to handle temporal features in playback history and real-time status data. More specifically, the LSTM network uses forget gates, input gates, and output gates to progressively filter and update data, allowing the network to learn patterns in user operations and accurately calculate the trigger probability of each playback control command. These trigger probabilities are then used as the basis for command sorting. After calculating the command trigger probabilities, potential playback control commands, such as pause, skip, playback speed adjustment, and volume adjustment, are sorted according to these probabilities. Commands with trigger probabilities exceeding a preset probability threshold are added to the high-probability playback control command sequence, forming the final command sequence. This process ensures that the most likely commands are processed first, improving the system's response speed and accuracy.

[0042] The specific method for calculating the command trigger probability is as follows: First, the volume adjustment frequency analysis results and playback speed change trend data extracted from the behavioral feature vector are used as inputs to the Long Short-Term Memory (LSTM) network model. The volume adjustment frequency is quantized as a sequence of adjustment times per unit time, and the playback speed change trend is represented as a series of speed change rates within a continuous time window. These time-series data, after normalization, are organized into a time step sequence of fixed length, forming the input tensor of the LTM network. Next, a LTM network with a clear hierarchical structure is constructed, consisting of an input layer, two stacked LTM hidden layers, and a fully connected output layer. The LTM network utilizes its gating mechanism, including a forget gate to determine how much past state is retained, an input gate to control the inflow of new information, and an output gate to adjust... The network outputs the current state to handle long-term dependencies in the sequence data. Then, it performs supervised training on a prepared historical user action dataset, stopping early on the validation set to prevent overtraining. During the forward propagation of the Long Short-Term Memory (LSTM) network, the hidden state at each time step is passed to the next. The final hidden state is processed and mapped to the trigger probability values ​​of various playback control commands within the future time window, including pause, skip, playback speed adjustment, and volume adjustment commands. These probability values ​​form the basis for command prediction. Finally, the calculated command trigger probability values ​​are sorted, and commands exceeding a preset probability threshold are marked as high-probability commands and included in the priority processing sequence. This achieves accurate prediction of potential user actions, providing data support for subsequent network state evaluation and command pre-dispatch.

[0043] Furthermore, by combining real-time data to verify the accuracy of command predictions, the command sequence can be further adjusted. Real-time adjustments to the model using verification feedback ensure accurate responses even in dynamic network environments and under varying user operations. This technical solution not only optimizes the timing of command transmission but also dynamically adjusts command priority based on network quality, thereby guaranteeing the synchronization of multimedia content under different network conditions. The implementation of this technical solution effectively solves the synchronization problem in traditional playback control systems caused by the inability to accurately predict user behavior and network status changes. Especially in network environments with high latency and large bandwidth fluctuations, it ensures efficient synchronization of content display across multiple terminal devices, enhancing the user experience when watching multimedia content such as videos, live streams, and online education.

[0044] The network status assessment process includes: obtaining data packet loss rate statistics by monitoring the data packet transmission process, activating the transmission delay measurement method, and calculating the link delay by sending test data packets; weighting and fusing the data packet loss rate and link delay to obtain the link quality assessment index value; and triggering the load balancing optimization process when the link quality assessment index value is lower than the preset quality threshold.

[0045] Specifically, in order to solve the problems of untimely command transmission and asynchronous playback content in existing multimedia playback systems when faced with network fluctuations or delays, the existing playback control system usually cannot dynamically adjust playback commands according to real-time network conditions, resulting in delays, stuttering, or content asynchrony during playback, which affects user experience; this application achieves accurate synchronization of displayed content by monitoring network status in real time and optimizing command transmission strategies, thereby improving the stability of playback content and user experience.

[0046] Specifically, in this embodiment, the network status assessment process mainly includes two parts. First, by monitoring the data packet transmission process, data packet loss rate statistics are obtained. The data packet loss rate reflects network quality and indicates the frequency of data packet loss during network transmission, helping to determine the stability and reliability of the current network. Second, a transmission delay measurement method is activated to calculate link delay. Link delay affects playback smoothness, especially in real-time video or audio playback, where high link delay may lead to content desynchronization or playback interruption. Therefore, testing the round-trip time of data packets is a fundamental method for calculating link delay, accurately reflecting the network's latency performance. The data packet loss rate and link delay data are weighted and fused to generate a link quality assessment index value. This index value comprehensively considers both packet loss and latency performance, providing a more comprehensive assessment of the current network environment's quality. In this embodiment, the weighted fusion of the link quality assessment index value ensures a comprehensive evaluation of network quality. For example, in high-latency scenarios, the impact of delay may be significant. Therefore, the assessment results can be optimized by adjusting the weights, and subsequent instruction sending can be optimized accordingly. If the link quality assessment index value is lower than a preset quality threshold, the load balancing optimization process is immediately initiated.

[0047] The load balancing optimization process primarily improves data transmission efficiency by adjusting network traffic distribution, ensuring that urgent commands can be transmitted to the target device in a timely manner, as detailed below. This application, through real-time link quality assessment and load balancing optimization, not only effectively improves the accuracy of network quality assessment but also dynamically adjusts the timing and path of playback command transmission, ensuring efficient and stable synchronization of playback content across various network environments. It is particularly suitable for multimedia content display fields such as real-time video and audio playback and online education, optimizing user experience, reducing unnecessary command triggering and playback latency, and improving system response speed and stability.

[0048] Step S3: Process the link quality assessment index values ​​through load balancing allocation strategy, combine the bandwidth utilization calculation results and network jitter monitoring feedback results, and determine the optimal transmission time window and instruction pre-dispatch time.

[0049] The above step S3 further includes: analyzing the link quality evaluation index values ​​through the load balancing allocation strategy, combining the bandwidth utilization calculation results and the network jitter monitoring feedback results to obtain the link allocation results; determining the optimal transmission time window based on the link allocation results, calculating the instruction pre-dispatch time based on the optimal transmission time window, and adjusting the instruction pre-dispatch time when the network jitter monitoring feedback results indicate that the network jitter exceeds the preset jitter threshold, and generating a scheduling plan as a constraint condition for the transmission time window; wherein, the optimal transmission time window includes the network low congestion period.

[0050] Specifically, in order to address the issue of optimizing the timing and path of playback control commands when network conditions are unstable, bandwidth fluctuates, or latency is high, existing technologies are generally unable to adjust the timing of command transmission in a timely manner when faced with dynamic network conditions and complex user operations, resulting in asynchronous playback content or user operation delays. This application ensures that commands can be sent accurately and promptly under different network environments by monitoring network conditions in real time and using load balancing technology to achieve stable multimedia content display.

[0051] Specifically, in this embodiment, a load balancing allocation strategy is used to optimize the timing of instruction transmission. First, the link quality assessment metrics are analyzed, reflecting the quality of the network connection, primarily including parameters such as signal strength and link latency. By collecting this data, a basic performance assessment of the current network is obtained. Based on this, bandwidth utilization and network jitter monitoring feedback results are incorporated. Bandwidth utilization represents the actual usage of the network, while network jitter reflects the fluctuations in latency during data packet transmission; changes in both directly affect the stability of data transmission. Through the load balancing allocation strategy, the load capacity of each link is calculated based on the aforementioned parameters, determining the links suitable for carrying more data. This strategy ensures that... This process allows a single link to operate under overload conditions, thereby avoiding network congestion. During this process, network traffic allocation is adjusted in real time, for example, using a weighted round-robin method to dynamically allocate traffic, determining the proportion of traffic each link should carry based on the weights of link quality assessment indicators. The load balancing allocation strategy allocates the traffic proportions of multiple transmission paths according to the current network status, and optimizes network resource usage by adjusting bandwidth utilization and controlling network jitter parameters. This ensures efficient and stable data transmission even under poor network conditions. In cases of high jitter or network congestion, the load balancing process can dynamically adjust traffic allocation, prioritizing lower latency paths for data transmission, thereby reducing latency and packet loss.

[0052] Furthermore, based on the above link allocation results, the optimal transmission time window is determined, that is, the time period with low network traffic and less congestion is selected for instruction pre-distribution. This time window is determined by analyzing factors such as link load distribution, bandwidth utilization, and network jitter. Through the sliding window algorithm, the network load curve within a 24-hour period is scanned to identify low-congestion periods, and these periods are preferentially selected as the transmission time for instructions. The above process can avoid network congestion during peak periods, ensure that instructions can be delivered to the target terminal device in a timely manner, and reduce playback delay.

[0053] To adapt to different scenario requirements, especially in real-time video playback or live streaming applications, this method further adjusts the timing of instruction sending based on real-time user behavior and network feedback. If the network jitter value exceeds a preset threshold, the pre-dispatch time of the instruction will be adjusted and postponed to a sub-window with less network jitter, ensuring that the transmission of instructions is more stable and reliable in an unstable network environment.

[0054] In addition, to address the issue of inaccurate command transmission timing under dynamically changing network environments, this embodiment specifically discloses a collaborative decision-making mechanism for load balancing and optimal transmission window determination: First, a load balancing strategy based on link quality weighting is adopted to process the link quality evaluation index values. Specifically, a dynamic weight is assigned to each available transmission path. This weight is calculated by linearly weighting the link quality evaluation index value with its historical stability index, where the current quality index weight is set to 0.7 and the historical stability weight is set to 0.3. This ensures that while prioritizing high-quality links, performance fluctuations are also considered. Subsequently, real-time data collection is performed from the network monitoring module, including bandwidth utilization calculations and network jitter monitoring feedback. Bandwidth utilization is calculated as the ratio of successfully transmitted data per unit time to the theoretical maximum bandwidth, while network jitter is quantified by statistically analyzing the variance of the arrival time intervals of consecutive data packets. Based on this, the load balancer executes a weighted round-robin algorithm, allocating the amount of instruction data to be transmitted to each path according to the calculated dynamic weights. Simultaneously, a congestion avoidance mechanism is introduced: when the instantaneous jitter value of a path exceeds a preset threshold, its weight is automatically and temporarily reduced. A certain proportion of traffic is reduced and redistributed proportionally to other stable paths. Then, based on the optimized link allocation results, a sliding time window algorithm with a size of 2 hours and a sliding step size of 10 minutes is used to analyze the historical load data of each path over the past 24 hours to identify low-congestion periods. The criteria for determining low-congestion periods are that both of the following conditions are met: the average bandwidth utilization rate during this period is lower than a preset utilization rate, and the network jitter percentile is lower than a preset jitter percentile. The longest continuous window among the eligible periods is selected as the optimal transmission time window. Finally, within the determined optimal transmission time window, traffic is transmitted according to instructions. Based on the urgency level and real-time feedback of network jitter, the specific instruction pre-dispatch time is calculated. This is achieved by adding an offset based on instruction priority to the window start time. The offset for urgent instructions is set to 0 for immediate transmission. When real-time network jitter monitoring indicates that the current jitter value exceeds a certain threshold, a delay compensation proportional to the jitter value is added to the pre-dispatch time of all non-urgent instructions. This ultimately generates a scheduling plan containing the precise transmission times of all instructions. This plan serves as the transmission time window constraint upon which priority adjustment in the following steps depends. The delay compensation is calculated by extracting multiple parameters from the execution status feedback data. The media content is timestamped, and its deviation from the local system clock is calculated. Then, an exponentially weighted moving average algorithm is used to fuse the most recent deviations (e.g., 10 deviations), with recent deviations given a larger attenuation factor to quickly respond to network changes. The resulting weighted average is used as the baseline compensation. Next, a linear regression model is used to predict the delay trend over the next few hours (e.g., a 3-hour window). This trend prediction is then superimposed on the baseline compensation to generate the final compensation value. This compensation value is used to correct the playback progress jump prediction model, specifically by adjusting the time offset parameter in the model's input features to achieve synchronous updates of the predicted value.

[0055] By employing load balancing strategies and real-time network status monitoring, this application can precisely schedule the timing of instruction transmission, ensuring timely and effective transmission of playback control instructions under various network conditions, thereby maintaining the synchronization of multimedia content. This method is applicable to real-time content playback scenarios such as video streaming, online education, and live broadcasting, and can optimize user experience, reduce stuttering and latency during playback, and improve the overall playback effect.

[0056] Step S4: Based on the constraints of the optimal transmission time window, adjust the sending priority of the high-probability playback control command sequence, generate a command dispatch queue, extract emergency playback control commands, send them to the target terminal device through a pre-established multi-path transmission channel, and obtain execution status feedback data, such as... Figure 2 As shown.

[0057] The process of generating an instruction dispatch queue includes: obtaining transmission time window constraints from the optimal transmission time window, including duration limits and available bandwidth range; adjusting the transmission priority of high-probability playback control instruction sequences based on the transmission time window constraints using a flow control mechanism; sorting the instructions based on the adjusted transmission priority to generate an instruction dispatch queue containing both urgent and non-urgent instructions; and reducing the position of low-priority instructions in the instruction dispatch queue when the bandwidth utilization rate exceeds a preset bandwidth threshold.

[0058] Specifically, in order to solve the problems of playback control command delay and content synchronization caused by bandwidth limitations or network congestion in multi-terminal and unstable network environments, traditional playback control systems usually cannot optimize the timing of command sending according to real-time network conditions, resulting in asynchronous playback content and poor user experience. This application ensures accurate synchronization of multimedia content across multiple devices and network environments by real-time monitoring of network status, optimizing command sending timing, and dynamically adjusting command priority according to network quality, thereby improving user experience and reducing latency.

[0059] Specifically, in this embodiment, the constraints of the optimal transmission time window are first determined by analyzing key parameters such as network bandwidth and duration limits. The setting of the transmission time window takes into account the available bandwidth range and duration limits of the network. The constraints ensure that the utilization rate of network bandwidth does not exceed a predetermined threshold when sending instructions, thus avoiding overload. Through a flow control mechanism, this application adjusts the sending priority of high-probability playback control instruction sequences. The function of the flow control mechanism is to dynamically adjust the instructions based on the real-time bandwidth utilization rate, ensuring that emergency instructions, such as pause and jump, can be sent first when network bandwidth is insufficient, thereby optimizing the playback experience.

[0060] Specifically, when network bandwidth utilization exceeds a preset threshold, low-priority commands, such as volume adjustment and playback speed adjustment, are automatically identified. Based on the flow control mechanism, these low-priority commands are moved to the back of the queue, prioritizing the transmission of high-priority commands. This process is accomplished through priority sorting. High-priority commands are placed at the front of the queue, while low-priority commands adjust their positions according to real-time network conditions, ensuring the command transmission order adapts to the current network environment. The flow control mechanism is implemented through a dynamic adaptive algorithm. This algorithm uses the moving average of historical bandwidth utilization as a benchmark and adds the feedback adjustment of the current network jitter coefficient. The specific calculation formula is: Dynamic threshold = Historical average utilization × (1 - Jitter coefficient / Maximum tolerable jitter), where the historical average utilization is calculated using the exponentially weighted moving average of the past 15 sampling periods, and the jitter coefficient is normalized to a value within the range [0, 1].

[0061] Furthermore, during the generation of the instruction dispatch queue, the sending priority of instructions is adjusted in real time by combining network monitoring data and bandwidth utilization. When the bandwidth utilization reaches or exceeds a preset threshold, the position of low-priority instructions in the instruction queue is adjusted to ensure that the system load is within an acceptable range. At the same time, the structure of the queue is dynamically adjusted through a load balancing strategy to further ensure that the network load is reasonably distributed. The above adjustments help reduce network congestion and improve transmission efficiency and playback continuity in high bandwidth demand scenarios.

[0062] In addition, this application can also update the instruction allocation queue in real time according to network conditions, such as network jitter and bandwidth utilization, to cope with sudden network fluctuations; by dynamically adjusting the instruction priority, it ensures that even when bandwidth is insufficient, high-priority instructions, such as pause / resume and jump instructions, can be transmitted first, thus ensuring the stability of playback content and the continuity of user experience.

[0063] This technical solution effectively addresses the impact of network environment fluctuations on the transmission of playback control commands by adjusting command transmission priority in real time and optimizing link resource allocation. It ensures timely and stable transmission of commands under various network conditions, thereby achieving efficient synchronization of multimedia display content and reducing playback delays and stuttering.

[0064] The process of obtaining execution status feedback data includes: extracting playback control commands marked as urgent from the command dispatch queue, sending the urgent playback control commands to the target terminal device through a pre-established multi-path transmission channel, receiving command reception confirmation signals from the target terminal device, and collecting execution status feedback data based on the command reception confirmation signals. The execution status feedback data includes timestamps of the multimedia display content.

[0065] Specifically, in order to solve the problem of delayed playback control commands and content asynchrony when multimedia playback systems face network instability, bandwidth limitations, or latency fluctuations, existing technologies are usually unable to adjust the timing of command transmission in real time under complex network environments and changes in user behavior, resulting in asynchrony of multimedia content. This application can achieve accurate content synchronization between multiple terminal devices, reduce playback latency, and optimize user experience.

[0066] Specifically, in this embodiment, the processing of emergency playback control commands involves extracting commands marked as emergency from the command dispatch queue and sending them to the target terminal device through a pre-established multi-path transmission channel. This process ensures timely transmission of emergency commands even under conditions of significant network fluctuations by real-time monitoring and pre-configured network paths. First, the commands in the command dispatch queue are traversed, and playback control commands marked as emergency are selected based on a preset emergency flag. Commands are prioritized after being flagged and filtered. The flag for emergency commands is determined and set based on the command trigger probability value and a preset threshold. After the emergency commands are extracted, they are sent to the target terminal device through the pre-established multi-path transmission channel. The multi-path transmission design aims to improve the reliability of command transmission, especially when network jitter or congestion occurs. Redundant copies can be sent via backup paths to ensure that commands reach the terminal device promptly and accurately. This multi-path transmission method effectively addresses network transmission interference and ensures the priority transmission of emergency commands.

[0067] Once the target terminal device receives an emergency playback control command, it will acquire a command reception confirmation signal from the target terminal device. This confirmation signal contains a unique identifier for the command, used to ensure that the command has been correctly received. Based on this confirmation signal, further execution status feedback data is collected. The execution status feedback data mainly includes timestamps of the multimedia display content, which indicate the actual playback time. Through the above feedback data, the execution status of the command can be understood, and the playback progress can be adjusted in a timely manner to ensure the synchronization of the multimedia display content. In this process, the delay of the command reception confirmation signal can also be processed. When the delay of the confirmation signal exceeds a preset threshold, the command is resent from a backup path to ensure that the command can be executed in a timely manner in the event of network congestion or packet loss, thereby maintaining the synchronization of the content.

[0068] Through the aforementioned technical means, this application can flexibly adjust the timing and path of instruction sending in real-time multimedia display scenarios based on network status, user behavior, and instruction priority, ensuring that playback instructions can be transmitted to each terminal device in a timely and accurate manner. This method can not only effectively reduce the impact of network latency, but also improve the consistency of multimedia content display, avoid playback interruptions caused by network fluctuations or device asynchrony, and optimize the user viewing experience.

[0069] Step S5: Based on the execution status feedback data, adjust the playback progress jump prediction and update the behavior feature vector. When the updated behavior feature vector indicates that there is a content segmentation transmission control requirement, activate the inter-device clock synchronization protocol and synchronize the multimedia content segments in time through the clock synchronization protocol.

[0070] The process of updating the behavior feature vector includes: extracting the timestamp of the multimedia display content from the execution status feedback data; calculating the delay deviation value of the timestamp based on the timestamp; performing weighted averaging on the delay deviation value; correcting the playback progress jump prediction value based on the weighted averaging result; and updating the behavior feature vector based on the corrected playback progress jump prediction value.

[0071] Specifically, in order to solve the problem of playback progress being out of sync with content during multimedia playback due to network latency and bandwidth fluctuations, existing technologies usually cannot adjust the playback progress in real time, especially in different terminal devices and unstable network environments, where playback content is prone to being out of sync; this application adjusts the playback progress by providing real-time feedback information, optimizes the user experience, and can ensure the consistency and continuity of multimedia content.

[0072] Specifically, in this embodiment, the adjustment of playback progress jump prediction is achieved by acquiring execution status feedback data in real time. First, the timestamp of the multimedia display content is extracted from the execution status feedback data. The timestamp is used to indicate the playback time of the current frame or content segment. Based on the timestamp data, a delay deviation value is calculated, that is, by comparing the difference between the local timestamp and the received timestamp, the delay information in network transmission is obtained. In order to ensure accurate prediction of playback progress, this application adopts a weighted average method to process the delay deviation value. The weight of the historical deviation value decreases according to its occurrence frequency. This can prioritize the correction of recent deviations and reduce the impact of long-term deviations on the prediction. The delay deviation value after weighted averaging is used to correct the playback progress jump prediction value. By adjusting the prediction value, the jump prediction is made more accurate. For example, when there is a deviation between the compensated timestamp and the prediction value, the prediction time is adjusted to the compensated time value, thereby optimizing the user experience and reducing the sense of jump in content playback.

[0073] Furthermore, when the updated behavioral feature vector indicates a need for content segmentation transmission control, this application activates an inter-device clock synchronization protocol. This protocol ensures the consistency of content segmentation transmission by precisely synchronizing the clocks of each device. Specifically, it checks the segmentation indicator flag in the synchronized behavioral feature vector. If the flag value is 1, it confirms the existence of a segmentation requirement and initiates a network time protocol. This protocol ensures clock synchronization of multiple devices by exchanging timestamp information between devices, thereby guaranteeing smooth multi-segment transmission of content and avoiding playback discontinuity or interruptions. This process is particularly suitable for video segmentation transmission scenarios, such as in real-time video scenarios like live streaming or online education, where user operation behavior patterns may... Frequent pausing or skipping of content necessitates dividing the video content into multiple segments for transmission. A clock synchronization protocol ensures precise synchronization of playback for each segment, maintaining consistent playback progress across devices even during periods of significant network fluctuation, thus preventing asynchrony or stuttering during segmented playback. By implementing this technical solution, this application effectively addresses playback latency issues caused by network fluctuations and insufficient bandwidth, ensuring accurate synchronization of multimedia content across different network environments. Whether in video playback, live streaming, or online education applications, this application provides a stable synchronization experience, improves the system's adaptability to network fluctuations, and optimizes the user's viewing experience.

[0074] Through the coordination of the above steps, this application improves the stability and response speed of multimedia content playback.

[0075] Example 2:

[0076] The synchronization method for multimedia display content in the embodiments of this application has been described above. The synchronization system for multimedia display content in the embodiments of this application is described below. Please refer to [link / reference]. Figure 3 One embodiment of the multimedia display content synchronization system in this application includes:

[0077] The feature vector acquisition unit is used to collect the playback history and real-time status data of the target terminal device, and analyze them based on time series feature extraction technology to obtain behavioral feature vectors that characterize the user's operation mode.

[0078] The link quality assessment unit is used to calculate the instruction trigger probability value of each playback control instruction within a future time window based on the behavior feature vector, determine the high-probability playback control instruction sequence based on the instruction trigger probability value, and start the network status assessment process to obtain the link quality assessment index value of the current network environment.

[0079] The optimal transmission determination unit is used to process the link quality evaluation index values ​​through load balancing allocation strategy, and combine the bandwidth utilization calculation results and network jitter monitoring feedback results to determine the optimal transmission time window and instruction pre-dispatch time.

[0080] The execution status feedback unit is used to adjust the sending priority of high-probability playback control command sequences based on the constraints of the optimal transmission time window, generate a command dispatch queue, extract emergency playback control commands, send them to the target terminal device through a pre-established multi-path transmission channel, and obtain execution status feedback data.

[0081] The content time synchronization unit is used to adjust the playback progress jump prediction based on the execution status feedback data and update the behavior feature vector. When the updated behavior feature vector indicates that there is a content segmentation transmission control requirement, the inter-device clock synchronization protocol is activated, and the multimedia content segments are synchronized in time through the clock synchronization protocol.

[0082] Through the synergistic cooperation of the aforementioned components, this application further improves the stability and response speed of multimedia content playback.

[0083] Example 3:

[0084] This application also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the multimedia display content synchronization method.

[0085] In summary, this application collects playback history and real-time status data from the target terminal device, and uses time-series feature extraction technology to analyze user operation patterns, generating behavioral feature vectors representing user behavior. This allows for the calculation of command trigger probabilities and the determination of high-probability playback control command sequences. Based on this, a network status assessment mechanism is used to obtain the current network environment's packet loss rate and link quality assessment indicators. Combined with bandwidth utilization and network jitter feedback, the timing and priority of command transmission are adjusted in real time to ensure timely transmission of emergency commands. Furthermore, this application sends emergency control commands through a multi-path transmission channel and adjusts the playback progress based on the execution status feedback data returned by the target terminal device. Finally, when the synchronized behavioral feature vector indicates a need for segmented content transmission control, this application activates an inter-device clock synchronization protocol to ensure precise clock synchronization between devices, thereby guaranteeing the consistency of segmented multimedia content transmission. Through these technical means, this application effectively solves the playback delay and content asynchrony problems caused by network fluctuations, insufficient bandwidth, and device asynchrony in existing technologies, significantly improving content synchronization and playback stability among multiple terminal devices and optimizing the user experience.

[0086] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0087] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0088] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for synchronizing multimedia display content, characterized in that, The method includes: Step S1: Collect the playback history and real-time status data of the target terminal device, and analyze them based on time series feature extraction technology to obtain behavioral feature vectors that represent user operation patterns; Step S2: Based on the behavior feature vector, calculate the instruction trigger probability value of each playback control instruction within the future time window; based on the instruction trigger probability value, determine the high-probability playback control instruction sequence; and start the network status assessment process to obtain the link quality assessment index value of the current network environment. Step S3: Process the link quality evaluation index values ​​through the load balancing allocation strategy, and combine the bandwidth utilization calculation results and network jitter monitoring feedback results to determine the optimal transmission time window and instruction pre-dispatch time. Step S3 further includes: analyzing the link quality evaluation index values ​​through a load balancing allocation strategy, combining the bandwidth utilization calculation results and network jitter monitoring feedback results to obtain link allocation results; determining the optimal transmission time window based on the link allocation results; calculating the instruction pre-dispatch time based on the optimal transmission time window; and adjusting the instruction pre-dispatch time when the network jitter monitoring feedback results indicate that the network jitter exceeds a preset jitter threshold, and generating a scheduling plan as a constraint condition for the transmission time window. Step S4: Based on the constraints of the optimal transmission time window, adjust the sending priority of the high-probability playback control instruction sequence, generate an instruction dispatch queue, extract emergency playback control instructions, send them to the target terminal device through a pre-established multi-path transmission channel, and obtain execution status feedback data. Step S5: Based on the execution status feedback data, adjust the playback progress jump prediction and update the behavior feature vector. When the updated behavior feature vector indicates that there is a content segmentation transmission control requirement, activate the inter-device clock synchronization protocol and synchronize the multimedia content segments in time through the clock synchronization protocol.

2. The method according to claim 1, characterized in that, Step S1 further includes: The playback history data and real-time status data are obtained from the target terminal device and organized into time series data according to timestamps. The playback history data includes historical playback progress jump records and pause / resume timing information. The real-time status data includes the current playback progress, playback speed, volume level, and play / pause operation time points. Time series feature extraction technology is used to extract time series features from the playback history data and the real-time status data to identify the user's operation pattern. Based on the user's operation pattern, a behavioral feature vector is generated, which includes playback progress jump prediction, pause and resume timing judgment, volume adjustment frequency analysis results, and playback speed change trend data.

3. The method according to claim 1, characterized in that, In step S2, determining the high-probability playback control instruction sequence includes: Volume adjustment frequency analysis results and playback speed change trend data are extracted from the behavioral feature vector, and a long short-term memory network is used to process the volume adjustment frequency analysis results and playback speed change trend data to calculate the trigger probability value of each instruction within the future time window. The potential instructions are sorted based on the instruction trigger probability value, and the instructions whose trigger probability value exceeds a preset probability threshold are added to the high probability playback control instruction sequence. The high probability playback control instruction sequence includes pause, jump, playback speed adjustment and volume adjustment instructions.

4. The method according to claim 3, characterized in that, In step S2, the network state assessment process is initiated, including: By monitoring the data packet transmission process, we obtain data packet loss rate statistics and simultaneously activate the transmission delay measurement method to calculate link delay by sending test data packets. The packet loss rate and the link latency are weighted and fused to obtain a link quality evaluation index value. When the link quality evaluation index value is lower than a preset quality threshold, the load balancing optimization process is triggered.

5. The method according to claim 1, characterized in that, Step S3 further includes: The optimal transmission time window includes periods of low network congestion.

6. The method according to claim 1, characterized in that, In step S4, generating the instruction dispatch queue includes: The transmission time window constraints, including duration limits and available bandwidth range, are obtained from the optimal transmission time window. Based on the transmission time window constraints, a flow control mechanism is used to adjust the transmission priority of the high-probability playback control instruction sequence. The instructions are sorted based on the adjusted transmission priority to generate an instruction dispatch queue containing both urgent and non-urgent instructions. When the bandwidth utilization rate exceeds a preset bandwidth threshold, the position of low-priority instructions in the instruction dispatch queue is reduced.

7. The method according to claim 6, characterized in that, In step S4, obtaining execution status feedback data includes: Extract playback control commands marked as urgent from the command dispatch queue, send the urgent playback control commands to the target terminal device through a pre-established multi-path transmission channel, and receive command reception confirmation signals from the target terminal device. Collect execution status feedback data based on the command reception confirmation signals, wherein the execution status feedback data includes timestamps of multimedia display content.

8. The method according to claim 1, characterized in that, In step S5, updating the behavior feature vector includes: The timestamp of the multimedia display content is extracted from the execution status feedback data. Based on the timestamp, the delay deviation value of the timestamp is calculated. The delay deviation value is weighted and averaged. The playback progress jump prediction value is corrected based on the weighted average value. The behavior feature vector is updated based on the corrected playback progress jump prediction value.

9. A synchronization system for multimedia display content, used to implement the synchronization method for media display content as described in any one of claims 1-8, characterized in that, The system includes: The feature vector acquisition unit is used to collect the playback history and real-time status data of the target terminal device, and analyze them based on time series feature extraction technology to obtain behavioral feature vectors that characterize the user's operation mode. The link quality assessment unit is used to calculate the instruction trigger probability value of each playback control instruction within a future time window based on the behavior feature vector, determine the high-probability playback control instruction sequence based on the instruction trigger probability value, and start the network status assessment process to obtain the link quality assessment index value of the current network environment. The optimal transmission determination unit is used to process the link quality evaluation index values ​​through a load balancing allocation strategy, and combine the bandwidth utilization calculation results and network jitter monitoring feedback results to determine the optimal transmission time window and instruction pre-dispatch time. The execution status feedback unit is used to adjust the sending priority of the high-probability playback control instruction sequence based on the constraints of the optimal transmission time window, generate an instruction dispatch queue, extract emergency playback control instructions, send them to the target terminal device through a pre-established multi-path transmission channel, and obtain execution status feedback data. The content time synchronization unit is used to adjust the playback progress jump prediction based on the execution status feedback data and update the behavior feature vector. When the updated behavior feature vector indicates that there is a content segmentation transmission control requirement, the inter-device clock synchronization protocol is activated, and the multimedia content segments are time-synchronized through the clock synchronization protocol.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instruction is executed by the processor, it implements the method for synchronizing multimedia display content as described in any one of claims 1-8.

Images