An optimization method for realizing zero delay of video switching

Abstract

The application discloses an optimization method for realizing zero delay of video switching, especially for online video playing and streaming media service. The method comprises monitoring the remaining playing time of the current video through a background thread, starting the frame extraction and decoding operation of the next video in advance in the last few seconds of the video playing, and buffering the decoded frame to the cache area in parallel. When the video switches, the time stamps of the current video and the next video are accurately connected through a double time stamp synchronization mechanism, reducing the black screen and delay phenomenon. Through the adaptation of different hardware platform graphic API, the rendering path and priority are dynamically adjusted. The application is suitable for ordinary video playing, advertisement insertion and real-time video streaming media playing in complex network environment, and can greatly improve the fluency of video switching and user experience. Through multi-thread parallel decoding and cache management, the utilization of device resources is effectively optimized, and the efficiency and stability of video playing can be maintained in low-end devices or high-load environment.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to an optimized method for achieving zero-latency video switching in online video playback or streaming media services. Background Technology

[0002] With the rapid development of video streaming and online video playback technologies, users' demands for video viewing experiences are increasing. To ensure smooth playback, existing technologies typically rely on increasing network bandwidth, expanding video buffers, and improving the processing power of decoders. Under good network conditions, video switching can be completed relatively smoothly, with relatively few delays and stutters. However, when faced with complex network environments or insufficient device performance, existing technologies still present significant challenges in handling video switching.

[0003] Existing technologies have several shortcomings when dealing with different network conditions and hardware configurations. First, under conditions of unstable network bandwidth or limited device resources, video switching often results in delays, black screens, or even playback stuttering. These problems mainly stem from the inability to extract and decode the next video frame in a timely manner, leading to imperfections in seamless video transitions. Second, timestamp synchronization is a critical issue in video switching. Existing solutions often fail to accurately align the timestamps of the current video with the next video, resulting in frame skipping or playback misalignment during the transition. Furthermore, existing technologies perform poorly on various hardware platforms, especially on low-end devices or under high load environments, where the utilization rate of the graphics processing unit (GPU) is low, resulting in insufficient rendering efficiency and further exacerbating the issues of smoothness during video transitions and degrading the visual experience.

[0004] In certain specific application scenarios, the limitations of existing video switching technologies are particularly pronounced. For example, in ad insertion scenarios, the video frequently switches between the main video and the ad content, and existing technologies struggle to ensure a seamless transition between the two, often resulting in black screens and latency issues before and after ad playback. Furthermore, when switching to high-resolution videos, the large data volume of video frames makes it difficult for existing caching and decoding technologies to handle instantaneous high-resolution video switching, leading to playback pauses and image quality degradation, severely impacting the user's viewing experience. Summary of the Invention

[0005] To address issues such as video switching latency and black screens, timestamp asynchrony, low rendering efficiency, and limitations of ad insertion and high-resolution video switching, this invention provides an optimized method for achieving zero-latency video switching. By employing multi-threaded parallel decoding technology and a high-speed cache management strategy, this invention can preload and decode the next video segment before switching, reducing latency during switching. Furthermore, the introduction of a dual timestamp mechanism effectively solves the problem of time asynchrony, significantly reducing time misalignment and black screens during video switching. Simultaneously, this invention optimizes the rendering path through hardware acceleration technology, adapting to the performance of different devices, especially maintaining stable rendering efficiency and video smoothness under high load environments.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] An optimization method for achieving zero-latency video switching includes the following steps:

[0008] Step 1: In the last few seconds of the current video playback, monitor the remaining playback time of the video through a background thread. When the remaining playback time reaches the preset time period, start the frame extraction operation of the next video in advance.

[0009] Step 2: Extract the frames of the next video, decode these frames in parallel, and cache the decoded frames in the cache area;

[0010] Step 3: When switching videos, the last frame of the current video is synchronized with the initial frame of the next video through timestamp synchronization, and the decoded frame is read from the buffer and played.

[0011] Step 4: Use graphics APIs adapted to different platforms for video rendering, and improve rendering efficiency and smoothness of video switching through hardware acceleration.

[0012] Preferably, the background thread dynamically adjusts the extraction and decoding rate of the next video frame based on the remaining playback time of the video, the network bandwidth status, and the load of the video decoder. Specifically, when the network fluctuates, video frames with lower bit rates are extracted first, and when the load is high, the number of parallel decoding operations of the decoder is reduced to ensure a smooth video switching process and reduce system stuttering.

[0013] Preferably, the high-speed cache is dynamically adjusted according to different video resolutions, frame rates, and encoding formats. The cache includes at least two independent cache modules: one for storing the frame data of the current video and the other for preloading and storing the decoded frames of the next video. When switching videos, the cache module of the next video is called first to avoid frame delay.

[0014] Preferably, the timestamp synchronization adopts a dual timestamp mechanism, which connects the logical timestamp and the actual playback timestamp of the last frame of the current video, and synchronizes the logical timestamp and the actual playback timestamp of the initial frame of the next video. The maximum allowable error between the logical timestamp and the actual timestamp is set to no more than 10 milliseconds.

[0015] Preferably, the dual timestamp mechanism adopts the AKDS model, and the implementation steps are as follows:

[0016] S1: Capture the logical timestamp of the last frame of the current video. and actual playback timestamp And the logical timestamp of the initial frame of the next video. and actual playback timestamp Calculate the timestamp difference ΔT n for:

[0017]

[0018] S2: Use a Kalman filter to calculate the timestamp difference ΔT n Smoothing is performed using the Kalman gain K. n The logical timestamp is updated using the following formula:

[0019] S3: Dynamically adjust the learning rate α based on the changing trend of timestamp error. n Through the function α n =f(ΔT) n-1 ,ΔT n-2 ,...) Calculate the learning rate to optimize the feedback response during time synchronization;

[0020] S4: Predict timestamp drift using a Long Short-Term Memory (LSTM) network, inputting the historical timestamp difference ΔT. n Output the predicted timestamp drift ΔT for the next frame. pred This predicted value is then used to update the logical timestamp of the next frame.

[0021]

[0022] S5: Based on the objective of minimizing timestamp error, a reinforcement learning algorithm is used to dynamically adjust parameters such as the Kalman filter and adaptive control. The reward function is... By optimizing control parameters, the synchronization accuracy is maximized, and the delay and black screen phenomenon during video switching are reduced.

[0023] S6: Combining Kalman filtering, adaptive control, deep learning prediction, and reinforcement learning dynamic adjustment strategies, the logical timestamp and actual playback timestamp of each frame during video switching are optimized to ensure that the error between the logical timestamp and the actual timestamp does not exceed 10 milliseconds, reducing time misalignment and black screen phenomena during switching.

[0024] Preferably, the graphics API includes OpenGL, DirectX, and Metal. Rendering modes are selected to adapt to different hardware platforms, and the optimal rendering path is automatically selected through a hardware characteristic detection mechanism. The detected hardware characteristics include processor type, graphics processing unit (GPU) model, video memory size, and device temperature. By combining these characteristics, rendering priority and thread allocation are dynamically adjusted to ensure the stability and efficiency of the device under high load.

[0025] Preferably, the video switching optimization process further includes the following steps:

[0026] 1) Detect the processor architecture, graphics processing unit (GPU) type, video memory capacity, and current device temperature of the device;

[0027] 2) Select the optimal rendering path based on the detection results, and use multi-threaded parallel decoding technology on devices that support multi-core processing;

[0028] 3) When the device temperature is too high, automatically reduce the rendering priority and reduce the number of parallel decoding threads to avoid overheating while maintaining basic video smoothness.

[0029] Preferably, the method is applicable to online streaming media, real-time video playback, and advertising insertion scenarios. When switching advertising videos, it prioritizes the automatic selection of low-resolution advertising frames for preloading based on network bandwidth. When playing the main video, it automatically adjusts to high resolution to ensure a good viewing experience. In real-time video playback, it further optimizes the caching mechanism to reduce latency caused by network jitter.

[0030] Preferably, the cached data is adaptively optimized according to the video encoding formats including H.264, H.265, and AV1. Each encoding format uses a specific compression and decoding optimization algorithm, and the decoding priority is adjusted according to the different encoding formats to ensure that different video formats maintain a low latency state when switching.

[0031] Preferably, in the scenario of ad insertion on a streaming media platform, the method includes:

[0032] Step S1: Just before the main video is about to start the ad insertion, load at least the first segment of the ad video in advance through a background thread according to a predetermined time threshold, and complete the decoding and caching before insertion;

[0033] Step S2: During the ad playback, the next segment of the main video is preloaded via a background thread to ensure a seamless transition of the main video after the ad ends;

[0034] Step S3: During ad playback, automatically adjust the ad playback order and duration based on user viewing history, ad skip rate, and user interaction data to increase ad completion rate;

[0035] Step S4: Dynamically adjust the resolution and bitrate of the ad video according to the current network status, prioritize playing low-resolution ad videos to reduce playback delay, ensure smooth ad playback, and load high-resolution main video content in the background before the ad ends.

[0036] Compared with existing technologies, the beneficial effects of this invention are as follows: By initiating the frame extraction operation of the next video in the last few seconds of the current video playback and combining it with multi-threaded parallel decoding technology, this invention can effectively reduce the latency during video switching by 70% to 80%. Through a dual timestamp synchronization mechanism, this invention ensures precise alignment between the last frame of the current video and the initial frame of the next video, controlling the time error within 10 milliseconds, significantly reducing the black screen phenomenon during video switching by more than 80%, thus significantly improving the user viewing experience. This invention uses graphics APIs adapted to different hardware platforms, such as OpenGL, DirectX, and Metal, and dynamically adjusts rendering priority and thread allocation using a hardware detection mechanism, ensuring that the smoothness of video switching is not affected on high-load devices. Whether on high-performance or low-end devices, this invention maintains a high frame rate and video rendering efficiency. When inserting advertisements, this invention prioritizes loading low-resolution frames of the advertisement video while preloading the main video in the background, ensuring a seamless transition from the advertisement to the main video after playback. The black screen time during the transition between the advertisement and the main video is reduced by 80%, the frame rate difference is significantly reduced, and the smoothness of the switching process is improved. This invention dynamically adjusts video decoding rate and cache management strategies in complex environments with fluctuating network bandwidth and high latency, ensuring stability during video switching. This significantly reduces the number of stutters during video switching and keeps frame rate fluctuations within a reasonable range. Furthermore, this invention selects the optimal rendering path by detecting device hardware characteristics (such as processor type, GPU model, video memory size, and temperature), ensuring device stability under high load. Simultaneously, by dynamically adjusting rendering priority and thread allocation, it achieves smooth video switching effects even on mobile devices and low-end hardware. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] in:

[0039] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;

[0040] Figure 2 This is a flowchart of the dual timestamp synchronization mechanism in an embodiment of the present invention;

[0041] Figure 3 This is a flowchart illustrating an advertising insertion scenario applied to a streaming media platform in an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0043] Example 1

[0044] like Figure 1 As shown, this is an embodiment of the present invention, which provides an optimization method for achieving zero-latency video switching, including:

[0045] Step 1: In the last few seconds of the current video playback, monitor the remaining playback time of the video through a background thread. When the remaining playback time reaches the preset time period, start the frame extraction operation of the next video in advance.

[0046] During video playback, the system uses a dedicated background thread to monitor the remaining playback time of the current video. This background thread periodically checks the playback progress, and when it detects that the remaining playback time has reached a preset timeframe (e.g., 3 to 5 seconds remaining), it automatically initiates the frame extraction process for the next video segment. This ensures that frames for the next video segment are prepared for extraction and decoding before the current video ends, reducing waiting time during video transitions and preventing stuttering or black screens.

[0047] Specifically, the background thread dynamically adjusts the extraction and decoding rate of the next video frame based on the remaining playback time of the video, the network bandwidth status, and the load of the video decoder. When the network fluctuates, it prioritizes the extraction of video frames with lower bit rates, and when the load is high, it reduces the number of parallel decoders to ensure smooth video switching and reduce system stuttering.

[0048] Further, step 2: extract the frames of the next video, decode these frames in parallel, and cache the decoded frames in the cache area;

[0049] After initiating the frame extraction operation, the system processes the extracted frames using multi-threaded parallel decoding. Multi-threaded parallel decoding can process multiple video frames simultaneously, significantly improving decoding speed and efficiency. The decoded frames are temporarily stored in a high-speed cache. This cache dynamically adjusts based on the video's resolution, frame rate, and encoding format, ensuring optimal decoding buffer space for different video formats. By pre-caching the decoded frames, the system can immediately read them during video transitions, avoiding stuttering caused by decoding delays.

[0050] Specifically, the high-speed cache is dynamically adjusted according to different video resolutions, frame rates, and encoding formats. The cache includes at least two independent cache modules: one for storing the frame data of the current video and the other for preloading and storing the decoded frames of the next video. When switching videos, the cache module of the next video is called first to avoid frame delay.

[0051] Furthermore, in step 3: when switching videos, the last frame of the current video is synchronized with the initial frame of the next video through timestamp synchronization, and the decoded frame is read from the buffer and played.

[0052] During video switching, the system uses a dual-timestamp synchronization mechanism to align the last frame of the current video with the initial frame of the next video. Dual-timestamp synchronization includes a logical timestamp and an actual playback timestamp. By synchronizing these two timestamps, the playback progress of the current and next videos can be precisely matched. This precise time synchronization avoids frame skipping, black screens, and time misalignment issues during video switching. Simultaneously, during switching, the system reads and plays the decoded next video frame from the high-speed cache, achieving a seamless transition and enhancing the user's viewing experience.

[0053] Specifically, the timestamp synchronization adopts a dual timestamp mechanism, which connects the logical timestamp and the actual playback timestamp of the last frame of the current video, and synchronizes the logical timestamp and the actual playback timestamp of the initial frame of the next video. The maximum allowable error between the logical timestamp and the actual timestamp is set to no more than 10 milliseconds.

[0054] In existing technologies, timestamp synchronization issues often lead to frame skipping and latency during video transitions. The "dual timestamp synchronization mechanism" proposed in this invention involves two key timestamps:

[0055] Logical timestamp: This is a timestamp generated by the video playback logic, indicating the current playback progress of the video.

[0056] Actual playback timestamp: This timestamp reflects the actual playback time at the device system level.

[0057] In practical applications, there may be discrepancies between the logical timestamp and the actual playback timestamp, especially under network fluctuations or high device load. Through a dual timestamp synchronization mechanism, the system can align the logical timestamp with the initial frame of the next video while simultaneously matching the actual playback time, ensuring accurate timing between the two video segments. This mechanism prevents time misalignment issues and achieves seamless switching by monitoring and adjusting the difference between these two timestamps in real time.

[0058] Multi-threaded parallel decoding is a crucial component of this invention, primarily used to accelerate the video frame decoding process. Specifically, the decoding task is distributed across multiple threads, which can work concurrently, thereby increasing decoding speed and reducing waiting time. The tasks of each thread are as follows:

[0059] Thread allocation: The video frame decoding task is divided into multiple smaller tasks, each handled by a different thread. For example, one thread might be responsible for decoding keyframes, while other threads might be responsible for decoding non-keyframes.

[0060] Task scheduling: Decoding tasks are allocated and scheduled through the CPU and GPU of multi-core devices to ensure that decoding operations are executed in parallel on multi-core systems, thereby maximizing the utilization of system resources.

[0061] Adjustments when resources are limited: When device resources (such as CPU, memory, etc.) are limited, the system will reduce the number of threads or dynamically adjust the priority of decoding tasks to ensure that the device does not crash or its performance degrades due to excessive use of resources.

[0062] This mechanism is particularly suitable for high-resolution videos or scenarios with large video frames and complex decoding, reducing latency while ensuring video quality.

[0063] During video switching, the system automatically selects the appropriate graphics API for video rendering based on different hardware platforms, such as OpenGL, DirectX, or Metal. By detecting hardware characteristics such as the device's processor architecture, graphics processing unit (GPU) type, video memory capacity, and device temperature, the system can dynamically adjust the rendering path, prioritizing hardware acceleration features (such as GPU acceleration) to improve rendering efficiency. When the device load is high, the system automatically reduces the rendering priority and the number of parallel rendering threads to prevent the device from overheating while ensuring smooth video switching.

[0064] Hardware-accelerated rendering technology relies on graphics APIs (such as OpenGL, DirectX, and Metal) from different platforms to accelerate video processing and rendering. Specifically, this invention achieves efficient rendering through the following methods:

[0065] Hardware acceleration: On devices that support hardware acceleration, video decoding and rendering tasks are allocated to the GPU, thereby improving rendering efficiency and reducing CPU load. During video transitions, the GPU can render new video frames faster, reducing stuttering.

[0066] API Adaptation: This invention automatically selects the appropriate graphics API based on the device's hardware characteristics. For example, DirectX is used on Windows platforms, while Metal is used on Apple devices. Each API is optimized for different hardware, ensuring cross-platform rendering efficiency.

[0067] Dynamically adjust the rendering path: By detecting the device's hardware characteristics (such as processor architecture, video memory size, and temperature), the system can automatically adjust the rendering path based on the load. Under high load, the system reduces the rendering priority to prevent the device from overheating and ensure smooth video transitions.

[0068] Dynamically adjusting the rendering path is based on the following hardware characteristics: CPU utilization, GPU temperature, video memory usage, and network bandwidth. The specific implementation steps are as follows:

[0069] Processor utilization and video memory information are collected every 100 milliseconds.

[0070] When the GPU temperature exceeds 70°C, the system automatically reduces the rendering quality to reduce the load.

[0071] When network bandwidth is below 5Mbps, switch to low latency mode to avoid lag.

[0072] Specifically, such as Figure 2 As shown, the dual timestamp mechanism adopts the AKDS (Adaptive Kalman Dynamic Synchronization) model, and the implementation steps are as follows:

[0073] S1: Capture the logical timestamp of the last frame of the current video. and actual playback timestamp And the logical timestamp of the initial frame of the next video. and actual playback timestamp Calculate the timestamp difference ΔT n for:

[0074]

[0075] S2: Use a Kalman filter to calculate the timestamp difference ΔT n Smoothing is performed using the Kalman gain K. n The logical timestamp is updated using the following formula:

[0076] The Kalman gain parameter is used to correct and update errors during video synchronization. By adjusting this parameter, the system can dynamically correct timestamp deviations based on the difference between the current video frame and the predicted frame, thereby improving synchronization accuracy during video switching.

[0077] S3: Dynamically adjust the learning rate α based on the changing trend of timestamp error. n Through the function α n =f(ΔT) n-1 ,ΔT n-2 ,...) Calculate the learning rate to optimize the feedback response during time synchronization;

[0078] The learning rate is a crucial parameter affecting model convergence speed. A higher learning rate can accelerate the training process but may lead to model instability; a lower learning rate can improve model accuracy but requires more training time. Depending on the specific application requirements, the learning rate can be dynamically adjusted to balance model stability and response speed.

[0079] S4: Predict timestamp drift using a Long Short-Term Memory (LSTM) network, inputting the historical timestamp difference ΔT. n Output the predicted timestamp drift ΔT for the next frame. pred This predicted value is then used to update the logical timestamp of the next frame.

[0080] S5: Based on the objective of minimizing timestamp error, a reinforcement learning algorithm is used to dynamically adjust parameters such as the Kalman filter and adaptive control. The reward function is... By optimizing control parameters, the synchronization accuracy is maximized, and the delay and black screen phenomenon during video switching are reduced.

[0081] During video switching optimization, a reward function is used to measure the impact of each operation on video quality and switching speed. The system adjusts its operation strategy in real time based on the viewer's experience, such as video smoothness, buffering time, and frame latency, to minimize latency and improve user experience.

[0082] S6: Combining Kalman filtering, adaptive control, deep learning prediction, and reinforcement learning dynamic adjustment strategies, the logical timestamp and actual playback timestamp of each frame during video switching are optimized to ensure that the error between the logical timestamp and the actual timestamp does not exceed 10 milliseconds, reducing time misalignment and black screen phenomena during switching.

[0083] Further, step 4: Use graphics APIs adapted to different platforms for video rendering, and improve rendering efficiency and smoothness of video switching through hardware acceleration.

[0084] Specifically, the graphics APIs include OpenGL, DirectX, and Metal. Rendering modes are selected to adapt to different hardware platforms, and the optimal rendering path is automatically selected through a hardware characteristic detection mechanism. The detected hardware characteristics include processor type, graphics processing unit (GPU) model, video memory size, and device temperature. By combining these characteristics, rendering priority and thread allocation are dynamically adjusted to ensure the stability and efficiency of the device under high load.

[0085] Furthermore, the video switching optimization process further includes the following steps:

[0086] 1) Detect the processor architecture, graphics processing unit (GPU) type, video memory capacity, and current device temperature of the device;

[0087] 2) Select the optimal rendering path based on the detection results, and use multi-threaded parallel decoding technology on devices that support multi-core processing;

[0088] 3) When the device temperature is too high, automatically reduce the rendering priority and reduce the number of parallel decoding threads to avoid overheating while maintaining basic video smoothness.

[0089] Furthermore, the method is applicable to online streaming media, real-time video playback, and advertising insertion scenarios. When switching advertising videos, it prioritizes the automatic selection of low-resolution advertising frames for preloading based on network bandwidth. When playing the main video, it automatically adjusts to high resolution to ensure a good viewing experience. In real-time video playback, the caching mechanism is further optimized to reduce latency caused by network jitter.

[0090] Furthermore, the cached data is adaptively optimized according to video encoding formats including H.264, H.265, and AV1. Each encoding format uses a specific compression and decoding optimization algorithm, and the decoding priority is adjusted according to the different encoding formats to ensure that different video formats maintain a low latency state when switching.

[0091] Furthermore, such as Figure 3 As shown, in an advertising insertion scenario applied to a streaming media platform, the method includes:

[0092] Step S1: Just before the main video is about to start the ad insertion, load at least the first segment of the ad video in advance through a background thread according to a predetermined time threshold, and complete the decoding and caching before insertion;

[0093] Step S2: During the ad playback, the next segment of the main video is preloaded via a background thread to ensure a seamless transition of the main video after the ad ends;

[0094] Step S3: During ad playback, automatically adjust the ad playback order and duration based on user viewing history, ad skip rate, and user interaction data to increase ad completion rate;

[0095] Step S4: Dynamically adjust the resolution and bitrate of the ad video according to the current network status, prioritize playing low-resolution ad videos to reduce playback delay, ensure smooth ad playback, and load high-resolution main video content in the background before the ad ends.

[0096] Example 2

[0097] In scenarios involving ad insertion, the video frequently switches between the main video and the ad content. Existing technologies struggle to ensure seamless transitions, often resulting in black screens or delays. This invention proposes an optimized method that achieves zero-latency video switching in ad insertion scenarios.

[0098] The specific steps are as follows:

[0099] When the main video is about to switch to an advertisement, at least the first frame of the advertisement video is preloaded via a background thread. Before the scheduled time for the advertisement to be inserted, the system monitors the remaining playback time of the current video via a thread. When the remaining playback time reaches a preset threshold, the frame extraction and decoding operation of the advertisement video is started in advance to avoid delays during advertisement insertion.

[0100] The frame decoding of the ad video is processed in parallel using multiple threads, and the decoded frames are stored in a high-speed cache. During ad playback, background threads continue to extract and decode subsequent parts of the ad and preload the next segment of the main video during playback. The cache includes two independent modules, one for the ad video and the other for the main video, thus ensuring a seamless transition between the ad and the main video and avoiding stuttering or black screens during video switching.

[0101] During ad playback, the system dynamically adjusts the ad video resolution and bitrate based on the current network bandwidth. When network conditions are poor, lower-resolution ad frames are prioritized to ensure smooth playback. As network conditions improve, the system can revert to a higher resolution to enhance the user's viewing experience. Simultaneously, as the ad playback nears its end, the system automatically adjusts the main video resolution to a high-quality version to ensure optimal viewing quality.

[0102] The timestamp synchronization between the ad video and the main video adopts a dual timestamp mechanism. The background thread connects the logical timestamp of the last frame of the ad with the timestamp of the initial frame of the main video to ensure that the main video can play seamlessly after the ad ends, avoiding frame skipping or delay.

[0103] During ad playback, the system automatically adjusts the order and duration of ad playback based on user viewing history, ad skip rate, and user interaction data (such as ad clicks and skip times) to increase ad completion rates. This dynamic adjustment mechanism not only improves ad effectiveness but also prevents user experience delays or interruptions caused by unnecessary ad insertions.

[0104] Example 3

[0105] This embodiment mainly focuses on optimizing rendering efficiency under different hardware platforms to ensure smooth video switching on high-load or low-end devices, especially when device resources are limited, reducing latency and black screen during video switching.

[0106] When a device starts playing video, the system first automatically detects the device's hardware characteristics through a background thread, specifically including processor architecture, graphics processing unit (GPU) model, video memory capacity, and current device temperature. Based on these detection results, the system selects the optimal rendering path to ensure smooth video rendering under the current device hardware environment. For example, on devices that support multi-core processing, the system uses multi-threaded parallel decoding technology to maximize the utilization of the device's processing power.

[0107] When the video memory capacity is low or the device temperature is high, the system will automatically reduce the rendering priority and the number of parallel decoding threads to avoid overheating of the device while maintaining smooth video playback.

[0108] During video playback, the system dynamically adjusts rendering priority based on the device's real-time load. When the system detects excessively high device temperature or excessive processor load, it prioritizes rendering keyframes and reduces the number of non-keyframes to alleviate device stress. This effectively prevents video stuttering and black screens caused by hardware overload.

[0109] This invention employs multi-platform compatible graphics APIs, such as OpenGL, DirectX, and Metal, to leverage the hardware acceleration capabilities of different devices. On devices that support hardware acceleration, the system automatically switches to hardware acceleration mode to improve video rendering efficiency. In this way, even during high-resolution video switching, the system can maintain a high frame rate, ensuring smooth video playback.

[0110] Depending on the processing power of the device, the system can dynamically switch between different rendering modes. On high-performance devices, the system prioritizes the high-precision rendering mode to ensure the clarity and detail of the video image; while on low-end devices or under high load, the system will automatically switch to the low-precision rendering mode to save device resources and ensure smooth video playback.

[0111] In real-time video playback scenarios, the system dynamically adjusts the rendering mode and decoding strategy by monitoring network bandwidth and device load in real time. For example, in a weak network environment, the system will prioritize reducing the video resolution to ensure the continuity of video playback; while when the device load is reduced, the system will automatically restore high-resolution playback.

[0112] Example 4

[0113] To verify the beneficial effects of this invention, multiple experimental tests were conducted to evaluate the performance of the video switching optimization method in different scenarios. The experiments were divided into three scenarios: 1) ordinary online video playback; 2) advertising insertion scenarios; and 3) video switching performance in complex network environments. Experimental data, compared with existing technologies, demonstrate the superiority of this invention in reducing latency, improving smoothness, and eliminating black screen phenomena.

[0114] Experimental setup and equipment configuration:

[0115] Test equipment: ordinary laptops (i5 processor, 8GB RAM, integrated GPU), high-end desktop computers (i7 processor, 16GB RAM, dedicated GPU), and mobile devices (mid-range smartphones).

[0116] Network environment: including standard broadband (50Mbps), network jitter (latency 100-300ms), and bandwidth fluctuation (10Mbps-50Mbps).

[0117] Video content: 1080p high-definition video, 5-minute clip.

[0118] Testing tools: Ffmpeg and a custom background thread were used for frame extraction and timestamp synchronization.

[0119] 1. Typical online video playback scenario:

[0120] This scenario tested video switching latency, black screen time, and frame rate performance under a normal network environment (50Mbps). The optimization effects of the present invention were compared with those of existing technologies.

[0121] Test Project Existing technology (no optimization) This invention (using optimization) Average video switching latency 300ms 90ms Black screen time 200ms 50ms Average frame rate 25fps 30fps User experience rating (out of 10) 7 9

[0122] Data analysis: Under normal network conditions, the video switching latency was reduced by about 70% and the black screen time was reduced by 75% through the optimization method of this invention. The user experience score was significantly improved, indicating that this invention can effectively improve the smoothness of video switching in common scenarios.

[0123] 2. Advertising insertion scenarios:

[0124] The test focused on performance during ad insertion, particularly the transition time and smoothness between ads and the main video. This scenario was tested using standard broadband (50Mbps).

[0125] Test Project Existing technology (without optimization) This invention (using optimization) Ad switching delay 700ms 130ms Black screen time between advertisements and main video 300ms 60ms Frame rate difference between main video and advertisement 10fps 2fps

[0126] Data analysis: In ad insertion scenarios, the latency of ad video switching is significantly reduced through optimization by this invention, and the black screen time when switching between ads and main videos is reduced by 80%. Frame rate differences are also significantly reduced, ensuring seamless and smooth switching between ads and main videos.

[0127] 3. Video switching performance in complex network environments:

[0128] This scenario simulates network jitter (latency 100-300ms) and bandwidth fluctuations (10Mbps-50Mbps) to test video switching latency, black screen time, and frame rate fluctuations.

[0129] Test Project Existing technology (without optimization) This invention (using optimization) Average latency under network jitter 1200ms 280ms Video switching black screen time 500ms 80ms Frame rate fluctuation range ±15fps ±3fps User experience rating (out of 10) 5 8

[0130] Data analysis: In complex network environments, this invention significantly reduces video switching latency, reduces black screen time by 80%, and narrows frame rate fluctuation range by 80%. User experience rating improved from 5 to 8, indicating that even under unstable network conditions, this invention can maintain good smoothness and stability.

[0131] 4. Mobile device testing:

[0132] Test the performance of mobile devices (smartphones) under complex network conditions (bandwidth fluctuations, network latency) and evaluate the effectiveness of video switching optimization.

[0133] Test Project Existing technology (without optimization) This invention (using optimization) Video switching delay 1000ms 300ms Black screen time 600ms 100ms Average frame rate 20fps 28fps Difference in power consumption (%) 15％ 8％

[0134] Data analysis: On mobile devices, the optimization method of this invention effectively reduces video switching latency and reduces black screen time by 83%. Simultaneously, the average frame rate is also improved, and it performs well in reducing power consumption.

[0135] The experimental data shows that:

[0136] This invention significantly reduces video switching latency by up to 80%.

[0137] The black screen time is significantly reduced, especially during ad interruptions and in complex network environments.

[0138] The frame rate remained stable with minimal fluctuations, ensuring smooth video playback.

[0139] On mobile devices, the optimization is significant, saving power consumption and enhancing the user experience.

[0140] The experimental data fully demonstrates the significant technical effects of this invention in practical applications, especially its superiority in dealing with complex networks and multi-device scenarios.

[0141] In summary, this invention significantly reduces latency and black screen issues during video switching by pre-loading the next video frame, employing multi-threaded parallel decoding technology, a dual timestamp synchronization mechanism, and dynamic rendering optimization, thereby improving video smoothness and user viewing experience. This invention is also applicable to applications such as ad insertion, high-resolution video switching, and complex network environments, ensuring stable and efficient video switching across different scenarios and device platforms. Through intelligent hardware adaptation and cache management, this invention can maintain smooth video playback on low-end devices and in high-load environments, demonstrating broad application prospects and practical value.

[0142] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0143] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0144] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An optimized method for achieving zero-latency video switching, characterized in that, Includes the following steps: Step 1: In the last few seconds of the current video playback, monitor the remaining playback time of the video through a background thread. When the remaining playback time reaches the preset time period, start the frame extraction operation of the next video in advance. Step 2: Extract the frames of the next video, decode these frames in parallel, and cache the decoded frames in the cache area; The high-speed cache is dynamically adjusted according to different video resolutions, frame rates, and encoding formats. The cache includes at least two independent cache modules: one for storing the frame data of the current video and the other for preloading and storing the decoded frames of the next video. The cache module of the next video is called first when switching videos. Step 3: When switching videos, the last frame of the current video is synchronized with the initial frame of the next video through timestamp synchronization, and the decoded frame is read from the buffer and played. The timestamp synchronization adopts a dual timestamp mechanism, which connects the logical timestamp and the actual playback timestamp of the last frame of the current video, and synchronizes the logical timestamp and the actual playback timestamp of the initial frame of the next video. The maximum error between the logical timestamp and the actual timestamp is set to no more than 10 milliseconds. Step 4: Use graphics APIs adapted to different platforms for video rendering, and improve rendering efficiency and smoothness of video switching through hardware acceleration; The method, applied to advertising insertion scenarios on streaming media platforms, includes: Step S1: Just before the main video is about to start the ad insertion, load at least the first segment of the ad video in advance through a background thread according to a predetermined time threshold, and complete the decoding and caching before insertion; Step S2: During the ad playback, the next segment of the main video is preloaded simultaneously via a background thread; Step S3: During ad playback, automatically adjust the ad playback order and duration based on user viewing history, ad skip rate, and user interaction data; Step S4: Dynamically adjust the resolution and bitrate of the ad video according to the current network status, prioritize playing low-resolution ad videos, and load high-resolution main video content in the background before the ad ends.

2. The optimized method for achieving zero-latency video switching according to claim 1, characterized in that, The background thread dynamically adjusts the extraction and decoding rate of the next video frame based on the remaining playback time of the video, network bandwidth status, and the load of the video decoder. Specifically, when the network fluctuates, it prioritizes the extraction of video frames with lower bit rates, and when the load is high, it reduces the number of parallel decoding operations of the decoder.

3. The optimized method for achieving zero-latency video switching according to claim 1, characterized in that, The dual timestamp mechanism uses the AKDS model, and the implementation steps are as follows: S1: Capture the logical timestamp of the last frame of the current video. and actual playback timestamp And the logical timestamp of the initial frame of the next video. and actual playback timestamp Calculate the timestamp difference for: ; S2: Use a Kalman filter to calculate the timestamp difference. Smoothing is performed using Kalman gain. The logical timestamp is updated using the following formula: ; S3: Dynamically adjust the learning rate based on the changing trend of timestamp error. Through function Calculate the learning rate and optimize the feedback response during the time synchronization process; S4: Predict timestamp drift using a Long Short-Term Memory (LSTM) network, inputting historical timestamp differences. Output the predicted timestamp drift for the next frame. This predicted value is then used to update the logical timestamp of the next frame. ; S5: Based on the objective of minimizing timestamp error, a reinforcement learning algorithm is used to dynamically adjust parameters such as the Kalman filter and adaptive control. The reward function is... By optimizing control parameters and synchronization accuracy, the latency and black screen phenomenon during video switching are reduced. S6: Combining Kalman filtering, adaptive control, deep learning prediction, and reinforcement learning dynamic adjustment strategies, the logical timestamp and actual playback timestamp of each frame during video switching are optimized to ensure that the error between the logical timestamp and the actual timestamp does not exceed 10 milliseconds.

4. The optimized method for achieving zero-latency video switching according to claim 1, characterized in that, The graphics APIs include OpenGL, DirectX, and Metal. Rendering modes are selected to adapt to different hardware platforms, and the optimal rendering path is automatically selected through a hardware characteristic detection mechanism. The detected hardware characteristics include processor type, graphics processing unit model, video memory size, and device temperature. By combining these characteristics, rendering priority and thread allocation are dynamically adjusted to ensure the stability and efficiency of the device under high load.

5. The optimized method for achieving zero-latency video switching according to claim 1, characterized in that, The video switching optimization process further includes the following steps: 1) Detect the processor architecture, graphics processing unit type, video memory capacity, and current device temperature of the device; 2) Select the optimal rendering path based on the detection results, and use multi-threaded parallel decoding technology on devices that support multi-core processing; 3) When the device temperature is too high, automatically reduce the rendering priority and reduce the number of parallel decoding threads to avoid overheating while maintaining basic video smoothness.

6. The optimized method for achieving zero-latency video switching according to claim 1, characterized in that, The method is applicable to online streaming media, real-time video playback, and ad insertion scenarios. When switching ad videos, it prioritizes the automatic selection of low-resolution ad frames for preloading based on network bandwidth. When playing the main video, it automatically adjusts to high resolution to ensure a good viewing experience. In real-time video playback, it optimizes the caching mechanism to reduce latency caused by network jitter.

7. The optimized method for achieving zero-latency video switching according to claim 1, characterized in that, The cached data is adaptively optimized according to video encoding formats including H.264, H.265, and AV1, and the decoding priority is adjusted according to different encoding formats.

Images