Cloud storage-based multi-screen television synchronous playing control method and system
By injecting UTC standard timestamps and synchronizing with the local clock in the cloud, the problem of inconsistent playback progress on multi-screen TVs was solved, achieving a synchronized effect where multiple screen TVs display the same video frames at the same time, thus improving display consistency and user experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN KONTECH ELECTRONICS CO LTD
- Filing Date
- 2025-12-19
- Publication Date
- 2026-04-24
AI Technical Summary
Existing multi-screen synchronization technologies lack a unified global time base, resulting in inconsistent progress when playing videos on different terminals, and cannot effectively cope with playback rhythm deviations caused by network fluctuations and differences in decoding speed.
By injecting an absolute playback timestamp based on UTC standard time into the PCR field of each GOP in the cloud, a unified time base is established across multiple regions and terminals. Local system clocks are synchronized via the SNTP protocol to ensure playback progress is aligned with the global timeline. To address frame buffering latency, a strategy of pausing and waiting or jumping to the nearest I-frame is employed to adjust playback progress. Playback status data is collected in real-time in the cloud for anomaly detection and resynchronization.
It achieves a synchronized effect where multiple TVs display the same video frames at the same absolute moment, improving display consistency and user experience, and effectively addressing playback rhythm deviations caused by network fluctuations and differences in decoding speed.
Smart Images

Figure CN121367805B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of multi-screen television technology, and in particular to a method and system for controlling synchronized playback of multi-screen televisions based on cloud storage. Background Technology
[0002] Multiple television terminals need to achieve precise synchronized playback of video content when geographically dispersed. Existing multi-screen synchronization technologies mainly rely on independent playback control for each terminal, lacking a unified global time reference. This leads to inconsistent playback progress when different terminals receive the same video stream due to factors such as network latency and differences in decoding speed.
[0003] In traditional solutions, terminals rely solely on their local clocks for playback control, unable to perceive the playback status of other terminals. Even if initial synchronization is achieved, accumulated errors gradually arise during extended playback due to clock drift and differences in buffering strategies. Furthermore, existing technologies lack dynamic compensation mechanisms for terminal frame buffering delays, failing to adaptively adjust playback rhythm when a terminal experiences buffering ahead or behind due to network fluctuations. Summary of the Invention
[0004] This invention provides a method and system for controlling synchronized playback of multi-screen TVs based on cloud storage. This invention achieves the synchronization effect of displaying the same video frames on multiple screen TVs at the same absolute time, thereby improving the display consistency and user experience of multi-screen TVs.
[0005] The first aspect of this invention provides a method for controlling synchronized playback of multi-screen TVs based on cloud storage, the method comprising:
[0006] The video file is sliced into multiple GOPs and pushed to various TV terminals, where the TV terminals parse to obtain the GOP encoded data and the first PCR timestamp.
[0007] The GOP encoded data is video decoded to generate YUV frame data, and the YUV frame data is written into the frame buffer queue according to the first PCR timestamp.
[0008] The television terminal obtains the second PCR timestamp, searches for the read pointer position corresponding to the second PCR timestamp in the frame buffer queue, and calculates the time deviation value.
[0009] When the time deviation value exceeds the lead threshold, the read pointer position remains unchanged and a timer is started to wait. When the time deviation value is lower than the lag threshold, the read pointer position is switched to the target I-frame position.
[0010] In conjunction with the first aspect, in a first implementation of the first aspect of the present invention, the video file is sliced into multiple GOPs and pushed to various television terminals. The television terminals parse to obtain GOP encoded data and a first PCR timestamp, including:
[0011] Extract the frame type flag bit of the NAL unit in the video file;
[0012] The UTC standard time is obtained from the time server via the NTP protocol, and the first PCR timestamp of each GOP is calculated based on the UTC standard time, GOP sequence number, GOP frame number and video frame rate.
[0013] The video file is divided into multiple GOPs according to the frame type flag, the PCR field is located by parsing the MPEG-TS transport stream header, and the first PCR timestamp is written into the PCR field to obtain the GOP video data packet.
[0014] The GOP video data packet is pushed to each television terminal, and the television terminal parses it to obtain the GOP encoded data and the first PCR timestamp.
[0015] In conjunction with the first aspect, in a second implementation of the first aspect of the present invention, the GOP video data packet is pushed to each television terminal, and the television terminal parses to obtain the GOP encoded data and the first PCR timestamp, including:
[0016] The GOP video data packet is uploaded to the CDN edge node and an HLS playlist file is generated. The TV terminal downloads the HLS playlist file and parses it to obtain the GOP download address.
[0017] The television terminal downloads the GOP video data packet from the CDN edge node according to the GOP download address and parses the PCR field of the MPEG-TS packet header;
[0018] The first PCR timestamp and GOP encoded data are obtained by reverse calculation based on the PCR field.
[0019] In conjunction with the first aspect, in a third implementation of the first aspect of the present invention, video decoding is performed on the GOP encoded data to generate YUV frame data, and the YUV frame data is written into the frame buffer queue according to the first PCR timestamp, including:
[0020] Based on the GOP encoded data, I-frames, P-frames, and B-frames are sequentially decoded into YUV frame data, and the standard PCR timestamp of each frame is calculated based on the first PCR timestamp, frame number, and video frame rate.
[0021] The YUV frame data and the standard PCR timestamp are encapsulated into frame elements, written to the write pointer position of the frame buffer queue in ascending order of the standard PCR timestamp, and the write pointer position is incremented.
[0022] In conjunction with the first aspect, in a fourth implementation of the first aspect of the present invention, the television terminal obtains the second PCR timestamp, searches for the read pointer position corresponding to the second PCR timestamp in the frame buffer queue, and calculates the time deviation value, including:
[0023] The television terminal sends a synchronization request to the time server via the SNTP protocol to obtain the UTC standard time, calculates the clock deviation between the local system clock and the UTC standard time, adjusts the local system clock, and obtains the adjusted local time.
[0024] The adjusted local time is converted into MPEG-TS clock frequency units to obtain the second PCR timestamp;
[0025] Traverse each frame element from the frame buffer queue, calculate the absolute difference between the standard PCR timestamp and the second PCR timestamp of each frame element, and take the frame element with the smallest absolute difference as the read pointer position.
[0026] The YUV frame data and the corresponding standard PCR timestamp are extracted from the frame element read from the read pointer position and used as the third PCR timestamp. The difference between the third PCR timestamp and the second PCR timestamp is calculated to obtain the time deviation value.
[0027] In conjunction with the first aspect, in the fifth implementation of the first aspect of the present invention, traversing each frame element from the frame buffer queue, calculating the absolute difference between the standard PCR timestamp and the second PCR timestamp of each frame element, and using the frame element position with the smallest absolute difference as the read pointer position, includes:
[0028] Starting from the frame element corresponding to the pointer position in the frame buffer queue, traverse the frame elements forward sequentially and extract the standard PCR timestamp of each frame element;
[0029] Calculate the absolute difference between the standard PCR timestamp and the second PCR timestamp for each frame element;
[0030] Find the absolute difference with the smallest value, take the position index of the frame element corresponding to the absolute difference in the frame buffer queue as the target frame position, and update the read pointer position to the target frame position.
[0031] In conjunction with the first aspect, in the sixth implementation of the first aspect of the present invention, when the time deviation value exceeds the lead threshold, the read pointer position is kept unchanged and a timer is started to wait; when the time deviation value is lower than the lag threshold, the read pointer position is jumped to the target I-frame position, including:
[0032] When the time deviation value is greater than the lead threshold, the read pointer position remains unchanged and a timer is started to wait;
[0033] When the time deviation value is less than the hysteresis threshold, the nearest target I-frame position is searched forward in the frame buffer queue, and the read pointer position is jumped to the target I-frame position.
[0034] In conjunction with the first aspect, in the seventh implementation of the first aspect of the present invention, when the time deviation value is less than the hysteresis threshold, searching forward in the frame buffer queue for the nearest target I-frame position and jumping from the read pointer position to the target I-frame position includes:
[0035] Starting from the read pointer position of the frame buffer queue, traverse each frame element backward and check whether the frame type flag of each frame element is an I-frame flag;
[0036] From the candidate frame elements marked as I-frames, select the frame element whose standard PCR timestamp is greater than or equal to the standard PCR timestamp of the read pointer position and whose position index is closest as the target I-frame position;
[0037] The read pointer position is jumped to the target I-frame position and all frame elements between the original read pointer position and the target I-frame position are discarded. The video decoder is then notified to clear the internal reference frame buffer.
[0038] In conjunction with the first aspect, in the eighth implementation of the first aspect of the present invention, the multi-screen TV synchronous playback control method based on cloud storage further includes:
[0039] Extract the YUV frame data to be displayed from the read pointer position and encapsulate the third PCR timestamp, the effective number of frames in the frame buffer queue and the compensation mode to generate a playback status data packet;
[0040] The cloud receives the playback status data packets reported by each TV terminal and calculates the first playback progress index of each TV terminal based on the third PCR timestamp and single frame duration of each TV terminal.
[0041] The second playback progress index is calculated based on the UTC standard time and the first PCR timestamp, and the deviation between the first playback progress index and the second playback progress index is calculated. The standard deviation of the deviation is then calculated.
[0042] Based on the deviation value and the standard deviation value, TV terminals with playback abnormalities are identified, and the target PCR value is calculated according to the second playback progress index, and the target GOP download address is generated.
[0043] The target PCR value and the target GOP download address are packaged into a resynchronization command and sent to the TV terminal experiencing playback issues.
[0044] A second aspect of the present invention provides a multi-screen TV synchronous playback control system based on cloud storage, the multi-screen TV synchronous playback control system based on cloud storage comprising:
[0045] The slicing module is used to slice the video file into multiple GOPs and push them to various TV terminals, wherein the TV terminals parse the GOP encoded data and the first PCR timestamp.
[0046] The video decoding module is used to perform video decoding on the GOP encoded data, generate YUV frame data, and write the YUV frame data into the frame buffer queue according to the first PCR timestamp.
[0047] The calculation module is used by the television terminal to obtain the second PCR timestamp, find the read pointer position corresponding to the second PCR timestamp in the frame buffer queue, and calculate the time deviation value.
[0048] The processing module is used to keep the read pointer position unchanged and start a timer to wait when the time deviation value exceeds the lead threshold, and to jump the read pointer position to the target I-frame position when the time deviation value is lower than the lag threshold.
[0049] Compared to existing technologies, this invention establishes a unified time base for multiple terminals across regions by injecting an absolute playback timestamp based on UTC standard time into the PCR field of each GOP in the cloud. This enables distributed TV terminals to control playback based on the same global clock, solving the synchronization drift problem caused by inconsistent time bases in traditional solutions. The TV terminals continuously synchronize their local system clocks via the SNTP protocol and calculate the target PCR timestamp corresponding to the current moment, accurately locating the frame to be displayed in the frame buffer queue to ensure playback progress is always aligned with the global timeline. To address frame buffer latency issues, a pause is executed when a terminal is ahead of its time to ensure strict timestamp alignment; when playback is severely lagging, the system jumps to the nearest I-frame to quickly catch up, effectively addressing playback rhythm deviations caused by network fluctuations and decoding speed differences. The cloud collects playback status data from each terminal in real time via WebSocket, uses a standard deviation statistical algorithm to detect terminals with abnormal deviations, and sends a forced resynchronization command containing the target PCR value and GOP download address. This achieves synchronization of multiple screens displaying the same video frames at the same absolute time, improving display consistency and user experience. Attached Figure Description
[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0051] The structures, proportions, sizes, etc., shown in the accompanying drawings of this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.
[0052] Figure 1 This is a flowchart illustrating the multi-screen TV synchronous playback control method based on cloud storage provided in an embodiment of the present invention.
[0053] Figure 2 This is a schematic diagram of the frame buffer queue structure provided in an embodiment of the present invention;
[0054] Figure 3 This is a schematic diagram of the multi-screen TV synchronous playback control architecture provided in an embodiment of the present invention;
[0055] Figure 4 This is a schematic block diagram of the structure of the multi-screen TV synchronous playback control system based on cloud storage provided in an embodiment of the present invention. Detailed Implementation
[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0057] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.
[0058] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0059] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items, and all possible combinations, and includes such combinations. See also Figure 1 One embodiment of the multi-screen TV synchronous playback control method based on cloud storage in this invention includes:
[0060] Step 100: Slice the video file into multiple GOPs and push them to each TV terminal. The TV terminal parses the GOP encoded data and the first PCR timestamp.
[0061] Step 200: Decode the GOP encoded data to generate YUV frame data, and write the YUV frame data into the frame buffer queue according to the first PCR timestamp.
[0062] Step 300: The TV terminal obtains the second PCR timestamp, finds the read pointer position corresponding to the second PCR timestamp in the frame buffer queue, and calculates the time deviation value.
[0063] Step 400: When the time deviation value exceeds the lead threshold, keep the read pointer position unchanged and start the timer to wait. When the time deviation value is lower than the lag threshold, jump the read pointer position to the target I-frame position.
[0064] In one specific embodiment, the process of performing step 100 may specifically include the following steps:
[0065] Extract the frame type flag from the NAL unit in the video file;
[0066] The system obtains UTC standard time from the time server via the NTP protocol and calculates the first PCR timestamp for each GOP based on the UTC standard time, GOP sequence number, GOP frame count, and video frame rate.
[0067] The video file is divided into multiple GOPs based on the frame type flag. The PCR field is located by parsing the MPEG-TS transport stream header. The first PCR timestamp is written into the PCR field to obtain the GOP video data packet.
[0068] The GOP video data packet is pushed to each TV terminal, and the TV terminal parses it to obtain the GOP encoded data and the first PCR timestamp.
[0069] Specifically, the server loads H.264 or H.265 encoded video files from the cloud storage system and sequentially reads the NAL unit data streams. During parsing, it extracts the first byte of the header for each NAL unit and separates the frame_type field using bitwise operations to determine the type of the current frame. When the frame_type field indicates an I-frame, it marks this frame as the starting boundary of a new GOP, records its position offset and frame sequence number information in the video byte stream, and builds a GOP index structure in memory. As the video stream continues to be parsed, the server gradually completes the boundary identification and frame count of multiple GOPs, assigning a GOP sequence number n to each GOP. The server then starts the time synchronization module, communicates with a remote time server based on the NTP protocol, obtains the current UTC standard timestamp, and calculates the calibrated time after network transmission delay compensation. Based on UTC time, the server calculates the absolute playback timestamp of each GOP, i.e., the first PCR timestamp, according to the formula: PCR_gop(n) = T_utc_corrected × 90000 + n × frames_per_gop × 90000 ÷ fps, where frames_per_gop is the number of video frames within the GOP, fps is the video frame rate, and 90000 is the system clock frequency unit specified by the MPEG-TS transport stream. The server slices the raw video data according to the identified GOP boundaries, with each GOP corresponding to a segment of video encoded stream data, which is encapsulated into standard MPEG-TS data packets. During each GOP encapsulation process, the server parses the TS header information, locates the PCR field, and decomposes the calculated first PCR timestamp into a high 33-bit base part and a low 15-bit extension part, writing them into the two structural positions defined in the PCR field. After writing, the CRC check field and the continuity counter field in the header are recalculated and updated to ensure that the TS data packet structure is valid and can be correctly recognized by the terminal decoder. After encapsulation, the server uploads the GOP video data packet with the PCR timestamp written to it to the CDN system and pushes it to multiple edge nodes worldwide through content distribution strategies for TV terminals to pull and use. Upon receiving the GOP video data packet, the TV terminal reads the transport stream header information and extracts the embedded PCR field. It then recovers the complete first PCR timestamp through standard format conversion, extracts the GOP encoded data as the compressed video stream to be decoded, and establishes a one-to-one mapping between the GOP encoded data and its corresponding absolute playback time.
[0070] In one specific embodiment, the process of pushing the GOP video data packet to each television terminal and the television terminal parsing to obtain the GOP encoded data and the first PCR timestamp can specifically include the following steps:
[0071] The GOP video data packet is uploaded to the CDN edge node and an HLS playlist file is generated. The TV terminal downloads the HLS playlist file and parses it to obtain the GOP download address.
[0072] The TV terminal downloads the GOP video data packet from the CDN edge node according to the GOP download address and parses the PCR field of the MPEG-TS packet header;
[0073] The first PCR timestamp and GOP encoded data are obtained by reverse calculation based on the PCR field.
[0074] Specifically, after completing the GOP slicing and the first PCR timestamp injection, the cloud media processing server sequentially uploads all timestamped GOP video data packets to the CDN's origin node. Subsequently, the CDN system distributes the GOP video data packets to edge nodes distributed across various locations based on the content delivery network's scheduling strategy, improving the speed and availability of data transmission based on proximity. Simultaneously with data upload, the cloud server generates an m3u8 format playlist file according to the HLS protocol specification. The playlist file contains the access URL address for each GOP video data packet, GOP duration information, and additional extended fields such as the PCR timestamp tag. The playlist file is stored in text format, with each GOP segment marked with an EXTINF tag indicating its duration, and the next line recording the specific URL link. This URL link points to the .ts format GOP data file cached on the CDN edge nodes. The TV terminal initiates a request to the CDN edge nodes via HTTP to obtain the m3u8 playlist file. After downloading and parsing, it obtains a complete list of URL addresses for the GOP segments. The terminal then initiates a new HTTP GET request based on the addresses in the list, requesting the corresponding GOP video data packet file and downloading and saving it. After downloading, the terminal parses the MPEG-TS format data packets, extracting the PCR field from the transport stream header. This field is encoded in a 48-bit structure, with the high 33 bits representing the base value and the low 15 bits representing the extension value. The terminal combines these two parts according to standard rules to form the complete PCR timestamp, i.e., the first PCR timestamp, using the formula PCR = base × 300 + extension. This allows the terminal to reconstruct the absolute playback time of the GOP. Simultaneously, the terminal extracts the GOP-encoded data stream from the TS packet, extracting and buffering it as the decoder's input data segment.
[0075] The process includes a GOP data packet integrity detection and retransmission control step after the TV terminal downloads the GOP video data packet from the CDN edge node according to the GOP download address. The steps are as follows: After receiving multiple consecutive GOP video data packets, the TV terminal extracts the first PCR timestamp of each GOP video data packet in the order of reception time. The extracted first PCR timestamps are arranged according to the GOP sequence number to form a PCR timestamp sequence. The actual increment value between adjacent first PCR timestamps is calculated by traversing the PCR timestamp sequence. The theoretical increment value is calculated based on the GOP frame number and video frame rate. The difference between the actual increment value and the theoretical increment value is used to obtain the increment deviation value. When the absolute value of the increment deviation value exceeds the fault tolerance threshold, it is determined that the GOP data packet is lost or the timestamp is abnormal. The GOP sequence number at the abnormal position is recorded and marked as a missing GOP sequence number. All missing GOP sequence numbers are counted to form a retransmission request list. A retransmission request message containing the retransmission request list and the terminal identifier is generated and sent to the CDN edge node. After receiving the retransmitted GOP video data packet, it inserts it into the correct position in the frame buffer queue according to the first PCR timestamp, ensuring the continuity and integrity of the PCR timestamp sequence.
[0076] In one specific embodiment, the process of performing step 200 may specifically include the following steps:
[0077] Based on the GOP encoded data, I-frames, P-frames, and B-frames are sequentially decoded into YUV frame data. The standard PCR timestamp for each frame is calculated based on the first PCR timestamp, frame number, and video frame rate.
[0078] YUV frame data and standard PCR timestamps are encapsulated into frame elements, written to the write pointer position of the frame buffer queue in ascending order of standard PCR timestamps, and the write pointer position is incremented.
[0079] Specifically, GOP encoded data is a sequence of video frames packaged according to the MPEG compression standard, containing one I-frame (keyframe), several P-frames (forward prediction frames), and multiple B-frames (bidirectional prediction frames). This type of GOP structure is reconstructed frame by frame sequentially by the decoder module. During the decoding stage, the decoder selects the corresponding decoding algorithm based on the GOP encoding format, such as H.264 or H.265, and maintains a reference frame buffer structure to satisfy the restoration of inter-frame prediction relationships, ensuring that each B-frame and P-frame can undergo inverse transformation and dequantization based on the correct reference frame, thereby reconstructing the image data. Each decoded frame yields the corresponding YUV frame data, represented in YUV420 format, where the Y component is used for grayscale information and the U / V components are used for color differences. Simultaneously, the terminal calculates the standard PCR timestamp based on the first PCR timestamp of the GOP, the frame sequence number of the current frame within the GOP, and the video frame rate. The calculation formula is: Standard PCR timestamp = First PCR timestamp + Current frame sequence number × 90000 ÷ fps, where 90000 is the clock frequency unit specified by the MPEG system, and fps is the video playback frame rate, such as 25 or 30 frames per second. The YUV image data of the frame and the calculated standard PCR timestamp are encapsulated together into a frame element structure. The frame element contains a frame type flag (used to identify whether the frame is an I-frame, P-frame, or B-frame) and the frame's index number in the GOP. The constructed frame element is written to a circular frame buffer queue. The queue is implemented using a fixed-capacity circular array structure. Frame elements are written to the current write pointer position in ascending order of the standard PCR timestamp. After writing, the write pointer is incremented modulo-based, i.e., the write pointer is updated to (write pointer + 1) mod the buffer capacity, completing the enqueue operation for one frame of data. The entire process continues until the entire GOP data is decoded and written to the buffer, forming a complete frame sequence data stream.
[0080] Figure 2 This is a schematic diagram of the frame buffer queue structure of the present invention. The queue uses a circular array structure to store the decoded YUV frame data. Each frame element contains pixel data in YUV420 format and a corresponding PCR timestamp identifier. The write pointer indicates the writing position of the YUV frame data after GOP decoding and writes it to the queue in ascending order of PCR timestamp. The read pointer indicates the reading position of the current frame to be displayed. Different frame types such as I-frames, P-frames, and B-frames are marked in the queue. I-frames are key frame boundaries of GOPs and can be decoded independently. The queue capacity is set to the number of GOP frames multiplied by the buffer depth (usually 4 GOPs), and the effective frame number N represents the number of frames that can be read in the current queue. The delay compensation strategy performs advanced pause and wait or delayed jump to I-frame operation based on the PCR timestamp deviation and monitors the buffer level to trigger bitrate adjustment, ensuring accurate alignment of timestamps during multi-screen TV playback.
[0081] The process includes encapsulating YUV frame data and standard PCR timestamps into frame elements and writing them into the frame buffer queue. The process also includes an adaptive control step for the frame buffer queue capacity: real-time monitoring of the write and read pointer positions of the frame buffer queue; calculating the number of frame elements between the two pointers as the current effective frame count; calculating the buffer utilization rate based on the ratio of the effective frame count to the total queue capacity; determining that the buffer is about to be full when the buffer utilization rate exceeds the upper limit threshold; calculating the average frame count of the most recently received N consecutive GOPs and the current network receiving rate; calculating the expansion increment frame count based on the average frame count multiplied by a preset expansion coefficient; dynamically allocating the memory space corresponding to the expansion increment frame count; linking the newly allocated memory space to the tail of the frame buffer queue to expand the queue capacity; updating the queue capacity parameters and adjusting the write pointer's loop boundary; determining that the buffer is redundant when the buffer utilization rate is below the lower limit threshold and the duration exceeds a set period; calculating the number of frame elements that can be released; releasing the corresponding memory space from the tail of the queue and updating the queue capacity parameters; and simultaneously checking the write pointer position to ensure it does not exceed the new queue boundary.
[0082] In this embodiment, when encapsulating YUV frame data and standard PCR timestamps into frame elements, the method further includes a PCR timestamp differential encoding and storage step: extracting the standard PCR timestamp of the first frame of each GOP as the GOP reference timestamp and storing it completely; calculating the difference between the standard PCR timestamps of subsequent frames within the same GOP and the GOP reference timestamp to obtain the frame-level timestamp offset; compressing the data type of the frame-level timestamp offset, compressing the original 8-byte complete timestamp into a 2-byte offset; and storing only the frame-level timestamp offset and the GOP reference timestamp in the frame element structure. The system employs a GOP (Group of Pictures) reference timestamp indexing method. A GOP reference timestamp mapping table is established in the frame buffer queue to store the mapping relationship between each GOP sequence number and its corresponding GOP reference timestamp. When reading a frame element, the reference timestamp is retrieved by looking up the table using the GOP reference timestamp index. The reference timestamp is then added to the frame-level timestamp offset to restore the complete standard PCR (Position of Component) timestamp. The GOP reference timestamp mapping table capacity is expanded synchronously as the frame buffer queue capacity increases. When old GOP data is overwritten, the corresponding GOP reference timestamp record is deleted from the mapping table. Differential coding reduces the memory footprint of the frame buffer queue and improves caching efficiency.
[0083] In this embodiment, when decoding I-frames, P-frames, and B-frames into YUV frame data sequentially according to GOP encoded data, a hardware-based decoding priority control step is also included: Upon startup, the TV terminal detects the local hardware decoder type and processing capability, obtains the number of CPU cores, GPU model, and available memory capacity, calculates the device decoding performance score, and classifies the terminal into three levels: high-performance, medium-performance, and low-performance. For high-performance terminals, all I-frames, P-frames, and B-frames in the GOP are fully decoded to generate all YUV frame data and written to the frame buffer queue, ensuring playback quality and smoothness. For medium-performance terminals, I-frames and P-frames are decoded first. When the number of effective frames in the frame buffer queue is lower than a safety threshold, the decoding of some B-frames is skipped, and the YUV frame data at the skipped B-frame position is set as a copy of the previous P-frame data, balancing the decoding load and playback continuity. For low-performance terminals, only I-frames and critical P-frames are decoded, skipping the decoding of non-critical P-frames and all B-frames. A frame repetition strategy is used to fill the YUV data of skipped frames, reducing the decoding computation load and avoiding playback stuttering due to decoding delay. The decoding priority strategy is dynamically adjusted according to the real-time buffer status and decoding load.
[0084] In one specific embodiment, the process of performing step 300 may specifically include the following steps:
[0085] The TV terminal sends a synchronization request to the time server via the SNTP protocol to obtain the UTC standard time, calculates the clock deviation between the local system clock and the UTC standard time, adjusts the local system clock, and obtains the adjusted local time.
[0086] The adjusted local time is converted to MPEG-TS clock frequency units to obtain the second PCR timestamp;
[0087] Traverse each frame element from the frame buffer queue, calculate the absolute difference between the standard PCR timestamp and the second PCR timestamp of each frame element, and take the frame element with the smallest absolute difference as the read pointer position.
[0088] The YUV frame data and the corresponding standard PCR timestamp are extracted from the frame element at the read pointer position and used as the third PCR timestamp. The time deviation value is obtained by subtracting the third PCR timestamp from the second PCR timestamp.
[0089] Specifically, the terminal activates its built-in SNTP (Simple Network Time Protocol) client module. Within a set synchronization period, it sends a time query request via UDP to the configured time server. The request includes the local originating time. Upon receiving the request, the server replies with the current UTC standard time and records the local receiving time when the terminal receives the response. Based on the basic compensation mechanism of the SNTP protocol, the terminal calculates the network latency as (local receiving time - local originating time) / 2 and adjusts the UTC time returned by the server accordingly to compensate for the one-way transmission delay. It then calculates the calibrated UTC time value and compares it with the current local system time to obtain the clock deviation. To avoid abrupt time jumps affecting video playback stability, the terminal employs a smooth time synchronization strategy, such as a phase-locked loop-based weighted adjustment method to slowly converge the local system time. The adjusted new local time is used as the current accurate playback time. The terminal multiplies the adjusted local time by a system clock frequency conversion factor of 90,000 to obtain the global playback timestamp of the theoretically playable frame, i.e., the second PCR timestamp. The terminal iterates through all filled frame elements in the frame buffer queue, starting from the current read pointer. For each frame, it extracts its encapsulated standard PCR timestamp and calculates the absolute difference between it and the target timestamp. The terminal selects the frame with the smallest absolute difference from all frame elements; its corresponding position is the position of the frame that should be played at the current moment. This frame's index position is used as the new read pointer position. From this position, the terminal reads the frame element and extracts its YUV image data and standard PCR timestamp. This timestamp serves as the third PCR timestamp, representing the time stamp of the currently played frame. The difference between the third PCR timestamp and the second PCR timestamp is calculated to obtain the time deviation value between the current played frame and the standard target frame.
[0090] In one specific embodiment, the process of traversing each frame element from the frame buffer queue, calculating the absolute difference between the standard PCR timestamp and the second PCR timestamp of each frame element, and using the frame element with the smallest absolute difference as the read pointer position can specifically include the following steps:
[0091] Starting from the frame element corresponding to the pointer position in the frame buffer queue, traverse the frame elements forward in sequence and extract the standard PCR timestamp of each frame element;
[0092] Calculate the absolute difference between the standard PCR timestamp and the second PCR timestamp for each frame element;
[0093] Find the smallest absolute difference, use the index of the frame element corresponding to the absolute difference in the frame buffer queue as the target frame position, and update the read pointer position to the target frame position.
[0094] Specifically, the TV terminal obtains the distribution of existing frame data in the current frame buffer queue and initiates the target frame retrieval process starting from the current read pointer position. During this process, the system traverses the entire frame buffer queue in reverse order, starting from the frame element pointed to by the read pointer. The traversal order is implemented using a modulo-length calculation method under a circular structure, i.e., by subtracting one from the current index and taking the modulo of the total queue capacity, ensuring that the traversal operation physically forms a closed loop. During the traversal, the terminal reads the pre-written standard PCR timestamp information from each frame element and compares it with the second PCR timestamp converted by the SNTP synchronization mechanism. By calculating the absolute difference between the two, i.e., Δ = |standard PCR timestamp - second PCR timestamp|, the distance between the frame and the current standard playback time on the timeline is evaluated. Throughout the traversal, a minimum difference variable and its corresponding frame index record are continuously maintained. When comparing the difference of each frame, if the current difference is found to be less than the previously recorded minimum difference, the minimum value and the corresponding frame's index position in the buffer queue are immediately updated to ensure that the frame element closest to the target time is always tracked. After the traversal is complete, the frame element index corresponding to the minimum absolute difference is set as the target frame position, and the frame buffer read pointer position read_index is updated to this index position, completing the relocation operation of the read pointer alignment.
[0095] In this embodiment, when the TV terminal downloads GOP video data packets from the CDN edge node according to the GOP download address, it also includes a GOP intelligent prefetching step based on PCR prediction: The TV terminal records the second PCR timestamp and the corresponding system timestamp for the most recent M moments, uses a linear regression algorithm to fit the PCR timestamp growth rate, and predicts the range of PCR timestamps that need to be played within the future time period T based on the growth rate; calculates the range of GOP sequence numbers needed in the future based on the PCR timestamp range and the first PCR timestamp of the GOP, queries the HLS playlist file to obtain the download address list of the corresponding GOP sequence number, and generates a GOP prefetch request queue; sends download requests to the CDN edge node in the order of GOP sequence number from the GOP prefetch request queue, caches the prefetched GOP video data packets in the local prefetch buffer, and prioritizes reading from the prefetch buffer when the frame buffer queue needs new GOP data; monitors the data volume of the prefetch buffer and the current network bandwidth, increases the number of concurrent prefetch requests when the data volume of the prefetch buffer is lower than the safety threshold, and suspends the prefetch operation when the network bandwidth is insufficient to avoid affecting the download of the GOP that is currently playing, dynamically balancing prefetch efficiency and network resource consumption.
[0096] In one specific embodiment, the process of performing step 400 may specifically include the following steps:
[0097] When the time deviation value is greater than the lead threshold, the read pointer position remains unchanged and a timer is started to wait;
[0098] When the time deviation value is less than the hysteresis threshold, the nearest target I-frame position is searched forward in the frame buffer queue, and the read pointer position is jumped to the target I-frame position.
[0099] Specifically, when the difference between the standard PCR timestamp of the frame currently pointed to by the read pointer in the frame buffer queue and the second PCR timestamp corresponding to the current global playback time is greater than the set advance threshold (e.g., 90,000, or 1 second), it indicates that the playback progress is too fast relative to the global time, and a delay compensation operation is performed. The terminal keeps the current read pointer position unchanged, pauses the output of the next frame data to the display module, and converts the difference by 90,000 to the number of seconds to obtain the precise length of time to wait, and sets the timer module accordingly. During the waiting period, the timer periodically triggers a fixed frame refresh command, causing the current frame YUV image to be continuously displayed repeatedly, keeping the screen still, until the timer expires and the normal movement of the read pointer resumes, so as to ensure that the screen playback is aligned with the timeline. When the time deviation is less than the lag threshold (e.g., -180000, or -2 seconds), it indicates that the terminal's playback progress is behind the target timeline. A frame skipping compensation strategy is then implemented. Starting from the target playback position in the current buffer queue, a reverse traversal is performed, checking the frame type identifier field of each frame element. When the first frame element with an I-frame identifier is detected and its standard PCR timestamp is greater than or equal to the target timestamp, the index position of that frame element in the buffer is set to the new target I-frame position, and the read pointer position `read_index` is updated to this position, completing the repositioning of the playback starting point. Simultaneously, the decoding module is notified to reset its internal reference frame buffer, clearing any remaining P-frame or B-frame state information, and reconstructing the decoding reference structure with the target I-frame as the new starting point, ensuring that the decoding process starting from this point has the correct reference chain. After completing the read pointer jump and decoding state reconstruction, the terminal resumes normal playback, continuing decoding and output from the new target I-frame, thereby enabling the playback progress to quickly catch up and return to the standard playback rhythm.
[0100] In this embodiment, after performing compensation operations when the time deviation value is greater than the lead threshold or less than the lag threshold, the system further includes a threshold adaptive adjustment step based on the compensation frequency: The television terminal records the time and corresponding time deviation value of each execution of delay waiting compensation and jump catch-up compensation, and counts the number of triggers of delay waiting compensation and jump catch-up compensation within a unit time window to obtain the compensation operation frequency; when the number of triggers of delay waiting compensation exceeds the upper frequency threshold, it is determined that the lead threshold is set too low, the lead threshold is multiplied by an amplification factor and the lead threshold parameter is updated to expand the tolerance range of lead compensation and reduce the compensation trigger frequency; when the number of triggers of jump catch-up compensation exceeds the upper frequency threshold, it is determined that the lag threshold is set too high, the lag threshold is multiplied by a reduction factor and the lag threshold parameter is updated to trigger catch-up compensation in advance to avoid serious lag in playback progress; when the number of triggers of delay waiting compensation and jump catch-up compensation are both lower than the lower frequency threshold and the duration exceeds the stable period, it is determined that the playback state is stable, and the lead threshold and lag threshold are gradually restored to their initial default values to achieve adaptive dynamic adjustment of the threshold parameters.
[0101] In one specific embodiment, the process of performing the step of searching forward for the nearest target I-frame position in the frame buffer queue and jumping to the read pointer position when the time deviation value is less than the hysteresis threshold can specifically include the following steps:
[0102] Starting from the read pointer position of the frame buffer queue, traverse each frame element backward and check whether the frame type flag of each frame element is an I-frame flag;
[0103] From the candidate frame elements marked as I-frames, select the frame element whose standard PCR timestamp is greater than or equal to the standard PCR timestamp of the read pointer position and whose position index is closest as the target I-frame position;
[0104] Jump the read pointer position to the target I-frame position and discard all frame elements between the original read pointer position and the target I-frame position, and notify the video decoder to clear the internal reference frame buffer.
[0105] Specifically, the TV terminal performs a frame type check and keyframe alignment process in the playback control module. Starting from the frame element pointed to by the current read pointer `read_index`, a forward traversal operation is initiated. A reverse scan is performed using an index-minus-1 modulo operation under a circular buffer structure, and each frame element is accessed one by one to extract its frame type flag information. For each traversed frame element, it is determined whether its frame type field is equal to the I-frame flag value. If not, the current frame is skipped and the search continues forward. If the frame type is an I-frame, it is recorded as a candidate keyframe, and its standard PCR timestamp is read simultaneously. All candidate I-frame sets are further filtered, retaining frame elements whose standard PCR timestamp is greater than or equal to the standard PCR timestamp of the frame pointed to by the current read pointer. The frame element with the position index closest to the current read pointer position is selected and set as the target I-frame position. That is, the target frame is closest to the current playback target on the timeline and has decoding self-starting capability. After determining the target I-frame position, the read pointer `read_index` is updated to the index position of that frame in the buffer queue. Simultaneously, all intermediate frames previously located between the old read pointer and the target I-frame position are marked as invalid and discarded. These discarded frames are P-frames and B-frames, which cannot be decoded independently due to a broken reference chain and therefore cannot be used. To avoid image errors caused by residual erroneous frames in the decoder's buffer, a control command is simultaneously issued to the video decoding module during the jump process, requesting it to immediately clear its internal reference frame buffer and reset the decoding state machine.
[0106] In this embodiment, before traversing each frame element backward from the read pointer position of the frame buffer queue to check the I-frame marker, the method includes an I-frame position pre-index table establishment and fast positioning step: when encapsulating YUV frame data and standard PCR timestamps into frame elements and writing them into the frame buffer queue, the frame type flag of each frame element is detected synchronously. When an I-frame marker is detected, the position index of the frame element in the queue and the corresponding standard PCR timestamp are recorded to obtain the I-frame position record; the I-frame position record is inserted into the I-frame pre-index table in ascending order of the standard PCR timestamp, and the I-frame pre-index table contains the I-frame position index. The system includes fields for the standard PCR timestamp and GOP sequence number. When the write pointer of the frame buffer queue overwrites old data, the corresponding expired I-frame position record is deleted synchronously. When it is necessary to jump to the target I-frame position, the system performs a binary search in the I-frame pre-index table to find the first I-frame record whose standard PCR timestamp is greater than or equal to the standard PCR timestamp of the read pointer position. The I-frame position index field of this record is directly read as the target I-frame position. The read pointer position is then jumped to the target I-frame position found in the I-frame pre-index table, skipping the traversal search process, reducing the time complexity of the jump operation and reducing jump latency.
[0107] In one specific embodiment, the multi-screen TV synchronous playback control method based on cloud storage further includes the following steps:
[0108] Extract the YUV frame data to be displayed from the read pointer position and encapsulate the third PCR timestamp, the effective number of frames in the frame buffer queue and the compensation mode to generate a playback status data packet;
[0109] The cloud receives playback status data packets reported by each TV terminal and calculates the first playback progress index of each TV terminal based on the third PCR timestamp and single frame duration of each TV terminal.
[0110] The second playback progress index is calculated based on the UTC standard time and the first PCR timestamp, and the deviation between the first playback progress index and the second playback progress index is calculated. The standard deviation of the deviation is then calculated.
[0111] Based on the deviation value and standard deviation value, TV terminals with playback abnormalities are identified, and the target PCR value is calculated according to the second playback progress index to generate the target GOP download address;
[0112] The target PCR value and target GOP download address are packaged into a resynchronization command and sent to the TV terminal experiencing playback issues.
[0113] Specifically, the TV terminal extracts the YUV image data from the frame element pointed to by the current read pointer `read_index` in real time within the playback control module, and encapsulates its corresponding standard PCR timestamp into a third PCR timestamp. This third timestamp is then combined with the number of valid frames that have not yet been read in the current frame buffer queue and the compensation mode used for current playback (such as normal playback, delayed waiting, frame skipping, etc.) to form a playback status data packet. This playback status data packet is reported to the cloud synchronization control server via a WebSocket connection or HTTP POST. After receiving data from each terminal, the server sequentially extracts the third PCR timestamp reported by each terminal and calculates the single-frame duration `frame_duration = 90000 ÷ fps` based on the video frame rate. The third PCR timestamp is then divided by `frame_duration` to obtain the terminal's current first playback progress index. Simultaneously, the cloud server obtains the current UTC standard time `T_utc` through the NTP mechanism and, combined with the first PCR timestamp at the start of video playback, calculates the current standard playback position, i.e., the second playback progress index. The calculation formula is: (T_utc × 90000 - first PCR timestamp) ÷ `frame_duration`. The server calculates the difference between the first and second playback progress indices for each terminal, i.e., the playback deviation value, and calculates the mean and standard deviation of all deviation values to quantify the overall synchronization accuracy. If the absolute value of the playback deviation value of a terminal exceeds three times the standard deviation, the terminal is determined to have deviated from the main playback rhythm and is marked as a playback abnormal terminal. Based on its current second playback progress index, the server reverse-engineers the target timestamp of the PCR that should be played at that moment, and queries the GOP timeline index table to determine the GOP number containing the target timestamp of the PCR, and generates the target GOP download address using the GOP number. The server encapsulates the target PCR timestamp and the corresponding GOP download address together into a resynchronization control command and sends it to the TV terminal with playback abnormality via a WebSocket connection.
[0114] In this embodiment, after the cloud receives the playback status data packets reported by each TV terminal, it further includes a grouped bitrate push step based on playback progress variance: the cloud extracts the first playback progress index and the second playback progress index of each TV terminal, calculates the playback progress deviation value of each TV terminal, calculates the mean and variance of all deviation values, and calculates the playback progress dispersion based on the variance; the terminals are grouped according to the relationship between the playback progress deviation value and the mean of each TV terminal, and terminals with deviation values greater than the mean plus one standard deviation are classified as advanced terminals, terminals with deviation values less than the mean minus one standard deviation are classified as lagging terminals, and the remaining terminals are classified as advanced terminals. The system divides the terminals into normal terminal groups; for TV terminals in the advanced terminal group, low bitrate GOP video data packets are selected and pushed via CDN to reduce buffer filling speed; for TV terminals in the lagging terminal group, high bitrate GOP video data packets are selected and pushed via CDN to speed up playback catch-up; and for the normal terminal group, standard bitrate GOP video data packets are pushed. The system periodically recalculates the playback progress deviation value of each TV terminal and updates the terminal grouping, dynamically adjusting the bitrate push strategy of each group of terminals. When the playback progress dispersion converges to below a set threshold, unified bitrate push is restored, achieving coordinated convergence control of the playback progress of each terminal.
[0115] The multi-screen TV synchronous playback control method based on cloud storage in the embodiments of the present invention has been described above. The multi-screen TV synchronous playback control system based on cloud storage in the embodiments of the present invention will be described below. Figure 3 This is a schematic diagram of the multi-screen TV synchronous playback control architecture of the present invention. The cloud service layer includes a time server providing a UTC standard time base, a media processing server responsible for GOP slicing and PCR timestamp injection, and a synchronization control server monitoring the playback status of each terminal via WebSocket. Multiple edge nodes of the CDN content delivery network push GOP data packets with PCR timestamps to multiple geographically dispersed TV terminals (terminals A, B, C to terminal N). Each TV terminal synchronizes its local system clock with the time server via the SNTP protocol, internally executing GOP decoding, frame buffer queue storage, clock synchronization, latency compensation, and display output processes in sequence. The terminals report playback status data packets containing PCR timestamps and buffer status in real time. The cloud synchronization control server calculates the playback progress deviation of each terminal and issues resynchronization commands to terminals with abnormal deviations (such as terminal C), achieving precise multi-screen synchronous playback control.
[0116] Please see Figure 4 One embodiment of the multi-screen TV synchronous playback control system based on cloud storage in this invention includes:
[0117] The slicing module 401 is used to slice the video file into multiple GOPs and push them to various TV terminals. The TV terminals parse the GOP encoded data and the first PCR timestamp.
[0118] The video decoding module 402 is used to perform video decoding on the GOP encoded data, generate YUV frame data, and write the YUV frame data into the frame buffer queue according to the first PCR timestamp.
[0119] The calculation module 403 is used for the TV terminal to obtain the second PCR timestamp, find the read pointer position corresponding to the second PCR timestamp in the frame buffer queue, and calculate the time deviation value.
[0120] The processing module 404 is used to keep the read pointer position unchanged and start a timer to wait when the time deviation value exceeds the lead threshold, and to jump the read pointer position to the target I-frame position when the time deviation value is lower than the lag threshold.
[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0122] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0123] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for controlling synchronized playback of multi-screen televisions based on cloud storage, characterized in that, include: The video file is sliced into multiple GOPs and pushed to various TV terminals, where the TV terminals parse to obtain the GOP encoded data and the first PCR timestamp. The GOP encoded data is video decoded to generate YUV frame data, and the YUV frame data is written into the frame buffer queue according to the first PCR timestamp. The television terminal obtains the second PCR timestamp, searches for the read pointer position corresponding to the second PCR timestamp in the frame buffer queue, and calculates the time deviation value. Specifically, the television terminal sends a synchronization request to the time server via the SNTP protocol to obtain the UTC standard time, calculates the clock deviation between the local system clock and the UTC standard time, adjusts the local system clock, and obtains the adjusted local time; converts the adjusted local time into MPEG-TS clock frequency units to obtain the second PCR timestamp; starting from the frame element corresponding to the read pointer position in the frame buffer queue, it traverses the frame elements sequentially and extracts the standard PCR timestamp of each frame element; calculates the absolute difference between the standard PCR timestamp of each frame element and the second PCR timestamp; finds the smallest absolute difference, uses the position index of the frame element corresponding to the absolute difference in the frame buffer queue as the target frame position, and updates the read pointer position to the target frame position; reads the frame element from the read pointer position, extracts YUV frame data and the corresponding standard PCR timestamp as the third PCR timestamp, and calculates the difference between the third PCR timestamp and the second PCR timestamp to obtain the time deviation value. When the time deviation value exceeds the lead threshold, the read pointer position remains unchanged and a timer is started to wait. When the time deviation value is less than the lag threshold, each frame element is traversed backward from the read pointer position in the frame buffer queue. The frame type flag of each frame element is checked to see if it is an I-frame. From the candidate frame elements marked as I-frames, the frame element whose standard PCR timestamp is greater than or equal to the standard PCR timestamp of the read pointer position and whose position index is closest to the read pointer position is selected as the target I-frame position, and the read pointer position is jumped to the target I-frame position.
2. The multi-screen TV synchronous playback control method based on cloud storage according to claim 1, characterized in that, The video file is sliced into multiple Groups of Pictures (GOPs) and pushed to various television terminals. The television terminals parse the GOP encoded data and the first PCR timestamp, including: Extract the frame type flag bit of the NAL unit in the video file; The UTC standard time is obtained from the time server via the SNTP protocol, and the first PCR timestamp of each GOP is calculated based on the UTC standard time, the GOP sequence number obtained by assigning a value to each GOP number, the number of video frames in the GOP, and the video frame rate. The video file is divided into multiple GOPs according to the frame type flag, the PCR field is located by parsing the MPEG-TS transport stream header, and the first PCR timestamp is written into the PCR field to obtain the GOP video data packet. The GOP video data packet is pushed to each television terminal, and the television terminal parses it to obtain the GOP encoded data and the first PCR timestamp.
3. The multi-screen TV synchronous playback control method based on cloud storage according to claim 2, characterized in that, The GOP video data packet is pushed to each television terminal, and the television terminal parses it to obtain the GOP encoded data and the first PCR timestamp, including: The GOP video data packet is uploaded to the CDN edge node and an HLS playlist file is generated. The TV terminal downloads the HLS playlist file and parses it to obtain the GOP download address. The television terminal downloads the GOP video data packet from the CDN edge node according to the GOP download address and parses the PCR field of the MPEG-TS packet header; The first PCR timestamp and GOP encoded data are obtained by reverse calculation based on the PCR field.
4. The multi-screen TV synchronous playback control method based on cloud storage according to claim 1, characterized in that, The process includes video decoding of the GOP-encoded data to generate YUV frame data, and writing the YUV frame data into the frame buffer queue according to the first PCR timestamp, including: Based on the GOP encoded data, I-frames, P-frames, and B-frames are sequentially decoded into YUV frame data. The standard PCR timestamp of each frame is calculated based on the first PCR timestamp, the frame sequence number of the current frame in the GOP, and the video frame rate. The YUV frame data and the standard PCR timestamp are encapsulated into frame elements, written to the write pointer position of the frame buffer queue in ascending order of the standard PCR timestamp, and the write pointer position is incremented.
5. The multi-screen TV synchronous playback control method based on cloud storage according to claim 1, characterized in that, The multi-screen TV synchronous playback control method based on cloud storage also includes: Extract the YUV frame data to be displayed from the read pointer position and encapsulate the third PCR timestamp, the effective number of frames in the frame buffer queue and the compensation mode to generate a playback status data packet. The compensation mode includes normal playback, delayed waiting or frame skipping catch-up. The cloud receives the playback status data packets reported by each TV terminal and calculates the first playback progress index of each TV terminal based on the third PCR timestamp and single frame duration of each TV terminal. The second playback progress index is calculated based on the UTC standard time and the first PCR timestamp, and the deviation between the first playback progress index and the second playback progress index is calculated. The standard deviation of the deviation is then calculated. Based on the deviation value and the standard deviation value, TV terminals with playback abnormalities are identified, and the target PCR value is calculated according to the second playback progress index, and the target GOP download address is generated. The target PCR value and the target GOP download address are packaged into a resynchronization command and sent to the TV terminal experiencing playback issues.
6. A multi-screen television synchronous playback control system based on cloud storage, characterized in that, The method for controlling synchronized playback of multi-screen TVs based on cloud storage as described in any one of claims 1-5 includes: The slicing module is used to slice the video file into multiple GOPs and push them to various TV terminals, wherein the TV terminals parse the GOP encoded data and the first PCR timestamp. The video decoding module is used to perform video decoding on the GOP encoded data, generate YUV frame data, and write the YUV frame data into the frame buffer queue according to the first PCR timestamp. The calculation module is used by the television terminal to obtain the second PCR timestamp, find the read pointer position corresponding to the second PCR timestamp in the frame buffer queue, and calculate the time deviation value. The processing module is configured to, when the time deviation value exceeds the lead threshold, keep the read pointer position unchanged and start a timer to wait; when the time deviation value is less than the lag threshold, traverse each frame element backward from the read pointer position in the frame buffer queue, check whether the frame type flag of each frame element is an I-frame marker; select the frame element whose standard PCR timestamp is greater than or equal to the standard PCR timestamp of the read pointer position and whose position index is closest to the read pointer position as the target I-frame position from the candidate frame elements marked as I-frames, and jump the read pointer position to the target I-frame position.
Citation Information
Patent Citations
Multi-screen video synchronous playing method and device
CN104168466A
A method, system, device and medium for real-time video streaming playback with reduced latency
CN119767050A