A video playback method and system based on websocket-ts

Abstract

The present invention relates to the technical field of streaming media video playback, and in particular to a WebSocket-TS-based video playback method and system, comprising the following steps: defining a multiplexing format for signaling and video data, designing frame header information, implementing a frame encapsulation mechanism, setting a priority control mechanism, and implementing a flow control algorithm. The WebSocket-TS-based video playback method and system proposed by the present invention transmit video data via the WebSocket protocol, thereby reducing the request / response round-trip time in the HTTP protocol and improving the efficiency of data transmission. The WebSocket protocol is used to support full-duplex communication, allowing a server to push video data and respond to user signaling in real time, thereby enhancing the real-time performance of video playback.

Description

Technical Field

[0001] The present invention relates to the technical field of streaming media video playback, and in particular to a websocket-ts-based video playback method and system. Background Art

[0002] Common solutions for video playback:

[0003] (1) HLS-based playback protocol. HLS-based playback has the advantage of good compatibility and is suitable for use in standard browser and mobile phone playback environments. However, some players do not support playback of videos with specific encodings, such as HEVC. This makes it suitable for use in situations where encoding type requirements are relaxed. Furthermore, the transmission efficiency and real-time signaling response are not high.

[0004] (2) Limited video playback control functions: Existing HLS or RTSP-based video transmission solutions often only support basic play and pause functions, and lack support for advanced functions such as double-speed playback and video jump, and the implementation method is relatively complex.

[0005] (3) Scalability limitations: With the diversification of video content and the personalization of user needs, the HTTP protocol has scalability limitations in supporting complex signaling control and advanced functions.

[0006] (4) Bandwidth waste: Under the HTTP protocol, video data is usually transmitted in blocks, which may lead to unnecessary repeated requests and data transmission, thereby wasting bandwidth resources. Summary of the Invention

[0007] The purpose of the present invention is to provide a websocket-ts based video playback method and system to solve the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solution: a video playback method based on websocket-ts, the method comprising the following steps:

[0009] Define the multiplexing format of signaling and video data, which are transmitted through the WebSocket channel. The signaling contains the control command type, timestamp and target location information, and the video data is encapsulated in TS stream format.

[0010] Design frame header information, including identifiers for distinguishing signaling frames from video data frames, as well as type identifiers, length information, and sequence numbers to ensure correct parsing and reassembly of data;

[0011] Implement a frame encapsulation mechanism to encapsulate signaling and video data into frames with frame headers and send them through the same WebSocket channel;

[0012] Set up a priority control mechanism to define priorities for different types of frames and manage received frames through priority queues to ensure that signaling frames are processed before video data frames;

[0013] Implement a flow control algorithm to dynamically adjust the transmission rate of video data based on network conditions and the processing capabilities of the receiving end to avoid network congestion and data loss.

[0014] Preferably, the design of heartbeat and timeout mechanisms is also included, which detects and maintains the activity of the WebSocket connection by sending heartbeat signaling at regular intervals, and designs a timeout mechanism to process signaling or data frames that are not confirmed within a predetermined time, thereby ensuring the stability of the connection and the integrity of the data.

[0015] Preferably, an error handling and recovery mechanism is also included, which detects errors in the transmission process through error detection mechanisms such as CRC check and implements error recovery strategies such as retransmission to ensure the integrity and reliability of signaling and video data.

[0016] Preferably, a synchronization mechanism between the client and the server is also included, which ensures that the processing logic of the signaling and video data of both parties is consistent through synchronization information such as timestamps, thereby maintaining consistency in the video playback progress.

[0017] Preferably, security considerations are also taken into account, and the WebSocket Secure protocol is used to encrypt WebSocket communications through TLS / SSL, and the signaling and video data are encrypted or signed to prevent man-in-the-middle attacks and data tampering, thereby ensuring the security of data transmission.

[0018] A websocket-ts-based video playback system is applied to a websocket-ts-based video playback method, the system comprising:

[0019] A WebSocket server, used to receive connection requests from clients and establish persistent connections;

[0020] A client that establishes a connection with the WebSocket server and sends control signals and receives video data;

[0021] A multiplexing mechanism that allows simultaneous transmission of control signaling and video data over a WebSocket channel. It distinguishes signaling frames from video data frames by defining frame formats and header information, and ensures that signaling frames are prioritized.

[0022] The frame encapsulation module is used to encapsulate control signaling and video data into frames with type identification, length information and sequence number, and send them through WebSocket;

[0023] The priority control module is used to ensure that the control signaling frames are processed before the video data frames.

[0024] Preferably, the WebSocket server further includes:

[0025] Signaling parsing module, used to parse the control signaling from the client and perform corresponding operations based on the signaling content;

[0026] The video data cache and flow control module is used to cache video data and adjust the transmission of video data according to the signaling content to ensure the timely transmission of key frames and respond to high-priority signaling.

[0027] Preferably, the system further comprises:

[0028] A real-time optimization module is used to reduce delays in video data transmission and signaling processing, improving response speed. In particular, signaling uses binary processing to improve transmission efficiency and bandwidth utilization.

[0029] The adaptive bitrate control module allows the player to dynamically adjust the required video bitrate based on the current playback effect. The server supports video data transmission at multiple bitrates.

[0030] Preferably, the system further comprises:

[0031] The security module is used to encrypt WebSocket communications and ensure the security of data transmission through the WSS protocol and TLS / SSL. At the same time, it encrypts or signs signaling and video data to prevent man-in-the-middle attacks and data tampering.

[0032] The error detection and recovery module is used to detect errors during transmission, such as CRC check, and implement error recovery strategies, such as retransmission mechanism, to ensure the integrity and reliability of data transmission.

[0033] Preferably, the system further comprises:

[0034] The synchronization module is used to achieve synchronization between the client and the server, ensuring that both sides have consistent processing logic for signaling and video data; synchronizing video playback progress through timestamps to ensure the coherence and consistency of video playback;

[0035] The heartbeat and timeout mechanism module is used to periodically send heartbeat signals to detect and maintain the activity of the WebSocket connection, and take corresponding measures, such as resending or disconnecting, when the signaling or data frame is not confirmed within a predetermined time.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The websocket-ts-based video playback method and system proposed in the present invention transmit video data through the WebSocket protocol, reducing the request / response round-trip time in the HTTP protocol, thereby improving the efficiency of data transmission; using WebSocket to support full-duplex communication, allowing the server to push video data and respond to user signaling in real time, thereby enhancing the real-time performance of video playback; reducing bandwidth consumption, utilizing the efficient encoding and transmission of TS stream containers, reducing unnecessary data transmission, and optimizing bandwidth usage; supporting multiple control functions such as video playback, pause, double-speed playback, and jump, allowing users to flexibly control video playback according to their needs, thereby improving the viewing experience; and by multiplexing signaling and video data on the same WebSocket channel, reducing the number of connections required by the system and simplifying the system architecture. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Flow chart of the method of the present invention;

[0039] Figure 2 A schematic diagram of signaling and data formats is defined for the present invention;

[0040] Figure 3 A schematic diagram of frame header information is designed for the present invention. DETAILED DESCRIPTION

[0041] In order to clearly and completely describe the objectives and technical solutions of the present invention and make the advantages more clearly understood, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, not all of them, and are only used to explain the embodiments of the present invention, not to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] For example 1, please refer to Figures 1 to 3 The present invention provides a technical solution: a video playback method based on websocket-ts, the method comprising the following steps:

[0043] Step 1: Multiplexing signaling and video data

[0044] Designing a mechanism for multiplexing signaling and video data over the same WebSocket channel involves efficiently transmitting both control signaling and video data over the same communication channel. The following are the detailed steps to implement this mechanism:

[0045] 1. Define signaling and data formats:

[0046] - Define a clear format for signaling, including information such as signaling type (such as play, pause, speed, jump, etc.), timestamp, target position, etc.

[0047] -Define a format for video data. Usually, video data is transmitted in TS stream format. Each TS packet contains a timestamp and payload data.

[0048] 2 Design frame header information:

[0049] -Define a frame header for each frame sent via WebSocket. The frame header contains an identifier to distinguish signaling frames from video data frames.

[0050] -The frame header contains type identification, length information, sequence number, etc., so that the receiving end can correctly parse and reassemble the data.

[0051] 3. Implement frame encapsulation mechanism:

[0052] -When the client or server needs to send signaling, it encapsulates the signaling information into a frame, adds a frame header, and sends it through WebSocket.

[0053] -When the server needs to send video data, it encapsulates the TS stream data into frames, adds a frame header, and sends it through WebSocket.

[0054] 4 Design priority control:

[0055] - Define priorities for different types of frames to ensure that control signaling frames can be processed before video data frames.

[0056] - Implement a priority queue to sort received frames to ensure timely response to signaling.

[0057] 5. Implement flow control:

[0058] -Dynamically adjust the transmission rate of video data according to network conditions and the processing capabilities of the receiving end to avoid congestion and packet loss.

[0059] - Implement flow control algorithms, such as sliding window protocols, to manage the sending and receiving of data.

[0060] 6 Design heartbeat and timeout mechanism:

[0061] -Send heartbeat signals regularly to detect and maintain the activity of the WebSocket connection.

[0062] -Design a timeout mechanism to resend or take appropriate measures when the signaling or data frame is not confirmed within the predetermined time.

[0063] 7 Implement error handling and recovery:

[0064] -Design error detection mechanisms, such as CRC check, to detect errors during transmission.

[0065] - Implement error recovery strategies, such as retransmission mechanisms, to ensure the integrity of signaling and video data.

[0066] 8 Synchronization between client and server:

[0067] -The client and server need to synchronize the signaling and video data processing logic to ensure that both sides parse and process the frames in the same way.

[0068] -Implement synchronization mechanisms, such as synchronizing video playback progress through timestamps to ensure consistency in video playback.

[0069] 9. Security considerations:

[0070] -Ensure the security of WebSocket communication by using the wss (WebSocket Secure) protocol and encrypting data transmission via TLS / SSL.

[0071] -Encrypt or sign signaling and video data to prevent man-in-the-middle attacks and data tampering.

[0072] Through the above steps, an efficient and reliable signaling and video data multiplexing mechanism can be implemented on the same WebSocket channel, thereby improving the performance of the video streaming service and user experience.

[0073] Step 2: The server and client implement the corresponding signaling control protocol

[0074] 1. Develop a WebSocket server that can receive connection requests from clients and establish persistent connections. The server itself must define methods that can parse client signaling and ensure that signaling has the highest priority and can be processed first.

[0075] 2. Implement video data caching and flow control mechanisms on the server side to ensure that video data can be transmitted on demand.

[0076] The server caches a key frame sequence GOP to ensure that the client can output the image in a timely manner. In terms of the flow control mechanism, it can decide which video data can be directly discarded based on the content of the signal itself, so as to respond to higher priority signaling.

[0077] Step 3: Real-time and efficiency optimization methods

[0078] 1. Optimize video data transmission and signaling processing to reduce delays and improve response speed.

[0079] Signaling uses binary processing, which can maximize the use of transmission bandwidth, avoid deserialization operations of text data, and be more efficient.

[0080] 2. Adaptive Bitrate (ABR) is controlled by signaling. The player can dynamically adjust the required bitrate based on the current playback effect. The server needs to support videos with different bitrates.

[0081] 3) Beneficial effects of the invention:

[0082] The video playback method based on WebSocket and TS stream has the following benefits:

[0083] 1. Improve transmission efficiency:

[0084] -Video data is transmitted through the WebSocket protocol, which reduces the request / response round-trip time in the HTTP protocol and thus improves the efficiency of data transmission.

[0085] 2. Enhance real-time performance:

[0086] -WebSocket supports full-duplex communication, allowing the server to push video data and respond to user signaling in real time, enhancing the real-time nature of video playback.

[0087] 3. Reduce bandwidth consumption:

[0088] -Utilizes the efficient encoding and transmission of TS stream containers to reduce unnecessary data transmission and optimize bandwidth usage.

[0089] 4. Improve user experience:

[0090] -Supports multiple control functions such as video play, pause, double-speed play and jump. Users can flexibly control video playback according to their needs, improving the viewing experience.

[0091] 5. Reduce system complexity:

[0092] -By multiplexing signaling and video data on the same WebSocket channel, the number of connections required by the system is reduced and the system architecture is simplified.

[0093] 6. Enhance system scalability:

[0094] -The designed video playback signaling protocol has good scalability and can easily add new control functions to adapt to future development needs.

[0095] 7. Enhanced Security:

[0096] -Uses wss (WebSocket Secure) protocol for data transmission, providing data encryption and security.

[0097] 8. Optimize resource allocation and improve system response speed:

[0098] - Through intelligent buffer management and adaptive bitrate streaming technology, client resource allocation is optimized, reducing lag and buffering time.

[0099] - Fast signaling processing and video data transmission reduces the time from user operation to system response, improving response speed.

[0100] These beneficial effects work together to make the video playback method based on WebSocket and TS stream an efficient, stable, secure and user-friendly video playback solution.

[0101] Example 2, based on Example 1, proposes a video playback system based on websocket-ts, which is applied to a video playback method based on websocket-ts. The system includes:

[0102] A WebSocket server is used to receive connection requests from clients and establish persistent connections; it also includes: a signaling parsing module, which is used to parse control signaling from the client and perform corresponding operations based on the signaling content; a video data caching and flow control module, which is used to cache video data and adjust the transmission of video data based on the signaling content, ensuring the timely transmission of key frames and responding to high-priority signaling.

[0103] A client that establishes a connection with the WebSocket server and sends control signals and receives video data;

[0104] A multiplexing mechanism that allows simultaneous transmission of control signaling and video data over a WebSocket channel. It distinguishes signaling frames from video data frames by defining frame formats and header information, and ensures that signaling frames are prioritized.

[0105] The frame encapsulation module is used to encapsulate control signaling and video data into frames with type identification, length information and sequence number, and send them through WebSocket;

[0106] The priority control module is used to ensure that the control signaling frames are processed before the video data frames.

[0107] The system also includes: a real-time optimization module for reducing delays in video data transmission and signaling processing, and improving response speed; in particular, signaling uses binary processing to improve transmission efficiency and bandwidth utilization; an adaptive bit rate control module that allows the player to dynamically adjust the required video bit rate based on the current playback effect. The server supports video data transmission at multiple bit rates.

[0108] The system also includes: a security module for encrypting WebSocket communication and ensuring the security of data transmission through the WSS protocol and TLS / SSL; at the same time, encrypting or signing signaling and video data to prevent man-in-the-middle attacks and data tampering; an error detection and recovery module for detecting errors in the transmission process, such as CRC check, and implementing error recovery strategies, such as retransmission mechanism, to ensure the integrity and reliability of data transmission.

[0109] The system also includes: a synchronization module for achieving synchronization between the client and the server to ensure that both parties have consistent processing logic for signaling and video data; synchronizing video playback progress through timestamps to ensure the continuity and consistency of video playback; a heartbeat and timeout mechanism module for periodically sending heartbeat signaling to detect and maintain the activity of the WebSocket connection, and taking corresponding measures, such as retransmission or disconnection, when the signaling or data frame is not confirmed within a predetermined time.

[0110] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A video playback method based on websocket-ts, characterized by: The method comprises the following steps: Define the multiplexing format of signaling and video data, which are transmitted through the WebSocket channel. The signaling contains the control command type, timestamp and target location information, and the video data is encapsulated in TS stream format. Design frame header information, including identifiers for distinguishing signaling frames from video data frames, as well as type identifiers, length information, and sequence numbers to ensure correct parsing and reassembly of data; Implement a frame encapsulation mechanism to encapsulate signaling and video data into frames with frame headers and send them through the same WebSocket channel; Set up a priority control mechanism to define priorities for different types of frames and manage received frames through priority queues to ensure that signaling frames are processed before video data frames; Implement a flow control algorithm to dynamically adjust the transmission rate of video data based on network conditions and the processing capabilities of the receiving end to avoid network congestion and data loss.

2. A websocket-ts based video playback method according to claim 1, characterized in that: It also includes the design of heartbeat and timeout mechanisms, which detect and maintain the activity of the WebSocket connection by sending heartbeat signals at regular intervals, and design a timeout mechanism to handle signaling or data frames that are not confirmed within the predetermined time, ensuring the stability of the connection and the integrity of the data.

3. A websocket-ts based video playback method according to claim 2, characterized in that: It also includes error handling and recovery mechanisms, which detect errors during transmission through error detection mechanisms such as CRC check and implement error recovery strategies such as retransmission to ensure the integrity and reliability of signaling and video data.

4. A websocket-ts based video playback method according to claim 1, characterized in that: It also includes a synchronization mechanism between the client and server, which uses synchronization information such as timestamps to ensure that the processing logic of signaling and video data on both sides is consistent, thereby maintaining consistency in video playback progress.

5. A websocket-ts based video playback method according to claim 1, characterized in that: Security considerations are also included. The WebSocket Secure protocol is used to encrypt WebSocket communications through TLS / SSL, and signaling and video data are encrypted or signed to prevent man-in-the-middle attacks and data tampering, ensuring the security of data transmission.

6. A websocket-ts based video playback system, applied to the websocket-ts based video playback method according to any one of claims 1 to 5, characterized in that: The system comprises: A WebSocket server, used to receive connection requests from clients and establish persistent connections; A client that establishes a connection with the WebSocket server and sends control signals and receives video data; A multiplexing mechanism that allows simultaneous transmission of control signaling and video data over a WebSocket channel. It distinguishes signaling frames from video data frames by defining frame formats and header information, and ensures priority processing of signaling frames. The frame encapsulation module is used to encapsulate control signaling and video data into frames with type identification, length information and sequence number, and send them through WebSocket; The priority control module is used to ensure that the control signaling frames are processed before the video data frames.

7. A websocket-ts based video playback system according to claim 6, characterized in that: The WebSocket server also includes: Signaling parsing module, used to parse the control signaling from the client and perform corresponding operations based on the signaling content; The video data cache and flow control module is used to cache video data and adjust the transmission of video data according to the signaling content to ensure the timely transmission of key frames and respond to high-priority signaling.

8. A websocket-ts based video playback system according to claim 6, characterized in that: The system also includes: A real-time optimization module is used to reduce delays in video data transmission and signaling processing, improving response speed. In particular, signaling uses binary processing to improve transmission efficiency and bandwidth utilization. The adaptive bitrate control module allows the player to dynamically adjust the required video bitrate based on the current playback effect. The server supports video data transmission at multiple bitrates.

9. A websocket-ts based video playback system according to claim 6, characterized in that: The system also includes: The security module is used to encrypt WebSocket communications and ensure the security of data transmission through the WSS protocol and TLS / SSL. At the same time, it encrypts or signs signaling and video data to prevent man-in-the-middle attacks and data tampering. The error detection and recovery module is used to detect errors during transmission, such as CRC check, and implement error recovery strategies, such as retransmission mechanism, to ensure the integrity and reliability of data transmission.

10. A websocket-ts based video playback system according to claim 6, characterized in that: The system also includes: The synchronization module is used to achieve synchronization between the client and the server, ensuring that both sides have consistent processing logic for signaling and video data; synchronizing video playback progress through timestamps to ensure the coherence and consistency of video playback; The heartbeat and timeout mechanism module is used to periodically send heartbeat signals to detect and maintain the activity of the WebSocket connection, and take corresponding measures, such as resending or disconnecting, when the signaling or data frame is not confirmed within a predetermined time.

Images