Websocket-ts-based video playback method and system

Abstract

The invention relates to the technical field of streaming media video playback, in particular to a websocket-ts-based video playback method and system, and the method comprises the following steps: defining a multiplexing format of signaling and video data, designing frame header information, realizing a frame packaging mechanism, setting a priority control mechanism, and implementing a flow control algorithm. The beneficial effects are that according to the websocket-ts-based video playback method and system provided by the invention, the video data are transmitted through the Websocket protocol, and the request / response round-trip time in the HTTP protocol is reduced, so that the data transmission efficiency is improved; webSocket is adopted to support full-duplex communication, and a server is allowed to push video data and respond to user signaling in real time, so that the real-time performance of video playing is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of streaming media video playback, in particular to a websocket-ts-based video playback method and system. Background Art

[0002] Common solutions for video playback:

[0003] (1) HLS-based playback protocol. The advantage of HLS-based playback is good compatibility, which is suitable for use in browsers and mobile phones. The disadvantage is that some players do not support playback of videos with specific encodings, such as HEVC encoding. It is suitable for occasions with loose encoding type requirements. Another disadvantage is that the transmission efficiency and real-time signaling response are not high.

[0004] (2) Limited video playback control functions: Existing HLS or RTSP-based video transmission solutions often only support basic play and pause functions. They lack support for advanced functions such as double-speed playback and video jump, and the implementation methods are relatively complex.

[0005] (3) Scalability limitations: With the diversification of video content and the personalization of user needs, the HTTP protocol has scalability limitations in supporting complex signaling control and advanced functions.

[0006] (4) Bandwidth waste: Under the HTTP protocol, video data is usually transmitted in blocks, which may lead to unnecessary repeated requests and data transmission, thus wasting bandwidth resources. Summary of the invention

[0007] The purpose of the present invention is to provide a websocket-ts based video playback method and system to solve the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solution: a video playback method based on websocket-ts, the method comprising the following steps:

[0009] Define the multiplexing format of signaling and video data, which are transmitted through the WebSocket channel. The signaling contains the control command type, timestamp and target location information, and the video data is encapsulated in TS stream format.

[0010] Design frame header information, including identifiers for distinguishing signaling frames from video data frames, as well as type identification, length information, and sequence numbers to ensure correct parsing and reassembly of data;

[0011] Implement the frame encapsulation mechanism to encapsulate signaling and video data into frames with frame headers respectively and send them through the same WebSocket channel;

[0012] Set up a priority control mechanism to define priorities for different types of frames and manage received frames through priority queues to ensure that signaling frames are processed before video data frames;

[0013] Implement a flow control algorithm to dynamically adjust the transmission rate of video data according to network conditions and the processing capacity of the receiving end to avoid network congestion and data loss.

[0014] Preferably, the design of heartbeat and timeout mechanisms is also included, which detects and maintains the activity of the WebSocket connection by sending heartbeat signaling at regular intervals, and designs a timeout mechanism to process signaling or data frames that are not confirmed within a predetermined time, thereby ensuring the stability of the connection and the integrity of the data.

[0015] Preferably, an error handling and recovery mechanism is also included, which detects errors in the transmission process through error detection mechanisms such as CRC check, and implements error recovery strategies such as retransmission to ensure the integrity and reliability of signaling and video data.

[0016] Preferably, a synchronization mechanism between the client and the server is also included, which ensures that the processing logic of the signaling and video data of both parties is consistent through synchronization information such as timestamps, thereby maintaining the consistency of the video playback progress.

[0017] Preferably, security considerations are also taken into account, using the WebSocket Secure protocol to encrypt WebSocket communications through TLS / SSL, and encrypting or signing signaling and video data to prevent man-in-the-middle attacks and data tampering, thereby ensuring the security of data transmission.

[0018] A websocket-ts based video playback system is applied to a websocket-ts based video playback method, the system comprising:

[0019] A WebSocket server, which receives connection requests from clients and establishes persistent connections;

[0020] A client that establishes a connection with the WebSocket server and sends control signaling and receives video data;

[0021] A multiplexing mechanism that allows simultaneous transmission of control signaling and video data over the WebSocket channel, distinguishing signaling frames from video data frames by defining frame formats and frame header information, and ensuring priority processing of signaling frames;

[0022] The frame encapsulation module is used to encapsulate control signaling and video data into frames with type identification, length information and sequence number, and send them through WebSocket;

[0023] The priority control module is used to ensure that the control signaling frame is processed before the video data frame.

[0024] Preferably, the WebSocket server further includes:

[0025] The signaling parsing module is used to parse the control signaling from the client and perform corresponding operations according to the signaling content;

[0026] The video data cache and flow control module is used to cache video data and adjust the transmission of video data according to the signaling content to ensure the timely transmission of key frames and respond to high-priority signaling.

[0027] Preferably, the system further comprises:

[0028] Real-time optimization module, used to reduce the delay of video data transmission and signaling processing, and improve the response speed; in particular, signaling adopts binary processing to improve transmission efficiency and bandwidth utilization;

[0029] The adaptive bitrate control module allows the player to dynamically adjust the required video bitrate according to the current playback effect. The server supports video data transmission at multiple bitrates.

[0030] Preferably, the system further comprises:

[0031] Security module, used to encrypt WebSocket communication and ensure the security of data transmission through wss protocol and TLS / SSL; at the same time, encrypt or sign signaling and video data to prevent man-in-the-middle attacks and data tampering;

[0032] The error detection and recovery module is used to detect errors in the transmission process, such as CRC check, and implement error recovery strategies, such as retransmission mechanism, to ensure the integrity and reliability of data transmission.

[0033] Preferably, the system further comprises:

[0034] The synchronization module is used to achieve synchronization between the client and the server to ensure that the processing logic of the signaling and video data is consistent between the two parties; the video playback progress is synchronized through the timestamp to ensure the continuity and consistency of the video playback;

[0035] The heartbeat and timeout mechanism module is used to periodically send heartbeat signals to detect and maintain the activity of the WebSocket connection, and take corresponding measures, such as resending or disconnecting, when the signaling or data frame is not confirmed within a predetermined time.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The websocket-ts based video playback method and system proposed in the present invention transmit video data through the WebSocket protocol, reduce the request / response round-trip time in the HTTP protocol, and thus improve the efficiency of data transmission; adopt WebSocket to support full-duplex communication, allow the server to push video data and respond to user signaling in real time, and enhance the real-time performance of video playback; reduce bandwidth consumption, utilize the efficient encoding and transmission of TS stream containers, reduce unnecessary data transmission, and optimize bandwidth usage; support multiple control functions such as video playback, pause, double-speed playback and jump, so that users can flexibly control video playback according to their needs, and improve viewing experience; by multiplexing signaling and video data on the same WebSocket channel, the number of connections required by the system is reduced, and the system architecture is simplified. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the method of the present invention;

[0039] Figure 2 A schematic diagram of signaling and data formats is defined for the present invention;

[0040] Figure 3 A schematic diagram of frame header information is designed for the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose and technical solution of the present invention clearly and completely described, and the advantages more clearly understood, the embodiments of the present invention are further described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, rather than all of the embodiments, and are only used to explain the embodiments of the present invention, and are not used to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] For example, see Figures 1 to 3 The present invention provides a technical solution: a video playback method based on websocket-ts, the method comprising the following steps:

[0043] Step 1: Multiplexing signaling and video data

[0044] Designing a multiplexing mechanism for signaling and video data on the same WebSocket channel involves how to effectively transmit control signaling and video data simultaneously on the same communication channel. The following are the detailed steps to implement this mechanism:

[0045] 1 Define signaling and data formats:

[0046] - Define a clear format for signaling, including information such as signaling type (such as play, pause, speed, jump, etc.), timestamp, target position, etc.

[0047] -Define a format for video data. Usually, video data is transmitted in TS stream format. Each TS packet contains a timestamp and payload data.

[0048] 2 Design frame header information:

[0049] -Define a frame header for each frame sent via WebSocket, which contains an identifier to distinguish signaling frames from video data frames.

[0050] -The frame header contains type identification, length information, sequence number, etc., so that the receiving end can correctly parse and reassemble the data.

[0051] 3. Implement frame encapsulation mechanism:

[0052] -When the client or server needs to send signaling, the signaling information is encapsulated into a frame, added with a frame header, and sent through WebSocket.

[0053] -When the server needs to send video data, it encapsulates the TS stream data into frames, adds a frame header, and sends it through WebSocket.

[0054] 4 Design priority control:

[0055] - Define priorities for different types of frames to ensure that control signaling frames can be processed before video data frames.

[0056] - Implement a priority queue to sort received frames to ensure timely response to signaling.

[0057] 5. Implement flow control:

[0058] -Dynamically adjust the transmission rate of video data according to network conditions and the processing capacity of the receiving end to avoid congestion and packet loss.

[0059] - Implement flow control algorithms, such as sliding window protocols, to manage the sending and receiving of data.

[0060] 6 Design heartbeat and timeout mechanism:

[0061] -Send heartbeat signals regularly to detect and maintain the activity of the WebSocket connection.

[0062] -Design a timeout mechanism to resend or take corresponding measures when the signaling or data frame is not confirmed within the predetermined time.

[0063] 7 Implement error handling and recovery:

[0064] -Design error detection mechanisms, such as CRC check, to detect errors during transmission.

[0065] - Implement error recovery strategies, such as retransmission mechanisms, to ensure the integrity of signaling and video data.

[0066] 8 Synchronization between client and server:

[0067] -The client and server need to synchronize the signaling and video data processing logic to ensure that both sides parse and process the frames in a consistent manner.

[0068] -Implement synchronization mechanisms, such as synchronizing video playback progress through timestamps to ensure consistency in video playback.

[0069] 9. Security considerations:

[0070] -Ensure the security of WebSocket communication, use the wss (WebSocket Secure) protocol, and encrypt data transmission through TLS / SSL.

[0071] -Encrypt or sign signaling and video data to prevent man-in-the-middle attacks and data tampering.

[0072] Through the above steps, an efficient and reliable signaling and video data multiplexing mechanism can be implemented on the same WebSocket channel, thereby improving the performance and user experience of the video streaming service.

[0073] Step 2: The server and client implement the corresponding signaling control protocol

[0074] 1. Develop a WebSocket server that can receive connection requests from clients and establish a persistent connection. The server itself must define a method that can parse client signaling and ensure that the signaling has the highest priority and can be processed first.

[0075] 2. Implement the video data caching and flow control mechanism on the server side to ensure that video data can be transmitted on demand.

[0076] The server caches a key frame sequence GOP to ensure that the client can output the image in time. In terms of the flow control mechanism, it can decide which video data can be directly discarded based on the content of the signal itself, so as to respond to the signal with higher priority.

[0077] Step 3: Real-time and efficiency optimization methods

[0078] 1Optimize video data transmission and signaling processing to reduce delays and improve response speed.

[0079] Signaling uses binary processing, which can maximize the use of transmission bandwidth, avoid deserialization operations on text data, and be more efficient.

[0080] 2. Controlled by signaling, adaptive bit rate (ABR). The player can dynamically adjust the required bit rate according to the current playback effect, and the server needs to support videos with different bit rates.

[0081] 3) Beneficial effects of the invention:

[0082] The beneficial effects of the video playback method based on WebSocket and TS stream include:

[0083] 1. Improve transmission efficiency:

[0084] -Video data is transmitted through the WebSocket protocol, which reduces the request / response round-trip time in the HTTP protocol, thereby improving the efficiency of data transmission.

[0085] 2. Enhance real-time performance:

[0086] -WebSocket supports full-duplex communication, allowing the server to push video data and respond to user signaling in real time, enhancing the real-time performance of video playback.

[0087] 3. Reduce bandwidth consumption:

[0088] - Utilizes efficient encoding and transmission of TS stream containers to reduce unnecessary data transmission and optimize bandwidth usage.

[0089] 4. Improve user experience:

[0090] -Supports multiple control functions such as video play, pause, double-speed play and jump. Users can flexibly control video playback according to their needs, improving the viewing experience.

[0091] 5. Reduce system complexity:

[0092] - By multiplexing signaling and video data on the same WebSocket channel, the number of connections required by the system is reduced and the system architecture is simplified.

[0093] 6. Enhance the scalability of the system:

[0094] -The designed video playback signaling protocol has good scalability and can easily add new control functions to adapt to future development needs.

[0095] 7. Enhanced Security:

[0096] -Use wss (WebSocket Secure) protocol for data transmission, providing data encryption and security protection.

[0097] 8. Optimize resource allocation and improve system response speed:

[0098] -Optimizes client resource allocation and reduces lag and buffering time through intelligent buffer management and adaptive bitrate streaming technology.

[0099] -Fast signaling processing and video data transmission reduces the time from user operation to system response and improves response speed.

[0100] These beneficial effects work together to make the video playback method based on WebSocket and TS stream an efficient, stable, secure and user-friendly video playback solution.

[0101] Embodiment 2, based on embodiment 1, proposes a video playback based on websocket-ts, which is applied to a video playback method based on websocket-ts, and the system includes:

[0102] A WebSocket server is used to receive connection requests from clients and establish persistent connections; it also includes: a signaling parsing module, which is used to parse control signaling from clients and perform corresponding operations according to the signaling content; a video data caching and flow control module, which is used to cache video data and adjust the transmission of video data according to the signaling content, ensuring the timely transmission of key frames and responding to high-priority signaling.

[0103] A client that establishes a connection with the WebSocket server and sends control signaling and receives video data;

[0104] A multiplexing mechanism that allows simultaneous transmission of control signaling and video data over the WebSocket channel, distinguishing signaling frames from video data frames by defining frame formats and frame header information, and ensuring priority processing of signaling frames;

[0105] The frame encapsulation module is used to encapsulate control signaling and video data into frames with type identification, length information and sequence number, and send them through WebSocket;

[0106] The priority control module is used to ensure that the control signaling frame is processed before the video data frame.

[0107] The system also includes: a real-time optimization module for reducing the delay of video data transmission and signaling processing and improving the response speed; in particular, the signaling adopts binary processing to improve transmission efficiency and bandwidth utilization; an adaptive bit rate control module allows the playback end to dynamically adjust the required video bit rate according to the current playback effect, and the server supports video data transmission at multiple bit rates.

[0108] The system also includes: a security module for encrypting WebSocket communication and ensuring the security of data transmission through the wss protocol and TLS / SSL; at the same time, encrypting or signing signaling and video data to prevent man-in-the-middle attacks and data tampering; an error detection and recovery module for detecting errors in the transmission process, such as CRC check, and implementing error recovery strategies, such as retransmission mechanism, to ensure the integrity and reliability of data transmission.

[0109] The system also includes: a synchronization module, which is used to achieve synchronization between the client and the server to ensure that the processing logic of the signaling and video data on both sides is consistent; synchronize the video playback progress through timestamps to ensure the continuity and consistency of video playback; a heartbeat and timeout mechanism module, which is used to send heartbeat signaling at regular intervals to detect and maintain the activity of the WebSocket connection, and take corresponding measures when the signaling or data frame is not confirmed within a predetermined time, such as retransmission or disconnection.

[0110] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A video playback method based on websocket-ts, characterized in that: The method comprises the following steps: Define the multiplexing format of signaling and video data, which are transmitted through the WebSocket channel. The signaling contains the control command type, timestamp and target location information, and the video data is encapsulated in TS stream format. Design frame header information, including identifiers for distinguishing signaling frames from video data frames, as well as type identification, length information, and sequence numbers to ensure correct parsing and reassembly of data; Implement the frame encapsulation mechanism to encapsulate signaling and video data into frames with frame headers respectively and send them through the same WebSocket channel; Set up a priority control mechanism to define priorities for different types of frames and manage received frames through priority queues to ensure that signaling frames are processed before video data frames; Implement a flow control algorithm to dynamically adjust the transmission rate of video data according to network conditions and the processing capacity of the receiving end to avoid network congestion and data loss.

2. A websocket-ts based video playback method according to claim 1, characterized in that: It also includes the design of heartbeat and timeout mechanisms, which detect and maintain the activity of the WebSocket connection by sending heartbeat signals at regular intervals, and design a timeout mechanism to handle signaling or data frames that are not confirmed within a predetermined time, ensuring the stability of the connection and the integrity of the data.

3. A websocket-ts based video playback method according to claim 2, characterized in that: It also includes error handling and recovery mechanisms, which detect errors during transmission through error detection mechanisms such as CRC check, and implement error recovery strategies such as retransmission to ensure the integrity and reliability of signaling and video data.

4. A websocket-ts based video playback method according to claim 1, characterized in that: It also includes a synchronization mechanism between the client and the server, which uses synchronization information such as timestamps to ensure that the processing logic of signaling and video data on both sides is consistent, thereby maintaining the consistency of video playback progress.

5. A websocket-ts based video playback method according to claim 1, characterized in that: It also takes security into consideration, using the WebSocket Secure protocol to encrypt WebSocket communications through TLS / SSL, and encrypting or signing signaling and video data to prevent man-in-the-middle attacks and data tampering, ensuring the security of data transmission.

6. A websocket-ts based video playback, applied to the websocket-ts based video playback method according to any one of claims 1 to 5, characterized in that: The system comprises: A WebSocket server, which receives connection requests from clients and establishes persistent connections; A client that establishes a connection with the WebSocket server and sends control signaling and receives video data; A multiplexing mechanism that allows simultaneous transmission of control signaling and video data over the WebSocket channel, distinguishing signaling frames from video data frames by defining frame formats and frame header information, and ensuring priority processing of signaling frames; The frame encapsulation module is used to encapsulate control signaling and video data into frames with type identification, length information and sequence number, and send them through WebSocket; The priority control module is used to ensure that the control signaling frame is processed before the video data frame.

7. A websocket-ts based video playback system according to claim 6, characterized in that: The WebSocket server also includes: The signaling parsing module is used to parse the control signaling from the client and perform corresponding operations according to the signaling content; The video data cache and flow control module is used to cache video data and adjust the transmission of video data according to the signaling content to ensure the timely transmission of key frames and respond to high-priority signaling.

8. A websocket-ts based video playback system according to claim 6, characterized in that: The system also includes: Real-time optimization module, used to reduce the delay of video data transmission and signaling processing, and improve the response speed; in particular, signaling adopts binary processing to improve transmission efficiency and bandwidth utilization; The adaptive bitrate control module allows the player to dynamically adjust the required video bitrate according to the current playback effect. The server supports video data transmission at multiple bitrates.

9. A websocket-ts based video playback system according to claim 6, characterized in that: The system also includes: Security module, used to encrypt WebSocket communication and ensure the security of data transmission through wss protocol and TLS / SSL; at the same time, encrypt or sign signaling and video data to prevent man-in-the-middle attacks and data tampering; The error detection and recovery module is used to detect errors in the transmission process, such as CRC check, and implement error recovery strategies, such as retransmission mechanism, to ensure the integrity and reliability of data transmission.

10. A websocket-ts based video playback system according to claim 6, characterized in that: The system also includes: The synchronization module is used to achieve synchronization between the client and the server to ensure that the processing logic of the signaling and video data is consistent between the two parties; the video playback progress is synchronized through the timestamp to ensure the continuity and consistency of the video playback; The heartbeat and timeout mechanism module is used to periodically send heartbeat signals to detect and maintain the activity of the WebSocket connection, and take corresponding measures, such as resending or disconnecting, when the signaling or data frame is not confirmed within a predetermined time.

Images