Method for seamlessly switching video streams based on HLS protocol

Abstract

The invention discloses a seamless video stream switching method based on an HLS protocol, which relates to video processing and comprises the following steps: S1, processing flow characteristics of a multi-code-rate HLS protocol switching request video; and S2, performing flow characteristics of correct data connection on the target video data and the current video cache data in the code rate switching process. And S3, the player can seamlessly and smoothly play flow characteristics of different stream data through target stream video data and refreshing a hard decoder, and through the flow characteristics of the method, when the player plays audio and video resources of a multi-code-rate HLS protocol, under the condition that extra performance loss and extremely low flow waste are not needed, the player can continuously play the audio and video resources of the multi-code-rate HLS protocol under the condition that the resources are continuously played, so that the audio and video resources of the multi-code-rate HLS protocol can be continuously played. According to the method, resources of different code rates are seamlessly switched, the user experience video playing is continuous, meanwhile, the interaction effect of resource switching is natural, the situation of switching failure can be correctly processed, and the user experience of video playing is improved.

Description

Technical Field

[0001] The present invention relates to the field of video processing, and more specifically, to a method for seamlessly switching video streams based on the HLS protocol. Background Art

[0002] In current video playback applications, the HLS protocol is often used to provide users with multi-bitrate or multi-resolution video playback. There are also different ways to switch video streams on the market. The simpler solution is to pause the playback of the original video stream and start the new video stream. The switching process has obvious screen interruptions and poor user experience. In order to improve the user experience, we need to transform the data transmission logic of the HLS protocol to provide users with seamless video stream switching without closing the original player.

[0003] To this end, a method for seamlessly switching video streams based on the HLS protocol is proposed. Summary of the invention

[0004] To solve the above problems, the present invention adopts the following technical solutions.

[0005] 1. A method for seamlessly switching video streams based on the HLS protocol, characterized in that it comprises the following steps: S1: Process characteristics of processing multi-bitrate HLS protocol switching request videos.

[0006] S2: The process characteristics of the bit rate switching process for correctly connecting the target video data and the current video buffer data.

[0007] S3. Refresh the hardware decoder through the target stream video data, so that the player can seamlessly and smoothly play the process characteristics of different stream data.

[0008] Compared with the prior art, the present invention has the following beneficial effects: Through the process characteristics of the method of the present invention, when playing audio and video resources of the multi-bitrate HLS protocol, the player can seamlessly switch resources of different bitrates without interrupting the playing of resources without additional performance loss and minimal traffic waste. The user experience of video playback is continuous, and the interactive effect of resource switching is natural, and the failure of switching can be properly handled, thereby improving the user experience of video playback. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a characteristic diagram of the process of switching a video request of a multi-bitrate HLS protocol according to the present invention; Figure 2 A flow characteristic diagram of correctly connecting target video data and current video buffer data during the bit rate switching process of the present invention; Figure 3 This is a characteristic diagram of the process by which the player of the present invention can play different stream data seamlessly and smoothly. DETAILED DESCRIPTION

[0010] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention; it is obvious that the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments, and all other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making creative work are within the scope of protection of the present invention.

[0011] In current video playback applications, the HLS protocol is often used to provide users with multi-bitrate or multi-resolution video playback. There are also different ways to switch video streams on the market. The simpler solution is to pause the playback of the original video stream and start the new video stream. The switching process has obvious screen interruptions and poor user experience. In order to improve the user experience, we need to transform the data transmission logic of the HLS protocol to provide users with seamless video stream switching effects without closing the original player. To this end, the data transmission logic of the HLS protocol is transformed through the following processing steps. Example

[0012] S1. Process characteristics of processing multi-bitrate HLS protocol switching request videos.

[0013] S2. Process characteristics of correctly connecting the target video data and the current video buffer data during the bit rate switching process.

[0014] S3. Refresh the hardware decoder through the target stream video data, so that the player can seamlessly and smoothly play the process characteristics of different stream data.

[0015] S1. Characteristics of the media information switching request process of the multi-bitrate HLS protocol, such as ( Figure 1 ), the process characteristic steps are as follows: [S11]. After accepting the bit rate switching request, first set the minimum switchable duration (the duration of the next two TS segments after the current playback progress, described as switchTime), and record the switch start time point beginTime.

[0016] [S12]. Determine the numerical relationship between the current bit rate cached video duration currentDuration and switchTime.

[0017] [S13] If currentDuration > switchTime, then execute step [S14]; if currentDuration < switchTime, then execute step [S15].

[0018] [S14]. Pause the current video stream request thread, and at the same time open the switched target bitrate video stream request thread to obtain the target bitrate data. Then execute step [S16].

[0019] [S15]. Continue the current bitrate request thread, record the current time point endTime, and judge the size relationship between the waiting time waitTime (waitTime = endTime - beginTime) and the preset timeout duration Timeout.

[0020] [S16]. If waitTime < Timeout, then play while accepting new video data and execute step [S12]; if waitTime > Timeout, then execute step [S19].

[0021] [S17]. Set the starting request progress of the target bitrate request thread to the next segment after the currently played segment.

[0022] [S18]. Set the starting position of the target bitrate request data return to the first key frame position switchPos before currentDuration, and return the video frame data pkt to the player.

[0023] [S19]. Continue the current bitrate playback logic, mark the status as bitrate switch timeout, and end the bitrate switch process.

[0024] S2. The process characteristics of correctly connecting the target video stream data and the current video stream cache data during the bitrate switch process are as shown in ( Figure 2 ), and the process characteristic steps are as follows: [S21]. When the video data pkt of [S17] is returned, update the current stream information of the player to the target video stream information.

[0025] [S22]. When the video data pkt of [S17] is returned, if the stream subscript of pkt is different from the stream subscript of the current data cache queue and pkt is a key frame, delete the node data in the current cache queue whose frame pts is greater than switchTime, and at the same time add pkt to the cache queue; otherwise, discard pkt.

[0026] [S23]. When pkt in S

[22] is video frame data and is a key frame, process the media data extraData in the information of the target bit rate stream and add it as the decoding information of pkt.

[0027] S3. Refresh the decoder through the target stream video data, so that the player can seamlessly and smoothly play the process features of different streams, such as ( Figure 3 ), the process characteristic steps are as follows: [S31]. When the current decoding mode is hard decoding, when obtaining the video frame data avpkt for decoding, if avpkt contains decoding information, a new decoder context object new_avctx is initialized by the decoding information.

[0028] [S32]. Refresh the hardware decoder HardwareCode of the corresponding device system through the decoder context object of [S31].

[0029] [S33]. Use the refreshed hardware decoder to decode avpkt and subsequent video frame data to complete the seamless switching process of different bit rates.

[0030] The above is only a preferred specific implementation of the present invention; however, the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solution and its improved conception within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for seamlessly switching video streams based on the HLS protocol, characterized in that: The following steps are involved: S1: Process characteristics of processing multi-bitrate HLS protocol switching request video; S2: Processing characteristics of the bit rate switching process for correctly connecting the target video data and the current video buffer data; S3. Refresh the hardware decoder through the target stream video data, so that the player can seamlessly and smoothly play the process characteristics of different stream data.

2. The method for seamlessly switching video streams based on the HLS protocol according to claim 1, characterized in that: The step S1 comprises the following steps: S11: After receiving the bit rate switching request, first set the minimum switchable duration, record the switch start time point as beginTime, and the minimum switchable duration is the duration of the next two TS segments of the current playback progress, described as switchTime; S12: Determine the numerical relationship between the duration of the video cached at the current bit rate, currentDuration, and switchTime.

3. The method for seamlessly switching video streams based on the HLS protocol according to claim 2, characterized in that: It also includes the following steps: S13: if the cached video duration currentDuration> the minimum switchable duration switchTime, execute step S14; if the cached video duration currentDuration< the minimum switchable duration switchTime, execute step S15; S14: Pause the current video stream request thread, and open the switch target bit rate video stream request thread to obtain the target bit rate data. Then execute step S16; S15: Continue the current bit rate request thread, record the current time point endTime, and determine the relationship between the waiting time waitTime and the preset timeout time Timeout, the waiting time waitTime = the current time point endTime - the starting time point beginTime.

4. The method for seamlessly switching video streams based on the HLS protocol according to claim 3, characterized in that: It also includes the following steps: S16: if the waiting time waitTime < the timeout time Timeout, then play and execute step S12 while receiving new video data; if the waiting time waitTime > the timeout time Timeout, then execute step S19; S17: setting the start request progress of the request thread of the target bit rate to the next segment of the current playback segment; S18: Set the starting position of the target bit rate request data returned to the first key frame position switchPos before currentDuration, and return the video frame data pkt to the player; S19: Continue the current bit rate playback logic, mark the state as bit rate switching timeout, and end the bit rate switching process.

5. The method for seamlessly switching video streams based on the HLS protocol according to claim 1, characterized in that: The step S2 comprises the following steps: S21: when the video data pkt of step S17 is returned, the current stream information of the player is updated to the target video stream information; S22: when the video data pkt of step S17 is returned, if it is determined that the stream index of the video data pkt is different from the stream index of the current data cache queue and the video data pkt is a key frame, delete the node data whose frame pts is greater than the minimum switch time switchTime in the current cache queue, and add the video data pkt to the cache queue, otherwise discard the video data pkt; S23: When the video data pkt in step S22 is video frame data and a key frame, the media data extraData in the information of the target bit rate stream is processed and added as decoding information of the video data pkt.

6. The method for seamlessly switching video streams based on the HLS protocol according to claim 1, characterized in that: The process of refreshing the decoder by using the target stream video data in step S3 so that the player can play different streams seamlessly and smoothly also includes the following steps: S31: when the current decoding mode is hard decoding, when obtaining the video frame data avpkt for decoding, if the video frame data avpkt contains decoding information, initialize a new decoder context object new_avctx through the decoding information; S32: Refresh the hardware decoder HardwareCode of the corresponding device system through the decoder context object of step S31; S33: Use the refreshed hardware decoder to decode avpkt and subsequent video frame data to complete the seamless switching process of different bit rates.

Images