Video dynamic code rate playing method, device, equipment, medium and program product

Abstract

The invention relates to the technical field of communication, and provides a video dynamic code rate playing method and device, equipment, a medium and a program product, and the method comprises the steps: obtaining the network fluctuation information of a communication network in a playing process of a current video stream; if the network fluctuation information reaches a preset threshold value, determining a target video code rate according to the network fluctuation information, and determining a key frame position according to the playing progress of the current video stream; and playing a target video stream corresponding to the target video code rate at the key frame position. A non-real-time coding and decoding mode is adopted, when the network fluctuation information of the communication network reaches the preset threshold value, the target video code rate is determined according to the network fluctuation information, the key frame position is determined according to the playing progress of the current video stream, and the current video stream is switched to the target video stream corresponding to the target video code rate at the key frame position. Bandwidth resource consumption caused by real-time coding and decoding is reduced, when the network state of the communication network changes, adaptive playing of the code rate can be achieved, and audio and video playing lagging is avoided.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a method, apparatus, device, medium, and program product for dynamic video bit rate playback. Background Art

[0002] With the rapid development of internet and multimedia transmission technologies, the application scenarios of media transmission have become increasingly complex and diverse. For example, short videos, live broadcasts, and other audio and video content have become an integral part of people's daily lives. Faced with diverse and complex terminal devices and network conditions, ensuring the efficient transmission of media resources has become a major concern for many technical personnel. Especially when user network bandwidth fluctuates, ensuring the smoothness and clarity of media resource playback has become a pressing challenge. Consequently, research on dynamic bitrate playback of media resources under varying network conditions has garnered increasing attention.

[0003] Among existing technologies, the mainstream approach is to adjust the media resource's bitrate or resolution to suit the current playback state. After the bitrate is changed, new encoding and decoding processes are performed based on the changed bitrate. Internet audio and video streams are typically encoded and decoded in real time during transmission. However, real-time encoding and decoding consumes significant bandwidth resources, making it unsuitable for communication networks where bandwidth is a premium. Summary of the Invention

[0004] The present application provides a method, apparatus, device, medium and program product for dynamic video bit rate playback, which can achieve adaptive bit rate playback when the communication network status changes, thereby avoiding audio and video playback freezes.

[0005] In a first aspect, the present application provides a method for playing a video at a dynamic bit rate, comprising:

[0006] Get the network fluctuation information of the communication network during the playback of the current video stream;

[0007] If the network fluctuation information reaches a preset threshold, determining a target video bit rate according to the network fluctuation information and determining a key frame position according to the playback progress of the current video stream;

[0008] A target video stream corresponding to the target video bit rate is played at the key frame position.

[0009] In one embodiment, determining the target video bit rate according to the network fluctuation information includes:

[0010] Determining the packet loss rate and / or delay of the current video according to the network fluctuation information;

[0011] Determine the current playback bit rate of the client according to the currently supported video bit rate fed back by the client;

[0012] The target video bit rate is determined according to the packet loss rate and / or the delay and the current playback bit rate.

[0013] In one embodiment, determining the target video bit rate according to the packet loss rate and / or the delay and the current playback bit rate includes:

[0014] If the packet loss rate and / or the delay meet the preset bit rate increase condition, the current playback bit rate is increased and adjusted based on a preset bit rate gain adjustment factor to obtain the target video bit rate;

[0015] If the packet loss rate and / or the delay meet the preset bit rate reduction conditions, the current playback bit rate is reduced based on a preset bit rate reduction adjustment factor to obtain the target video bit rate;

[0016] Otherwise, the current playback bit rate is used as the target video bit rate.

[0017] In one embodiment, before obtaining the network fluctuation information of the communication network during the playback of the current video stream, the method further includes:

[0018] In response to an incoming IMS call request initiated by a client, performing media negotiation with the client to determine a current playback bitrate of the client;

[0019] A video stream corresponding to the current playback bit rate is selected and transmitted to the client.

[0020] In one embodiment, before responding to the IMS incoming call request initiated by the client, the method further includes:

[0021] Acquire a media resource to be played, and eliminate bidirectional frames in the media resource;

[0022] Encoding the eliminated media resources to obtain an encoded video of at least one bit rate and an H264 video stream corresponding to the encoded video, and merging the encoded video and the H264 to obtain a video stream;

[0023] Associating each of the video streams to a corresponding video stream track and retaining the audio of one of the video streams;

[0024] Wherein, the frame rates of all the video streams are the same.

[0025] In one embodiment, determining a key frame position according to the playback progress of the current video stream, and playing a target video stream corresponding to the target video bit rate at the key frame position includes:

[0026] Determine whether the current frame is a key frame according to the playback progress of the current video stream; if so, use the current frame as the key frame position; otherwise, use the position of the key frame closest to the current frame as the key frame position;

[0027] Switch to the video stream track corresponding to the target video bit rate at the key frame position and play the target video stream.

[0028] In one embodiment, it further includes:

[0029] In response to the key frame request message fed back by the client, determine whether the frequency and / or number of received key frame request messages reaches a set threshold; if so, determine the target video bit rate according to the network fluctuation information and determine the key frame position according to the playback progress of the current video stream, and play the target video stream corresponding to the target video bit rate at the key frame position.

[0030] In a second aspect, the present application further provides a video dynamic bit rate playback device, comprising:

[0031] A network dynamic monitoring module is used to obtain network fluctuation information of the communication network during the playback of the current video stream and determine whether the network fluctuation information reaches a preset threshold;

[0032] a bit rate switching determination module, configured to determine a target video bit rate according to the network fluctuation information and determine a key frame position according to the playback progress of the current video stream when the network fluctuation information reaches a preset threshold;

[0033] The media processing module is used to play the target video stream corresponding to the target video bit rate at the key frame position.

[0034] In a third aspect, the present application also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for dynamic video bit rate playback as described above is implemented.

[0035] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for dynamic bit rate playback of video.

[0036] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-described methods for dynamic bit rate playback of videos.

[0037] The video dynamic bit rate playback method, device, equipment, medium and program product provided by the present application adopt a non-real-time encoding and decoding method. When the network fluctuation information of the communication network reaches a preset threshold, the target video bit rate is determined according to the network fluctuation information and the key frame position is determined according to the playback progress of the current video stream. The key frame position is used as the switching point to switch the current video stream to the target video stream corresponding to the target video bit rate, thereby reducing the bandwidth resource consumption caused by real-time encoding and decoding. When the network status of the communication network changes, adaptive playback of the bit rate can be achieved to avoid audio and video playback jams. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 This is one of the flow charts of the video dynamic bit rate playback method provided by this application;

[0040] Figure 2 This is a diagram of the video call incoming call and signaling negotiation process;

[0041] Figure 3 This is a diagram of the audio call incoming call and signaling negotiation process;

[0042] Figure 4 This is a flowchart of the timeout retransmission mechanism;

[0043] Figure 5 This is the second flow chart of the video dynamic bit rate playback method provided by this application;

[0044] Figure 6 This is a schematic diagram of bitrate switching under different video stream tracks;

[0045] Figure 7 This is a principle block diagram of the video dynamic bit rate playback device provided by this application;

[0046] Figure 8 It is a structural diagram of the electronic device provided in this application. DETAILED DESCRIPTION

[0047] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0048] Due to the complexity and diversity of user environments, people can place calls through various channels, such as mobile phones, apps, HTML5, and mini-programs, all of which connect to the call platform system through different means. Compared to traditional voice-only ringback tone (RRT), the emergence of video RRT not only provides users with more information, but also enhances a company's brand image and broadens its promotional channels. When a user calls into the call platform through the IP Multimedia Subsystem (IMS) network and enters the video process, the traditional method is to directly play the corresponding MP4 file. Given the different network bandwidths and video quality requirements of different customers, this approach is obviously not ideal. For users with good network bandwidth, if the video bitrate is too low, the video playback will be unclear. Conversely, for users with poor network support, a high bitrate will cause video playback to be choppy and audio and video to be out of sync. Furthermore, if the communication network bandwidth fluctuates during the user's call, these issues can also arise.

[0049] To this end, the present application provides a method, apparatus, device, medium, and program product for dynamic bit rate playback of video ringback tone before an audio or video call is connected in an IMS network. This method not only enables dynamic selection and playback of videos and dynamic bit rate switching of adaptive networks, but also uses non-real-time encoding and decoding to reduce resource consumption and media processing pressure of the communication network, effectively solving the current technical bottleneck of dynamic switching of video bit rates in the communication network and improving the service quality of video calls.

[0050] The following instructions Figure 1-Figure 7 This application is described in detail.

[0051] Figure 1 This is one of the flow charts of the video dynamic bit rate playback method provided by this application, such as Figure 1 As shown, this application provides a video dynamic bit rate playback method, applicable to the server, including:

[0052] Step S100, obtaining network fluctuation information of the communication network during the playback of the current video stream; the network fluctuation information includes packet loss rate, delay, freeze rate and network traffic information.

[0053] Step S200: If the network fluctuation information reaches a preset threshold, determine a target video bit rate according to the network fluctuation information and determine a key frame position according to the playback progress of the current video stream;

[0054] When the packet loss rate is low, it means that the current network status of the communication network is good and the bandwidth has not reached the transmission limit. When the packet loss rate is high, it means that the current network status is poor and congestion has occurred in the communication network. Otherwise, the current network status is slightly poor, but can still meet the current bit rate playback.

[0055] When the latency continues to decrease, it means that the current network status is gradually improving. When the latency continues to increase, it means that the current network status is poor.

[0056] The packet loss rate and delay can be used as the network status judgment conditions of the communication network separately, or they can be used as the network status judgment conditions of the communication network in combination. For example, after the packet loss rate exceeds a certain value, the change of the delay can be judged.

[0057] When the network fluctuation information changes from a preset threshold range to another threshold range, the video bit rate switching process is started to ensure a smooth transition of the video stream; if the network is detected to be restored, the video stream is switched again, thereby dynamically controlling the adaptive playback of video streams with different bit rates.

[0058] There is a predetermined mapping relationship between network fluctuation information and target video bit rate.

[0059] Step S300: Play a target video stream corresponding to the target video bit rate at the key frame position.

[0060] This application is applicable to dynamic bitrate playback scenarios for video ringback tones. These ringback tones include multiple video streams, each with a different bitrate. All video streams are located in different video stream tracks, and all video streams have a fixed frame rate format. Therefore, the number of key frames and the position of each key frame are the same for all video streams. Determining the key frame position based on the playback progress of the current video stream refers to determining the position of the key frame that is closest to the playback progress of the current video stream.

[0061] The target video stream corresponding to the target video bitrate is played at the keyframe position. This means switching to the target video stream by switching the video stream track at the keyframe position, and sending the target video stream to the client via a pre-established video playback link. Both the current and target video streams are played using the same video playback link, requiring only a single link to switch the video bitrate, avoiding bandwidth resource competition caused by establishing two links simultaneously. A switching window is also reserved for smooth switching, avoiding sudden changes during video switching and not affecting the user's visual perception.

[0062] Optionally, the target video stream is an H264 video stream, which can better adapt to the network code of the communication network, better adapt to the network characteristics of the communication network and reduce resource consumption caused by real-time encoding and decoding.

[0063] It is understandable that the present invention adopts a non-real-time encoding and decoding method. When the network fluctuation information of the communication network reaches a preset threshold, the target video bit rate is determined according to the network fluctuation information and the key frame position is determined according to the playback progress of the current video stream. The key frame position is used as a switching point to switch the current video stream to the target video stream corresponding to the target video bit rate, thereby reducing the bandwidth resource consumption caused by real-time encoding and decoding. When the network status of the communication network changes, adaptive playback of the bit rate can be achieved to avoid audio and video playback jams. The target video bit rate is determined according to the changes in network fluctuations in the communication network, and video streams with different bit rates can be dynamically switched to play, thereby avoiding problems such as video playback jams, jumps, screen distortion, and audio and video asynchrony caused by network fluctuations.

[0064] Based on the above embodiment, as an optional embodiment, before obtaining the network fluctuation information of the communication network during the playback of the current video stream, the method further includes:

[0065] In response to an incoming IMS call request initiated by a client, media negotiation is performed with the client to determine a current playback bit rate of the client; a video stream corresponding to the current playback bit rate is selected and the video stream is transmitted to the client.

[0066] Specifically, when a user initiates a call access on the IMS network through a client, an INVITE request is made via the Session Initiation Protocol (SIP). This request includes a Session Description Protocol (SDP) message, which contains specific media information. The fmtp (format parameters) field in the SDP message is used to specify the specific parameters of the media format. Compared to the existing SDP in the upgrade message initiated by the call platform, which only carries a single RTP payload (a field indicating the load in the protocol), the SDP in the upgrade message initiated by the call platform in this application carries two RTP payloads, each with its own fmtp. The AS (application maximum bandwidth, indicating the bandwidth of a single media at a single location) value is also increased. The client can respond with the corresponding SDP message based on its own situation. The media parsing module in the call platform can determine the client's currently supported bit rate, resolution, and level (a field used to indicate the video level in the H264 standard) based on the user's SDP message, thereby better selecting the target video stream for playback. This application modifies the SDP message and adds signaling negotiation content to achieve dynamic bit rate negotiation, thereby providing better video services. Depending on the call access method, the specific implementation principle and process are as follows:

[0067] Figure 2 This is a diagram of the video call incoming call and signaling negotiation process. When a user makes an incoming call via a video call, the incoming call and signaling negotiation process is as follows: Figure 2 As shown. The H264 standard uses levels to constrain resolution, frame rate and bit rate. The higher the level, the higher the bit rate, resolution and fps (frame rate of the video). The H264 standard document specifically divides the H264 levels from 1 to 5.1. This application mainly negotiates level 3.1 and level 3.0 during media negotiation. The SDP message in the user's incoming video call will carry multiple H264 levels. If the user only carries one, one will be passed through. After the media processing module receives the INVITE, it will perform encoding negotiation. First, it will determine which H264 levels the user supports, and then it will select the highest level for negotiation, and reply to the corresponding AS based on the AS (field that identifies the video bit rate) carried by the other party. For example, if the other party carries 952, it will reply 952. Similarly, the media processing module will record the negotiation results, and when playing the video, it will select the corresponding video stream for playback based on the negotiation results.

[0068] Figure 3This is a diagram of the audio call incoming call and signaling negotiation process. When a user makes an incoming call via an audio call, the incoming call and signaling negotiation process is as follows: Figure 3 As shown. After the user's audio calls into the call platform, the video self-service module initiates the upgrade of the one-way video. The media processing module will carry two levels of H264 and send an UPDATE message to the call platform access point, and the AS field in the SDP is the updated value. After the call platform access point receives the UPDATE message sent by the media processing module, it transparently transmits the two levels and AS values ​​to the user. The client selects a suitable level and AS according to its own situation, and transparently transmits it to the media processing module through the call platform access point. After the media processing module receives the user's 200OK message, it records the bit rate (AS), level and resolution supported by the current user according to the negotiation results. Then, when playing the video, the corresponding video stream is selected for playback based on the results of the negotiation record.

[0069] When playing a video file, the media processing module obtains all video tracks in the video file. If there is only one track, it directly selects this track for playback (compatible with old video files). If there are multiple tracks, it selects based on the stored video level, resolution, and AS bit rate value.

[0070] This application includes, but is not limited to, multi-bitrate negotiation for the two access scenarios mentioned above. It also supports application expansion in more scenarios. However, its core concept is to implement multi-bitrate negotiation by modifying the SDP message in signaling, thereby providing a multi-bitrate negotiated video playback mechanism. Regardless of the access method and application scenario, the following principles must be followed when performing multi-bitrate negotiation:

[0071] 1. Regardless of whether the UPDATE is in bridge or non-bridge mode, and whether it is a one-way (request initiated by either the client or the server) signaling UPDATE or a two-way (request initiated by both the client and the server) UPDATE, each call to the UPDATE interface will carry multiple levels of H264 for the call, including one-way UPDATE before the call is answered and one-way and two-way UPDATE after the call is answered.

[0072] 2. If channel A has negotiated a level, when calling channel B through channel A, channel B will only carry the level of channel A. In bridge mode, if channels A and B are negotiating a level, channels A and B will transparently transmit each other's level.

[0073] Correspondingly, the current video stream in step S100 may be a video stream determined by media negotiation before the call between the client and the server is established, or may be a video stream adjusted according to network fluctuations during the call.

[0074] It is understandable that this application adds a multi-bitrate negotiation mechanism before the call is established, establishes the optimal level of media playback based on the results of the multi-bitrate negotiation, and plays the video stream with the corresponding bitrate, effectively ensuring the effective transmission of media streams (including audio and video streams) under different network conditions. Compared with the non-negotiation mechanism of the existing technology, this application selects the optimal video level based on the video bitrate supported by the user. In terms of initial video playback bitrate selection, this application is more reasonable, which can reduce the switching problems caused by video bitrate incompatibility with the network and terminal.

[0075] Based on the above embodiment, as an optional embodiment, before responding to the IMS incoming call request initiated by the client, the method further includes:

[0076] Obtain the media resources to be played and eliminate the bidirectional frames in the media resources; specifically, since the bidirectional frames, i.e., B frames, need to refer to the following frames in order to be decoded during the real-time transmission and playback of the video, they are highly dependent on the network environment. If the network is poor, packet loss, retransmission, and delays will slow down the video decoding speed. In addition, the encoding of B frames is also relatively complicated, and image rearrangement is required before decoding and output display, resulting in a large consumption of resources. In order to improve transmission performance, the present application pre-processes and stores the media resources to be played, first eliminating the B frames in the media resources, and then placing the audio stream in the media resources and the video stream encoded and decoded into different bit rates in different tracks of a unified video file to facilitate adaptive playback.

[0077] The eliminated media resources are encoded to obtain an encoded video of at least one bit rate and an H264 video stream corresponding to the encoded video, and the encoded video and the H264 are merged to obtain a video stream; specifically, 2PASS, i.e., variable bit rate compression technology, can be used to analyze and compress the video twice to obtain encoded videos of different levels of bit rate and resolution, and extract the H264 video stream from the encoded video.

[0078] Each video stream is associated with its corresponding video stream track and the audio of one of the video streams is retained. Specifically, the audio of the first video stream can be retained and the audio of the other video streams can be discarded. It can be seen that the video ringtone to be played uniformly uses a single audio file with multiple video tracks. All video streams can be combined to form a video file.

[0079] The frame rates of all the video streams are the same. The frame rates of all the video streams are unified into a fixed frame rate format, and the number of frames in each group of pictures (GOP) is also the same.

[0080] It can be understood that this application adopts a non-encoding method to switch freely between different tracks in the same video file when switching video bit rates. The advantage of this processing is that it reduces the calculation and transmission pressure caused by real-time encoding during media transmission, while also reducing resource consumption and memory usage, and simplifying the complexity of video switching calculations.

[0081] Based on the above embodiment, as an optional embodiment, the video dynamic bit rate playback method provided by the present application further includes:

[0082] In response to the key frame request message fed back by the client, determine whether the frequency and / or number of received key frame request messages reaches a set threshold; if so, determine the target video bit rate according to the network fluctuation information and determine the key frame position according to the playback progress of the current video stream, and play the target video stream corresponding to the target video bit rate at the key frame position.

[0083] Optionally, the SIP protocol is used in the IMS network to establish the corresponding call. The SIP protocol requires the support of some network protocols in the process of managing the entire session, such as the SDP protocol and SIP for media negotiation, the RTP protocol for real-time media transmission, and the RTCP protocol for real-time transmission control of the RTP service quality to ensure the quality of audio and video. This application is based on the RTP / RTCP protocol to monitor the network fluctuation status in the communication network in real time, thereby better controlling the video bit rate for playback and transmission. The specific implementation principle is as follows:

[0084] In one embodiment, network status fluctuations are monitored in real time through a NACK mechanism.

[0085] Figure 4 This is a flow chart of the timeout retransmission mechanism. In real-time audio and video transmission, packet loss is a common problem and can also be used to measure the network status of the communication network. The network monitoring element in this application introduces a NACK (Negative Acknowledgement, timeout retransmission) mechanism to request retransmission of RTP packets and monitor the network packet loss rate by parsing NACK data. The NACK mechanism is implemented as follows Figure 4 shown.

[0086] During audio and video playback, media data is sent to the client via RTP packets in the form of data packets. Under poor network conditions, packets received by the client may be lost. A lower packet loss rate indicates a worse network condition. The client determines whether packet loss has occurred and the severity of the loss by analyzing the sequence number in the RTP protocol header. Under normal circumstances, data packets arrive at the client in an orderly manner. That is, the sequence numbers of the RTP packet currently received by the client and the largest packet recently received increase continuously. If the sequence number of the RTP packet currently received by the client is lower than the sequence number of the largest packet recently received, it indicates that the received packet has been delayed. If the sequence number of the RTP packet currently received by the client is higher than the sequence number of the largest packet recently received, it indicates that packet loss has occurred. A larger difference indicates more packet loss. When the client detects packet loss, it sends a NACK request message to the server for retransmission, along with the sequence number of the lost data packet. The server can monitor network changes in real time based on the number of NACK requests and the amount of packet loss.

[0087] In one embodiment, the client bitrate change is monitored by analyzing the client feedback media bitrate.

[0088] The NACK mechanism can monitor network changes in real time, but the target video bitrate cannot be accurately determined. Therefore, this application proposes another mechanism to assist in more accurately predicting the target video bitrate. Based on its own network status, the client will feedback the currently supported bitrate to the server through the RTCP message after receiving the RTP data packet. The server can obtain the current bitrate by parsing this field and use it to predict the target bitrate.

[0089] In one embodiment, network status fluctuations are monitored by analyzing the number of key frame requests from the client.

[0090] When network congestion increases, more media packets are lost, leading to client decoding failures and resulting in distorted, blue, or pixelated screens. Therefore, clients often request key frames to address these issues. These requests are primarily made through PLI (Picture Loss Indication) and FIR (Full Intra Request). Regardless of the method used, as long as the server detects key frame request data packets, the frequency and number of received packets can indicate network congestion and trigger video switching in a timely manner.

[0091] It can be understood that this application monitors and analyzes media messages in the communication network in real time, dynamically tracks user network changes and reports them in real time, and takes effective measures dynamically according to network fluctuations to ensure the user's video viewing experience.

[0092] Figure 5 This is the second flow chart of the video dynamic bit rate playback method provided by this application, such as Figure 5 As shown, based on the above embodiment, as an optional embodiment, determining the target video bit rate according to the network fluctuation information includes:

[0093] Step S210, determining the packet loss rate and / or delay of the current video based on the network fluctuation information; the network fluctuation information includes packet loss rate, delay, freeze rate and network traffic conditions, etc. The packet loss rate and / or delay of the current video can be extracted from the network fluctuation information.

[0094] In step S220, the current playback bit rate of the client is determined based on the currently supported video bit rate fed back by the client. Specifically, the client, based on its own network status, will feed back the currently supported bit rate size to the server through an RTCP message after receiving the RTP data packet, and use the currently supported video bit rate as the current playback bit rate of the client.

[0095] Step S230: determining the target video bit rate according to the packet loss rate and / or the delay and the current playback bit rate.

[0096] Optionally, determining the target video bit rate according to the packet loss rate and / or the delay and the current playback bit rate includes:

[0097] Step S231: If the packet loss rate and / or the delay meet the preset bit rate increase condition, the current playback bit rate is increased based on a preset bit rate gain adjustment factor to obtain the target video bit rate;

[0098] Step S232: If the packet loss rate and / or the delay meet the preset bit rate reduction condition, the current playback bit rate is reduced based on a preset bit rate reduction adjustment factor to obtain the target video bit rate;

[0099] Otherwise, the current playback bit rate is used as the target video bit rate.

[0100] This application predicts the target video bitrate based on the client's current playback bitrate and network status. Two methods, namely, packet loss-based target bitrate prediction and latency-based target bitrate prediction, are used as examples. The packet loss-based target video bitrate is calculated as follows:

[0101]

[0102] Among them, B n Denotes the predicted target bit rate, B c Indicates the current playback bitrate of the client, α indicates the bitrate gain adjustment factor, and β indicates the bitrate loss adjustment factor. The value ranges of α and β are [0.05, 0.08]. I Indicates the packet loss rate, P I = (Number of lost packets / Number of sent packets) * 100%. If the packet loss rate is less than 2%, the network is in good condition and the bandwidth has not reached the transmission limit. Therefore, the current bitrate can be increased by the bitrate gain adjustment factor α. If the packet loss rate is greater than 2% and less than 10%, the network is slightly poor, but still sufficient for playback at the current bitrate, so no bitrate switching is performed. If the packet loss rate is greater than 10%, the network is congested and a bitrate switching is required. The target bitrate is then reduced by the bitrate loss adjustment factor β.

[0103] The target bit rate prediction based on packet loss can predict the target bit rate through the packet loss rate, but once the bit rate exceeds 10%, it means that congestion may have occurred, and the recovery of the congested network takes some time, and the video bit rate cannot be adjusted in time. In addition, when congestion occurs, switching the bit rate will cause video freezes due to untimely switching. Therefore, in order to achieve smoother switching, the bit rate switching process can be triggered when the packet loss rate reaches 8%, and a buffer transition time can be reserved to achieve the effect of smooth switching. When the packet loss rate is greater than 10%, this application proposes a target bit rate prediction based on delay to ensure that the recovery adjustment after network congestion is more timely. The target bit rate prediction based on delay is calculated as follows:

[0104] When the server-side received delay is decreasing, it means that the network status is improving. The target video bitrate is predicted as follows:

[0105] B n =(1+α)*B c ;

[0106] If the server's received latency is still increasing, indicating that the network status is deteriorating, the target video bitrate is predicted as follows:

[0107] B n =(1-δ)*B c ;

[0108] Wherein, δ represents the bit rate loss adjustment factor under worse network conditions, and its value is [0.1, 0.15].

[0109] It is understandable that the multi-bitrate negotiation mechanism provided by this application can dynamically select the target video playback based on the video bitrate supported by the user. However, if the user's network status fluctuates during a call, the initially negotiated video level may no longer be appropriate, resulting in a poor video playback experience. Therefore, in order to dynamically adapt to the user's network fluctuations, this application can also switch the bitrate in real time based on the user's network fluctuations to ensure video playback quality.

[0110] Based on the above embodiment, as an optional embodiment, determining a key frame position according to the playback progress of the current video stream and playing a target video stream corresponding to the target video bit rate at the key frame position includes:

[0111] Determine whether the current frame is a key frame according to the playback progress of the current video stream; if so, use the current frame as the key frame position; otherwise, use the position of the key frame closest to the current frame as the key frame position;

[0112] Switch to the video stream track corresponding to the target video bit rate at the key frame position and play the target video stream.

[0113] Specifically, confirm the playback progress of the current video stream and determine whether the current frame is a key frame. If it is a key frame, retrieve the target bit rate video and switch directly to the key frame position corresponding to the target video for switching. Otherwise, determine the position of the next closest key frame and calculate the time required for the current video to play to the next key frame. Since this application adopts a fixed frame rate playback mode and the video to be played has been pre-processed, it is easy to determine the position and playback time of the current frame and subsequent key frames based on the playback time, etc. The reason for switching at the key frame moment is to avoid switching at non-key frame moments, which will cause a flowery screen or a black screen, which will affect the viewing experience and other image quality.

[0114] After determining the switching point of the video stream by calculating the current video stream, when the current video stream playback progress reaches the switching moment, the video stream track of the current bit rate is immediately switched to the target bit rate video stream track, and the audio file is played on the same track.

[0115] The calculation formula for the current frame playback progress is:

[0116] F q1 =T1×F r ;

[0117] Among them, F q1 Indicates the serial number of the current playing frame, T1 indicates the playing time, F r Indicates the frame rate. The current frame refers to the frame being sent by the server when the video bitrate switch is triggered, using fixed frame rate playback mode.

[0118] The frame sequence number of the next key frame closest to the current frame is:

[0119] F q2 =(T1 / T2+1)×F r ;

[0120] Among them, F q2 Indicates the frame sequence number of the next key frame closest to the current frame. T2 indicates the time required to play each GOP. Because a fixed frame rate is used and the video is processed before playback, the number of frames in each GOP is the same.

[0121] The time from the trigger switching moment to the switching point is:

[0122] T3=(F q2 -F q1 )×F r .

[0123] The key frame in the H264 video stream contains the complete information in the GOP. If the switching point is not from the key frame, the video information will be lost. Therefore, this application uses the method of finding the key frame as the switching point for switching.

[0124] Figure 6 This is a schematic diagram of bitrate switching under different video stream tracks, such as Figure 6 As shown in the figure, when network fluctuations are detected during video playback and the fluctuations reach the switching threshold, the bitrate switching process is triggered, and the nearest keyframe is found as the switching point to switch to the target video bitrate track for playback. If the network changes again after the video is switched and the switching threshold is reached, the bitrate switching continues smoothly. If the network changes again before the switching point is reached after the bitrate switch is triggered, the switch is terminated first. If the network recovers, the original video track continues to play. If the network deteriorates, the target bitrate is re-estimated and switched. During the video bitrate switching process, switching between different tracks in the same video file is always maintained.

[0125] As can be understood, this application can accurately predict network congestion and dynamically adjust the video bitrate more promptly after congestion occurs, achieving smooth dynamic video switching. Only a single link is required to switch the video bitrate, avoiding the bandwidth resource competition caused by establishing two links simultaneously. A switching window is also reserved for smooth switching, avoiding sudden changes during video switching and not affecting the user's visual perception.

[0126] The video dynamic bit rate playback device provided by the present application is described below. The video dynamic bit rate playback device described below and the video dynamic bit rate playback method described above can be referenced to each other.

[0127] Figure 7This is a block diagram of the principle of the video dynamic bit rate playback device provided by this application, such as Figure 7 As shown, the present application also provides a video dynamic bit rate playback device, comprising:

[0128] The network dynamic monitoring module 710 is used to obtain network fluctuation information of the communication network during the playback of the current video stream and determine whether the network fluctuation information reaches a preset threshold;

[0129] A bitrate switching determination module 720 is configured to determine a target video bitrate based on the network fluctuation information and a keyframe position based on the playback progress of the current video stream when the network fluctuation information reaches a preset threshold;

[0130] The media processing module 730 is configured to play the target video stream corresponding to the target video bit rate at the key frame position.

[0131] In one embodiment, the video dynamic bit rate playback device further includes a media resource storage module 740 .

[0132] The steps of implementing the video dynamic bit rate playback method based on the video dynamic bit rate playback device are as follows:

[0133] (1) The client initiates an incoming IMS call request. The media processing module negotiates multiple bitrates with the terminal, establishes a playback link, and selects the media resource with the optimal bitrate for playback and transmission based on the client's network conditions. Media resources include video streams, which are the same below.

[0134] (2) The network dynamic monitoring module will monitor the network status during the playback process in real time and return the network fluctuation situation to the media processing module.

[0135] (3) The media processing module determines whether the network threshold for video switching is reached based on network fluctuations. If so, the bit rate switching calculation of the media resource is triggered.

[0136] (4) The bit rate switching calculation module calculates the bit rate and position of the switching target video based on the current network status and triggering time, and then returns the result to the media processing module.

[0137] (5) The media processing module selects the target media resource according to the bit rate switching calculation result to perform dynamic switching of media playback.

[0138] This application is carried out in only one communication link throughout the entire process, ensuring seamless connection of audio and video frames of different bit rates while avoiding the waste of resources caused by establishing multiple links. In addition, this application adopts a non-decoding method, pre-storing videos of different bit rates in the media resource storage module. When playback and transmission are required, the H264 stream of the required target bit rate video file is directly sent out, which can better adapt to the network code of the communication network.

[0139] In one embodiment, the media resource storage module is used to preprocess and store the media resources to be played. First, the B frames in the media resources are eliminated, and then the media resources and videos encoded and decoded into different bit rates are placed in different tracks of the same file to facilitate adaptive playback.

[0140] In one embodiment, the network dynamic monitoring module 710 is used to monitor network status fluctuations in real time through the NACK mechanism, monitor client bitrate changes by analyzing client feedback media bitrate, and monitor network status fluctuations by analyzing the number of client key frame requests, thereby better controlling the video bitrate playback transmission. The specific implementation principles are as follows:

[0141] 1. Real-time monitoring of network status fluctuations through the NACK mechanism.

[0142] Packet loss is a common problem in real-time audio and video transmission and can also be used to measure the network status. This proposal introduces a Negative Acknowledgement (NACK) mechanism into the network monitoring component to request retransmission of RTP packets. NACK data is then parsed to monitor the network's packet loss rate.

[0143] During audio and video playback, media data is sent to the client via RTP packets in the form of data packets. Under poor network conditions, packets received by the client may be lost. A lower packet loss rate indicates a worse network condition. The client determines whether packet loss has occurred and the severity of the loss by analyzing the sequence number in the RTP protocol header. Under normal circumstances, data packets arrive at the client in an orderly manner. That is, the sequence numbers of the RTP packet currently received by the client and the largest packet recently received increase continuously. If the sequence number of the RTP packet currently received by the client is lower than the sequence number of the largest packet recently received, it indicates that the received packet has been delayed. If the sequence number of the RTP packet currently received by the client is higher than the sequence number of the largest packet recently received, it indicates that packet loss has occurred. A larger difference indicates more packet loss. When the client detects packet loss, it sends a NACK request message to the server for retransmission, along with the sequence number of the lost data packet. The server can monitor network changes in real time based on the number of NACK requests and the amount of packet loss.

[0144] 2. Monitor client bitrate changes by analyzing the media bitrate feedback from the client.

[0145] The NACK mechanism allows for real-time monitoring of network changes, but the target video bitrate cannot be accurately determined. Therefore, this proposal proposes another mechanism to more accurately predict the target video bitrate. Based on its network status, the terminal sends the server a feedback message in the RTCP message regarding the currently supported bitrate after receiving the RTP packet. The server then parses this field to determine the current bitrate and uses it to predict the target bitrate.

[0146] 3. Monitor network status fluctuations by analyzing the number of client key frame requests.

[0147] When network congestion increases, more media packets are lost, leading to terminal decoding failures and resulting in distorted, blue, or pixelated screens. Therefore, terminals use keyframe requests to address these issues. These requests are primarily made using PLI (Picture Loss Indication) and FIR (Full Intra Request). Regardless of which method is used, as long as the server detects keyframe request data packets, the frequency and number of received packets can indicate network congestion and trigger video switching in a timely manner.

[0148] The above is the working mechanism and principle of the network dynamic monitoring module. Combining the above solutions can effectively monitor network fluctuations and adjust the video playback bit rate in time according to the network status.

[0149] In summary, this application addresses the problem of poor audio and video playback experience in communication network conditions due to network fluctuations. From the aspects of signaling media negotiation, network dynamic monitoring, smooth switching of video bitrate, and media resource storage, this application proposes a high-performance video bitrate adaptive playback device in an IMS network.

[0150] 1. This application implements dynamic bit rate video playback in the IMS network, filling the problem of insufficient application of existing technologies in communication networks.

[0151] 2. This application implements a multi-bitrate negotiation mechanism, which selects the optimal video level based on the video bitrate supported by the user. Compared with the non-negotiation mechanism in the existing technology, this application is more reasonable in selecting the initial video playback bitrate, and can reduce switching problems caused by incompatibility between the video bitrate and the network and terminal.

[0152] 3. This application implements a dynamic monitoring mechanism for the IMS network, which can accurately monitor user network fluctuations in real time. Compared with the existing technology, the network dynamic monitoring element of this solution can monitor the communication network and monitor network changes more accurately and in real time.

[0153] 4. The target bit rate prediction method implemented in this application can accurately predict whether the network is congested, and the dynamic adjustment of the video bit rate after congestion is more timely.

[0154] 5. The dynamic video smooth switching technology implemented in this application only requires a single link to switch the video bit rate, avoiding the bandwidth resource competition caused by establishing two links simultaneously. A switching window is also reserved for smooth switching, avoiding sudden changes during the video switching process and not affecting the user's visual perception.

[0155] 6. This application uses a non-real-time encoding and decoding method to directly send the H264 stream in the video file to the client. The purpose of doing so is to better adapt to the network characteristics of the communication network and reduce the resource consumption caused by real-time encoding and decoding.

[0156] This application is mainly aimed at the application implementation of the dynamic bit rate playback scenario of video ringback tone before the audio and video call is connected through IMS in the IMS network. It can not only realize the dynamic selection and playback of video and the dynamic switching of bit rate of adaptive network, but also use non-real-time encoding and decoding to reduce network resource consumption and processing pressure of media processing modules, effectively solving the current technical bottleneck of dynamic switching of video bit rate in communication networks and improving the service quality of video calls.

[0157] This application realizes dynamic video bit rate playback in the communication network through a media negotiation mechanism, a communication network dynamic monitoring element, a media resource storage device, and a video bit rate dynamic switching device. It is applicable to all communication network platforms called through IMS. Some modules can also be applied separately to media resource playback scenarios in communication networks and the Internet.

[0158] Figure 8 An example of a physical structure diagram of an electronic device is shown below. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the video dynamic bit rate playback method, which includes:

[0159] Get the network fluctuation information of the communication network during the playback of the current video stream;

[0160] If the network fluctuation information reaches a preset threshold, determining a target video bit rate according to the network fluctuation information and determining a key frame position according to the playback progress of the current video stream;

[0161] A target video stream corresponding to the target video bit rate is played at the key frame position.

[0162] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0163] On the other hand, the present application further provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the video dynamic bit rate playback method provided by the above methods, which includes:

[0164] Get the network fluctuation information of the communication network during the playback of the current video stream;

[0165] If the network fluctuation information reaches a preset threshold, determining a target video bit rate according to the network fluctuation information and determining a key frame position according to the playback progress of the current video stream;

[0166] A target video stream corresponding to the target video bit rate is played at the key frame position.

[0167] In another aspect, the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for playing a video at a dynamic bit rate provided by the above methods is implemented. The method includes:

[0168] Get the network fluctuation information of the communication network during the playback of the current video stream;

[0169] If the network fluctuation information reaches a preset threshold, determining a target video bit rate according to the network fluctuation information and determining a key frame position according to the playback progress of the current video stream;

[0170] A target video stream corresponding to the target video bit rate is played at the key frame position.

[0171] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0172] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A video dynamic bit rate playback method, characterized in that: include: Get the network fluctuation information of the communication network during the playback of the current video stream; If the network fluctuation information reaches a preset threshold, determining a target video bit rate according to the network fluctuation information and determining a key frame position according to the playback progress of the current video stream; A target video stream corresponding to the target video bit rate is played at the key frame position.

2. The video dynamic bit rate playback method according to claim 1, characterized in that: The determining the target video bit rate according to the network fluctuation information includes: Determining the packet loss rate and / or delay of the current video according to the network fluctuation information; Determine the current playback bit rate of the client according to the currently supported video bit rate fed back by the client; The target video bit rate is determined according to the packet loss rate and / or the delay and the current playback bit rate.

3. The video dynamic bit rate playback method according to claim 2, characterized in that: The determining the target video bit rate according to the packet loss rate and / or the delay and the current playback bit rate includes: If the packet loss rate and / or the delay meet the preset bit rate increase condition, the current playback bit rate is increased and adjusted based on a preset bit rate gain adjustment factor to obtain the target video bit rate; If the packet loss rate and / or the delay meet the preset bit rate reduction conditions, the current playback bit rate is reduced based on a preset bit rate reduction adjustment factor to obtain the target video bit rate; Otherwise, the current playback bit rate is used as the target video bit rate.

4. The video dynamic bit rate playback method according to claim 1, characterized in that: Before obtaining the network fluctuation information of the communication network during the playback of the current video stream, the method further includes: In response to an incoming IMS call request initiated by a client, performing media negotiation with the client to determine a current playback bitrate of the client; A video stream corresponding to the current playback bit rate is selected and transmitted to the client.

5. The video dynamic bit rate playback method according to claim 4, characterized in that: Before responding to the IMS incoming call request initiated by the client, the method further includes: Acquire a media resource to be played, and eliminate bidirectional frames in the media resource; Encoding the eliminated media resources to obtain an encoded video of at least one bit rate and an H264 video stream corresponding to the encoded video, and merging the encoded video and the H264 to obtain a video stream; Associating each of the video streams to a corresponding video stream track and retaining the audio of one of the video streams; Wherein, the frame rates of all the video streams are the same.

6. The video dynamic bit rate playback method according to claim 5, characterized in that: The determining of a key frame position according to the playback progress of the current video stream, and playing a target video stream corresponding to the target video bit rate at the key frame position, includes: Determine whether the current frame is a key frame according to the playback progress of the current video stream; if so, use the current frame as the key frame position; otherwise, use the position of the key frame closest to the current frame as the key frame position; Switch to the video stream track corresponding to the target video bit rate at the key frame position and play the target video stream.

7. The video dynamic bit rate playback method according to claim 1, characterized in that: Also includes: In response to the key frame request message fed back by the client, determine whether the frequency and / or number of received key frame request messages reaches a set threshold; if so, determine the target video bit rate according to the network fluctuation information and determine the key frame position according to the playback progress of the current video stream, and play the target video stream corresponding to the target video bit rate at the key frame position.

8. A video dynamic bit rate playback device, characterized in that: include: A network dynamic monitoring module is used to obtain network fluctuation information of the communication network during the playback of the current video stream and determine whether the network fluctuation information reaches a preset threshold; a bit rate switching determination module, configured to determine a target video bit rate according to the network fluctuation information and determine a key frame position according to the playback progress of the current video stream when the network fluctuation information reaches a preset threshold; The media processing module is used to play the target video stream corresponding to the target video bit rate at the key frame position.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the video dynamic bit rate playback method as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video dynamic bit rate playback method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the video dynamic bit rate playback method according to any one of claims 1 to 7 is implemented.

Images