Method and apparatus for adjusting code rate, electronic device, and computer program product

Abstract

The present disclosure relates to a code rate adjustment method and device, electronic equipment and computer program product, belonging to the technical field of computers. The method comprises: obtaining a sending code rate of a current video frame when it is sent; obtaining a maximum available code rate of a media stream according to code rate statistical information in a historical statistical time period and the sending code rate of the current video frame; determining an expected code rate when a next video frame is sent according to the maximum available code rate of the media stream; adjusting the expected code rate according to a preset delay parameter, and controlling an encoder to encode the next video frame according to the adjusted expected code rate. The present disclosure adjusts the sending code rate of the video frame by comprehensively considering the anti-packet loss requirement and the delay control requirement in the real-time transmission scenario, which can improve the delay control capability during code rate adaptation and optimize the real-time communication experience.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular, to a code rate adjustment method, a code rate adjustment device, an electronic device and a computer program product. BACKGROUND

[0002] In the scenario of real-time communication, the bandwidth resource of the network is limited. When the network congestion detection algorithm responds to the bandwidth change by adjusting the sending rate, the real-time communication system also needs to quickly adjust the production rate of the media source (audio, video) to adapt to the change of the sending rate, so as to optimize the communication experience, including clarity, delay, smoothness, etc. For example, in the scenario of cloud game, the industry standard suggests that the interactive delay should be controlled within 100 milliseconds to obtain better user experience.

[0003] With the growing demand of real-time communication, cloud game and other applications, the delay control capability of the current code rate adaptation scheme is not accurate enough, and the above-mentioned demand cannot be effectively solved.

[0004] Therefore, there is an urgent need in the art for a code rate adjustment method that can improve the delay control capability during code rate adaptation and optimize the real-time communication experience. SUMMARY

[0005] The purpose of the present disclosure is to provide a code rate adjustment method, a code rate adjustment device, an electronic device and a computer program product, which can at least to some extent improve the delay control capability during code rate adaptation and optimize the real-time communication experience.

[0006] According to a first aspect of the present disclosure, a code rate adjustment method is provided, comprising:

[0007] obtaining a sending code rate of a current video frame when it is sent;

[0008] obtaining a maximum available code rate of a media stream according to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame;

[0009] determining an expected code rate when a next video frame is sent according to the maximum available code rate of the media stream;

[0010] adjusting the expected code rate according to a preset delay parameter, and controlling an encoder to encode the next video frame according to the adjusted expected code rate.

[0011] In an exemplary embodiment of the present disclosure, the obtaining a maximum available code rate of a media stream according to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame comprises:

[0012] According to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame, a maximum sending code rate of the media stream is obtained.

[0013] According to the maximum sending code rate of the media stream, a maximum available code rate of the media stream is obtained.

[0014] In an exemplary embodiment of the present disclosure, the obtaining of the maximum sending code rate of the media stream according to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame comprises:

[0015] According to the media average code rate, the forward error correction average code rate, the arrival confirmation average code rate, the total sending code rate in the historical statistical time period and the sending code rate of the current video frame, the maximum sending code rate of the media stream is obtained.

[0016] In an exemplary embodiment of the present disclosure, the obtaining of the maximum available code rate of the media stream according to the maximum sending code rate of the media stream comprises:

[0017] The buffer statistical information is obtained, and an expected buffer emptying rate is obtained according to the buffer statistical information.

[0018] According to the maximum sending code rate and the expected buffer emptying rate, the maximum available code rate of the media stream is obtained.

[0019] In an exemplary embodiment of the present disclosure, the obtaining of the expected buffer emptying rate according to the buffer statistical information comprises:

[0020] According to the ratio of the sending buffer value in the buffer statistical information to the expected buffer emptying time length, the expected buffer emptying rate is obtained.

[0021] In an exemplary embodiment of the present disclosure, the determining of the expected code rate when the next video frame is sent according to the maximum available code rate of the media stream comprises:

[0022] The expected forward error correction redundancy ratio is obtained, and the expected code rate when the next video frame is sent is obtained according to the maximum available code rate of the media stream and the forward error correction redundancy ratio.

[0023] In an exemplary embodiment of the present disclosure, the adjusting of the expected code rate according to the preset delay parameter comprises:

[0024] The video frame parameter of the maximum video frame in the historical sending process is obtained.

[0025] According to the video frame parameter of the maximum video frame and the preset delay parameter, the expected code rate after adjustment is obtained.

[0026] According to a second aspect of the present disclosure, there is provided a device for adjusting a code rate, comprising:

[0027] a current code rate obtaining module configured to obtain a sending code rate of a current video frame when sent;

[0028] a maximum code rate determining module configured to determine a maximum available code rate of a media stream according to code rate statistical information in a historical statistical time period and the sending code rate of the current video frame;

[0029] a desired code rate determining module configured to determine a desired code rate when a next video frame is sent according to the maximum available code rate of the media stream;

[0030] a desired code rate adjusting module configured to adjust the desired code rate according to a preset delay parameter, and control an encoder to encode the next video frame according to the adjusted desired code rate.

[0031] In an exemplary embodiment of the present disclosure, the maximum code rate determining module comprises:

[0032] a maximum sending code rate determining unit configured to determine a maximum sending code rate of the media stream according to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame;

[0033] a maximum available code rate determining unit configured to determine the maximum available code rate of the media stream according to the maximum sending code rate of the media stream.

[0034] In an exemplary embodiment of the present disclosure, the maximum sending code rate determining unit comprises:

[0035] a maximum sending code rate calculating unit configured to determine the maximum sending code rate of the media stream according to a media average code rate, a forward error correction average code rate, an acknowledgement average code rate, a total sending code rate in the historical statistical time period, and the sending code rate of the current video frame.

[0036] In an exemplary embodiment of the present disclosure, the maximum available code rate determining unit comprises:

[0037] a buffer emptying rate determining unit configured to obtain buffer statistical information, and determine a desired buffer emptying rate according to the buffer statistical information;

[0038] a maximum available code rate calculating unit configured to determine the maximum available code rate of the media stream according to the maximum sending code rate and the desired buffer emptying rate.

[0039] In an exemplary embodiment of the present disclosure, the buffer emptying rate determining unit comprises:

[0040] The cache emptying rate calculation unit is configured to obtain the expected cache emptying rate according to a ratio of the sending cache value to the expected cache emptying time length in the cache statistical information.

[0041] In an exemplary embodiment of the present disclosure, the expected code rate determination module comprises:

[0042] The expected code rate calculation unit is configured to obtain an expected forward error correction redundancy ratio, and obtain the expected code rate at the time of sending the next video frame according to the maximum available code rate of the media stream and the forward error correction redundancy ratio.

[0043] In an exemplary embodiment of the present disclosure, the expected code rate adjustment module comprises:

[0044] The maximum video frame parameter acquisition unit is configured to obtain the video frame parameter of the maximum video frame in the historical sending process.

[0045] The expected code rate adjustment unit is configured to obtain the adjusted expected code rate according to the video frame parameter of the maximum video frame and a preset delay parameter.

[0046] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the code rate adjustment method of any of the above.

[0047] According to a fourth aspect of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the code rate adjustment method of any of the above.

[0048] The exemplary embodiments of the present disclosure can have the following beneficial effects:

[0049] In the method for adjusting the code rate in the example embodiment of the present disclosure, the maximum available code rate of the media stream is obtained by the sending code rate of the current video frame when being sent and the code rate statistical information in the historical statistical time period, then the expected code rate when sending the next video frame is determined according to the maximum available code rate of the media stream, and then the expected code rate is adjusted according to the preset delay parameter, and the encoder is controlled to encode the next video frame according to the adjusted expected code rate. In the method for adjusting the code rate in the example embodiment of the present disclosure, on the one hand, by calculating the maximum available code rate of the media stream, the anti-packet loss requirement in the real-time transmission scenario can be met, and the problems of sending end buffer accumulation and coding frame size fluctuation are solved; on the other hand, by adjusting the expected code rate according to the preset delay parameter, the delay control requirement in the real-time transmission scenario can be met, the delay control capability in the code rate adaptation is improved, and the optimal real-time communication experience is realized under the constraint of the delay parameter. In the method for adjusting the code rate in the example embodiment of the present disclosure, by comprehensively considering the anti-packet loss requirement and the delay control requirement in the real-time transmission scenario, a low-delay code rate adaptation scheme is realized, which can control the sending end delay and guarantee the user experience of low-delay services.

[0050] It should be understood that the general description above and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0051] The drawings incorporated in the specification and constituting a part of the specification illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0052] Figure 1 A flowchart of the method for adjusting the code rate in the example embodiment of the present disclosure is shown;

[0053] Figure 2 A flowchart of obtaining the maximum available code rate of the media stream according to the code rate statistical information and the sending code rate of the current video frame in the example embodiment of the present disclosure is shown;

[0054] Figure 3 A flowchart of obtaining the maximum available code rate according to the maximum sending code rate of the media stream in the example embodiment of the present disclosure is shown;

[0055] Figure 4 A flowchart of the method for adjusting the code rate in one specific embodiment of the present disclosure is shown;

[0056] Figure 5A schematic diagram showing a system architecture of rate adjustment in one embodiment of the present disclosure is shown.

[0057] Figure 6 A block diagram of a rate adjustment apparatus in an example embodiment of the present disclosure is shown.

[0058] Figure 7 A structural schematic diagram of a computer system of an electronic device suitable for implementing an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0059] In order for those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings.

[0060] It should be noted that the terms "first", "second", and the like in the description of the present disclosure and the claims and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0061] The following example embodiments can be implemented in various forms, and should not be understood as being limited to the examples set forth herein; rather, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and will fully convey the ideas of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a sufficient understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or can employ other methods, components, devices, steps, etc. In other cases, well-known technical solutions are not shown or described in detail to avoid obscuring the aspects of the present disclosure.

[0062] In addition, the drawings are merely schematic illustrations of the present disclosure, and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus repeated description thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities, and do not necessarily have to correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0063] In an RTC (Realtime Communication) transmission system, flow control is mainly completed by congestion control and code rate adaptation. The congestion control algorithm is used to control the sending rate of the sending end, and the sending controller smooths the sending of data according to the rate. The code rate adaptation algorithm is used to control the generation rate of the upper layer media.

[0064] Due to the characteristics of the video encoding algorithm, the size of the video frame has natural volatility, and the instantaneous encoding code rate may be much higher than the sending rate, resulting in the accumulation of the sending buffer. At the same time, due to the change of the network available bandwidth, the sending rate decreases, and the encoder code control cannot respond in time, also resulting in the accumulation of the sending buffer. In addition, due to the problem of network packet loss, in order to ensure the arrival rate and delay of transmission, a certain rate needs to be consumed to resist packet loss.

[0065] In some related embodiments, the code rate adaptation algorithm in WebRTC (Web Realtime Communication) can be used to adjust the media source rate by the congestion control sending rate, the expected FEC (Forward Error Correction) rate, and the expected retransmission rate. However, the code rate adaptation algorithm in WebRTC only allocates the sending rate to the media, FEC, and retransmission according to historical statistics, and does not fully consider the influence factors such as the fluctuation of the encoded video frame size and the sending buffer, and cannot optimize the communication experience.

[0066] In other related embodiments, the media rate control algorithm based on network queuing delay can be used to adjust the media rate according to the BUR (Bandwidth Utilization Ratio) in MIMD (Multiplicative-Increase / Multiplicative-Decrease) or AIMD (Additive-Increase / Multiplicative-Decrease). However, this method does not consider the bandwidth consumed by the anti-packet loss requirement not met in a weak network, and does not consider the problem of fluctuation of the encoded video frame size, and the algorithm based on queuing delay is easily affected by network jitter, and cannot optimize the communication experience.

[0067] Based on the above problems, the present example embodiment first provides a code rate adjustment method. Referring to Figure 1 The code rate adjustment method can include the following steps:

[0068] Step S110. Obtain the sending code rate of the current video frame when it is sent.

[0069] Step S120. Obtain the maximum available code rate of the media stream according to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame.

[0070] Step S130. Determine the expected code rate when sending the next video frame according to the maximum available code rate of the media stream.

[0071] Step S140. Adjust the expected code rate according to the preset delay parameter, and control the encoder to encode the next video frame according to the adjusted expected code rate.

[0072] The adjustment method of the code rate in the example embodiment of the present disclosure, on the one hand, by calculating the maximum available code rate of the media stream, can meet the anti-packet loss requirement in the real-time transmission scenario, solve the problem of buffer accumulation at the sending end and fluctuation of the encoded frame size; on the other hand, by adjusting the expected code rate according to the preset delay parameter, the delay control requirement in the real-time transmission scenario can be met, the delay control capability when the code rate is adapted is improved, and the optimal real-time communication experience is realized under the constraint of the delay parameter. The adjustment method of the code rate in the example embodiment of the present disclosure, by comprehensively considering the anti-packet loss requirement and the delay control requirement in the real-time transmission scenario, realizes a low-delay code rate adaptation scheme, which can control the delay at the sending end and guarantee the user experience of low-delay services.

[0073] Next, the above steps of the example embodiment will be described in detail. Figures 2 to 5 The above steps of the example embodiment will be described in detail.

[0074] In step S110, the sending code rate of the current video frame when being sent is obtained.

[0075] In the example embodiment, after the current congestion control algorithm updates the sending code rate in the real-time transmission process of the media stream, the current sending code rate is sent to the sending controller, i.e., the sending end of the video frame, and the sending controller sends the current video frame according to the current sending code rate.

[0076] In step S120, the maximum available code rate of the media stream is obtained according to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame.

[0077] In the example embodiment, the maximum available code rate of the currently transmitted media stream can be obtained according to the sending code rate pacing_rate of the sending controller when sending the current video frame and the code rate statistical information in a historical statistical time period.

[0078] In the example embodiment, as Figure 2As shown, the maximum available code rate of the media stream is obtained according to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame, and can include the following steps:

[0079] Step S210. The maximum sending code rate of the media stream is obtained according to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame.

[0080] In the example embodiment, the maximum sending code rate of the media stream can be obtained according to the media average code rate, the forward error correction average code rate, the arrival confirmation average code rate, the total sending code rate in the historical statistical time period, and the sending code rate of the current video frame. The media average code rate app_rate is the average code rate of the media actually sent in the historical statistical time period, the forward error correction average code rate fec_rate is the average code rate of the fec (Forward Error Correction) in the historical statistical time period, and the arrival confirmation average code rate ack_rate is the average code rate of the received confirmation in the historical statistical time period.

[0081] The code rate statistical information in the historical statistical time period can include app_rate (media average code rate), fec_rate (fec average code rate), ack_rate (ack average code rate), and total_sent_rate (total sending code rate). Through the above historical statistical information, the maximum sending code rate of the media stream can be calculated.

[0082] For example, a first difference can be calculated based on the sending code rate pacing_rate and the arrival confirmation average code rate ack_rate, a first ratio can be calculated based on the media average code rate app_rate, the forward error correction average code rate fec_rate, the total sending code rate total_sent_rate, and the arrival confirmation average code rate ack_rate, and then the maximum sending code rate can be obtained according to the first difference and the first ratio.

[0083] For example, the first difference can be represented as pacing_rate-ack_rate, and the first ratio can be represented as The calculation formula of the maximum sending code rate is obtained by the product of the first difference and the first ratio:

[0084] Step S220. The maximum available code rate of the media stream is obtained according to the maximum sending code rate of the media stream.

[0085] In the example embodiment, the maximum available code rate of the media stream can be calculated according to the maximum sending code rate of the media stream and the buffer statistical information. The buffer statistical information refers to the statistical information related to the sending buffer.

[0086] In the example embodiment, as shown in Figure 3 The maximum available rate of the media stream is obtained according to the maximum sending rate of the media stream, which can include the following steps:

[0087] Step S310. Obtain the buffer statistics information, and obtain the expected buffer drain rate according to the buffer statistics information.

[0088] The buffer statistics information can include a sending buffer value and an expected buffer drain duration, wherein the sending buffer value buffer refers to the size of the sending buffer, and the expected buffer drain duration drain_duration refers to the expected value of the duration required to empty the sending buffer. According to the ratio of the sending buffer value to the expected buffer drain duration in the buffer statistics information, the expected buffer drain rate can be obtained, which refers to the expected value of the drain rate of the sending buffer.

[0089] Step S320. Obtain the maximum available rate of the media stream according to the maximum sending rate and the expected buffer drain rate.

[0090] According to the difference between the maximum sending rate and the expected buffer drain rate, the maximum available rate of the media stream can be obtained.

[0091] For example, the maximum sending rate calculated according to the rate statistics information in the historical statistics period is represented as The expected buffer drain rate is represented as The maximum available rate ideal_rate is calculated according to the following formula:

[0092]

[0093] In step S130, the expected rate at the time of sending the next video frame is determined according to the maximum available rate of the media stream.

[0094] In the example embodiment, the maximum available rate of the media stream can be adjusted according to a certain redundancy ratio to obtain the expected rate at the time of sending the next video frame. For example, the expected forward error correction redundancy ratio fec_ratio can be obtained, and the expected rate at the time of sending the next video frame is obtained according to the maximum available rate of the media stream and the forward error correction redundancy ratio. The expected forward error correction redundancy ratio fec_ratio can be used to combat network packet loss problems.

[0095] For example, the rate adjustment ratio can be calculated based on the expected forward error correction redundancy ratio fec_ratio, and the expected rate expect_rate of the next frame of media stream is obtained according to the rate adjustment ratio and the maximum available rate ideal_rate.

[0096] For example, the rate adjustment ratio is represented as The formula for calculating the expected bitrate of the next frame of the media stream is as follows:

[0097]

[0098] In step S140, the desired bit rate is adjusted according to the preset delay parameters, and the encoder is controlled to encode the next video frame according to the adjusted desired bit rate.

[0099] In this example implementation, the video frame parameters of the largest video frame in the historical transmission process can be obtained, and the adjusted expected bitrate can be obtained based on the video frame parameters of the largest video frame and the preset delay parameters.

[0100] If the transmission is performed directly based on the expected bitrate, the ideal transmission time for a video frame of size F at the expected bitrate is:

[0101]

[0102] Due to the fluctuation in the size of encoded video frames, the actual size of the video frames can be disregarded when adjusting the expected bitrate. Instead, the adjustment can be based on the size of the largest video frame in the historical transmission process. At the same time, the maximum acceptable latency can be combined to ensure that the adjusted expected bitrate meets the transmission latency requirements.

[0103] For example, the desired bitrate adjustment ratio can be obtained based on the desired bitrate, latency parameters, and maximum video frames, and then the desired bitrate can be adjusted according to the adjustment ratio. For instance, the adjusted desired bitrate can be obtained by multiplying the desired bitrate by the adjustment ratio.

[0104] For example, under the frame delay requirement max_sent_delay, the adjustment ratio of the expected bitrate can be expressed as: The adjusted media's expected bitrate satisfies the following formula:

[0105]

[0106] Here, `max_frame_size` is the maximum frame size based on historical statistics, and `max_sent_delay` is a delay parameter representing the maximum acceptable delay. Adjusting the desired bitrate using the preset delay parameter can meet the transmission delay requirements. Finally, the audio / video encoder is controlled to encode the next video frame based on the adjusted desired bitrate.

[0107] like Figure 4 The diagram shown is a complete flowchart of a bitrate adjustment method according to a specific embodiment of this disclosure. Figure 5A schematic diagram of a system architecture for code rate adjustment in one specific embodiment of the present disclosure is shown, which is an example of the above steps in the present example embodiment, and the specific steps of the flowchart are as follows:

[0108] Step S410. Update the sending code rate.

[0109] After the congestion control algorithm updates the sending code rate pacing_rate of the sending controller when sending the current video frame, the code rate adaptation algorithm flow is triggered.

[0110] Step S420. Calculate the maximum available code rate.

[0111] According to the code rate statistical information in the historical statistical time period and the sending code rate pacing_rate of the sending controller when sending the current video frame, the maximum available code rate ideal_rate of the media stream is calculated.

[0112] Step S430. Code rate allocation.

[0113] According to the expected FEC redundancy ratio, the code rate allocation is performed to obtain the expected code rate expect_rate of the next frame of media stream.

[0114] Step S440. Encoding adaptation.

[0115] Under the frame delay requirement max_sent_delay, the expected code rate expect_rate is adjusted to obtain expect_media_rate to meet the sending delay requirement.

[0116] Step S450. Feedback.

[0117] The adjusted expected code rate is fed back to the audio and video encoder, so as to perform sending of the next video frame.

[0118] In the present example embodiment, by proposing a low-delay code rate adaptation method, the sending end delay can be controlled within the maximum delay max_sent_delay, and the real-time communication experience can be optimized in the case of packet loss network and fluctuation of encoded frame size. The code rate adjustment method in the present example embodiment of the present disclosure has been fully implemented in the scenes of short video RTC, cloud game, low-delay live broadcast, etc., and the online average operation delay of the cloud game can be effectively controlled within 100 milliseconds.

[0119] It should be noted that although the various steps of the methods of the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in this particular order, or that all of the steps shown must be performed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, one step can be broken down into multiple steps, and / or the like.

[0120] Further, the present disclosure also provides a code rate adjustment apparatus. Referring to Figure 6 As shown in the figure, the code rate adjustment apparatus can include a current code rate obtaining module 610, a maximum code rate determining module 620, an expected code rate determining module 630, and an expected code rate adjusting module 640. Among them:

[0121] The current code rate obtaining module 610 is configured to perform obtaining the sending code rate of the current video frame when it is sent;

[0122] The maximum code rate determining module 620 is configured to perform obtaining the maximum available code rate of the media stream according to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame;

[0123] The expected code rate determining module 630 is configured to perform determining the expected code rate when the next video frame is sent according to the maximum available code rate of the media stream;

[0124] The expected code rate adjusting module 640 is configured to perform adjusting the expected code rate according to the preset delay parameter, and controlling the encoder to encode the next video frame according to the adjusted expected code rate.

[0125] In some example embodiments of the present disclosure, the maximum code rate determining module 620 can include a maximum sending code rate determining unit and a maximum available code rate determining unit. Among them:

[0126] The maximum sending code rate determining unit is configured to perform obtaining the maximum sending code rate of the media stream according to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame;

[0127] The maximum available code rate determining unit is configured to perform obtaining the maximum available code rate of the media stream according to the maximum sending code rate of the media stream.

[0128] In some example embodiments of the present disclosure, the maximum sending code rate determining unit can include a maximum sending code rate calculating unit configured to perform obtaining the maximum sending code rate of the media stream according to the media average code rate, the forward error correction average code rate, the arrival confirmation average code rate, the total sending code rate in the historical statistical time period, and the sending code rate of the current video frame.

[0129] In some example embodiments of the present disclosure, the maximum available code rate determining unit can comprise a buffer emptying rate determining unit and a maximum available code rate calculating unit. Wherein:

[0130] The buffer emptying rate determining unit is configured to perform obtaining buffer statistics information, and obtaining an expected buffer emptying rate according to the buffer statistics information;

[0131] The maximum available code rate calculating unit is configured to perform obtaining a maximum available code rate of the media stream according to the maximum sending code rate and the expected buffer emptying rate.

[0132] In some example embodiments of the present disclosure, the buffer emptying rate determining unit can comprise a buffer emptying rate calculating unit configured to perform obtaining the expected buffer emptying rate according to a ratio of the sending buffer value in the buffer statistics information to the expected buffer emptying time length.

[0133] In some example embodiments of the present disclosure, the expected code rate determining module 630 can comprise an expected code rate calculating unit configured to perform obtaining an expected forward error correction redundancy ratio, and obtaining the expected code rate at the time of sending the next video frame according to the maximum available code rate of the media stream and the forward error correction redundancy ratio.

[0134] In some example embodiments of the present disclosure, the expected code rate adjusting module 640 can comprise a maximum video frame parameter obtaining unit and an expected code rate adjusting unit. Wherein:

[0135] The maximum video frame parameter obtaining unit is configured to perform obtaining a video frame parameter of a maximum video frame in a historical sending process;

[0136] The expected code rate adjusting unit is configured to perform obtaining an adjusted expected code rate according to the video frame parameter of the maximum video frame and a preset delay parameter.

[0137] The specific details of each module / unit in the above code rate adjusting apparatus have been described in detail in the corresponding method embodiment part, and will not be repeated here.

[0138] Figure 7 A structural schematic diagram of a computer system of an electronic device suitable for implementing the embodiments of the present disclosure is shown.

[0139] It should be noted that, Figure 7 The computer system 700 of the electronic device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0140] As Figure 7As shown, the computer system 700 includes a central processing unit (CPU) 701 which can perform various appropriate actions and processes in accordance with a program stored in a read only memory (ROM) 702 or a program loaded from the storage section 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0141] Connected to the I / O interface 705 are an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as necessary. A removable recording medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 710 as necessary, so that a computer program read therefrom is installed into the storage section 708 as necessary.

[0142] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program in accordance with embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable recording medium 711. When the computer program is executed by the central processing unit (CPU) 701, various functions defined in the system of the present disclosure are executed.

[0143] An exemplary embodiment of the present disclosure also provides a computer program product. The computer program product includes a computer program which, when executed by a processor, implements the above-mentioned method for adjusting a code rate.

[0144] In an embodiment, the computer program product can be a tangible product that contains the computer program, such as a computer-readable storage medium having the computer program stored therein. The computer-readable storage medium can be a storage medium based on electric, magnetic, optical, electromagnetic, infrared, or any other type of signal that carries the computer program. Examples of the computer-readable storage medium include, but are not limited to, random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), hard disk drive (HDD), solid state drive (SSD), and so on. For example, the computer program product can be implemented as a non-volatile storage medium that stores the computer program, such as read-only memory (ROM), Nand Flash, and so on.

[0145] In an embodiment, the computer program product can be an intangible product that contains the computer program. For example, the computer program product can be implemented as a virtual digital product, such as an executable file, an installation package, or the like, that stores the computer program.

[0146] The code of the computer program can be written in one or more programming languages. Examples of programming languages include C, Java, C++, and so on. The program code can execute entirely on the user's computing device, or partly on the user's computing device and partly on a remote computing device, or entirely on a remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any kind of network, such as a local area network (LAN), a wide area network (WAN), or the like, or can be connected to an external computing device (for example, through an Internet service provider's connection).

[0147] The computer program can be carried or transmitted by an electronic device through an electric, magnetic, optical, electromagnetic, infrared, or any other type of signal. The electronic device can convert the signal carrying the computer program into a digital signal, and then run the computer program. When the computer program is run on the electronic device, its code is used to make the electronic device perform (more specifically, can make the processor of the electronic device perform) the method steps of various exemplary embodiments of the present disclosure, such as the adjustment method of the code rate described above.

[0148] The computer program product of the first aspect of the present disclosure can include a computer readable storage medium. The computer readable storage medium can be a tangible medium that can retain, store, or maintain the program code for use by a machine. The computer readable storage medium can be based on any suitable technology for programmable processing, including both numerous embodiments of memory media and processing mediums suitable for use with a computer. In this context, a "computer readable storage medium" can include tablets computers, memory cards, ROM, RAM, DVD, Blu-ray, hard drives, or any other suitable medium upon which a program can be maintained or processed.

[0149] It should be noted that although several modules of the device for action execution are mentioned in the detailed description above, such a division is not mandatory. Indeed, according to embodiments of the present disclosure, the features and functionalities of two or more modules described above can be embodied in one module. Conversely, the features and functionalities of one module described above can be further divided into embodied by a plurality of modules.

[0150] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features of the present disclosure disclosed herein. It is intended that the present disclosure cover any and all variations of the present disclosure that come within the scope of the present disclosure, including custom and practice of the art known to those of ordinary skill in the art, absent part from the present disclosure.

[0151] It is to be understood that the present disclosure is not limited to the precise construction described above and shown in the drawings, and that various modifications and changes can be made by those of ordinary skill in the art without departing from the scope of the present disclosure. The scope of the present disclosure is limited only by the claims that follow.

Claims

1. A method of adjusting a code rate, characterized by, The method comprises: acquiring a sending code rate of a current video frame when it is sent; obtaining a maximum available code rate of a media stream according to code rate statistical information in a historical statistical time period and the sending code rate of the current video frame; determining an expected code rate when a next video frame is sent according to the maximum available code rate of the media stream; adjusting the expected code rate according to a preset delay parameter, and controlling an encoder to encode the next video frame according to the adjusted expected code rate.

2. The method of adjusting the code rate according to claim 1, wherein, The obtaining of the maximum available code rate of the media stream according to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame comprises: obtaining a maximum sending code rate of the media stream according to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame; obtaining the maximum available code rate of the media stream according to the maximum sending code rate of the media stream.

3. The method of adjusting the code rate according to claim 2, wherein, The obtaining of the maximum sending code rate of the media stream according to the code rate statistical information in the historical statistical time period and the sending code rate of the current video frame comprises: obtaining the maximum sending code rate of the media stream according to a media average code rate, a forward error correction average code rate, an arrival confirmation average code rate, a total sending code rate in the historical statistical time period and the sending code rate of the current video frame.

4. The method of adjusting the code rate according to claim 2, wherein, The obtaining of the maximum available code rate of the media stream according to the maximum sending code rate of the media stream comprises: obtaining an expected cache emptying rate according to cache statistical information; obtaining the maximum available code rate of the media stream according to the maximum sending code rate and the expected cache emptying rate.

5. The method of adjusting the code rate according to claim 4, wherein, The obtaining of the expected cache emptying rate according to the cache statistical information comprises: obtaining the expected cache emptying rate according to a ratio of a sending cache value in the cache statistical information to an expected cache emptying time length.

6. The method of adjusting the code rate according to claim 1, wherein, The determination of the expected code rate when the next video frame is sent according to the maximum available code rate of the media stream comprises: obtaining an expected forward error correction redundancy ratio, and obtaining the expected code rate when the next video frame is sent according to the maximum available code rate of the media stream and the expected forward error correction redundancy ratio.

7. The method of adjusting the code rate according to claim 1, wherein, The adjustment of the expected code rate according to the preset delay parameter comprises: obtaining a video frame parameter of a maximum video frame in a historical sending process; obtaining the adjusted expected code rate according to the video frame parameter of the maximum video frame and the preset delay parameter.

8. A bitrate adjustment device, characterized in that, The method comprises: a current code rate acquiring module configured to acquire a sending code rate of a current video frame when it is sent; a maximum code rate determining module configured to obtain a maximum available code rate of a media stream according to code rate statistical information in a historical statistical time period and the sending code rate of the current video frame; an expected code rate determining module configured to determine an expected code rate when a next video frame is sent according to the maximum available code rate of the media stream; an expected code rate adjusting module configured to adjust the expected code rate according to a preset delay parameter, and control an encoder to encode the next video frame according to the adjusted expected code rate.

9. An electronic device, comprising: The method comprises: a processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the code rate adjustment method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the code rate adjustment method according to any one of claims 1 to 7.

Images