A multimedia code stream shaping method, device and storage medium for ensuring delay

Abstract

The present disclosure relates to a multimedia code stream shaping method, device and storage medium that guarantees delay. The method includes: calculating the average bit rate according to the frame length in the current preset time period, the current latest frame sequence number, the number of frames contained in a cycle and the frame interval duration; calculating the delay guarantee bit rate according to the current length of the data frame to be sent and the delay to be guaranteed; calculating the cache bit rate according to the total length of the data in the current cache and the delay to be guaranteed; in each sending cycle, calculating the actual sending bit rate according to the average bit rate, the delay guarantee bit rate and the cache bit rate. The present disclosure uses the calculated actual sending bit rate as the sending bit rate of subsequent data frames, which not only avoids the network packet loss problem caused by the impact on the network, but also ensures that the delay when sending data will not exceed the established delay size, thereby ensuring the timeliness of the service. In addition, it also ensures that the cache size will not exceed the established cache size, thereby improving the user experience.

Description

Technical Field

[0001] The present disclosure relates to the technical field of bitstream shaping, and in particular to a delay-guaranteed multimedia bitstream shaping method, device and storage medium. Background Art

[0002] The bitstream shaping algorithm is a common algorithm in multimedia transmission. It is mainly used to avoid network packet loss caused by drastic bitstream fluctuations. The simple bitstream shaping algorithm is implemented by caching messages and sending them according to the calculated average bitrate. The bitstream shaping algorithm will introduce delay. For intermittent data peaks (such as I frames in video encoding), the delay is more obvious, affecting the user experience.

[0003] Therefore, it is necessary to provide a new technical solution to improve one or more problems existing in the above solutions.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore, may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present disclosure is to provide a multimedia code stream shaping method, device and storage medium that ensure delay, thereby overcoming one or more problems caused by limitations and defects of related technologies at least to a certain extent.

[0006] According to a first aspect of an embodiment of the present disclosure, a multimedia code stream shaping method for ensuring delay is provided, the method comprising:

[0007] The average bit rate is calculated according to the frame length in the current preset time period, the current latest frame sequence number, the number of frames included in a cycle and the frame interval duration; wherein the current preset time period at least includes the entire cycle;

[0008] Calculate the delay guarantee code rate according to the current length of the data frame to be sent and the delay to be guaranteed;

[0009] Calculate the cache bit rate based on the total length of data in the current cache and the delay to be guaranteed;

[0010] In each sending cycle, the actual sending bit rate is calculated according to the average bit rate, the delay guaranteed bit rate and the buffered bit rate.

[0011] In the embodiment of the present disclosure, the calculation formula of the average bit rate is:

[0012]

[0013] Wherein, Ra represents the average bit rate, n represents the current latest frame number, p represents the number of frames contained in a cycle, L represents the frame length in the current preset time period, and Δt represents the frame interval duration.

[0014] In the embodiment of the present disclosure, the calculation formula of the delay guarantee code rate is:

[0015] Rd=Ln / D (2)

[0016] Among them, Rd represents the delay guaranteed bit rate, Ln represents the length of the current data frame to be sent, and D represents the delay to be guaranteed.

[0017] In the embodiment of the present disclosure, the calculation formula of the cache code rate is:

[0018] Rb=B / D (3)

[0019] Among them, Rb represents the cache bit rate, B represents the total length of data in the current cache, and D represents the delay to be guaranteed.

[0020] In the embodiment of the present disclosure, the calculation formula of the actual sending rate is:

[0021] S=max(Ra,Rd,Rb) (4)

[0022] Wherein, S represents the actual sending rate of each sending cycle.

[0023] In the embodiment of the present disclosure, the period is a GOP period.

[0024] In the embodiment of the present disclosure, the average bit rate is recalculated after each frame of data is received.

[0025] In the embodiment of the present disclosure, the delay guarantee bit rate is recalculated after each frame of data is sent.

[0026] In the embodiment of the present disclosure, the cache code rate is recalculated in each sending cycle.

[0027] According to a second aspect of an embodiment of the present disclosure, a multimedia code stream shaping device for ensuring delay is provided, the device comprising:

[0028] A first calculation module, used to calculate the average bit rate according to the frame length in the current preset time period, the current latest frame sequence number, the number of frames included in a cycle and the frame interval duration; wherein the current preset time period at least includes the entire cycle;

[0029] The second calculation module is used to calculate the delay guarantee code rate according to the current length of the data frame to be sent and the delay to be guaranteed;

[0030] A third calculation module is used to calculate the cache bit rate according to the total length of data in the current cache and the delay to be guaranteed;

[0031] The fourth calculation module is used to calculate the actual sending code rate according to the average code rate, the delay guarantee code rate and the cache code rate in each sending cycle.

[0032] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of the multimedia code stream shaping method for ensuring delay described in any one of the above-mentioned embodiments are implemented.

[0033] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0034] In one embodiment of the present disclosure, according to the above method, by calculating the average bit rate, the delay guarantee bit rate and the cache bit rate, and then in each sending cycle, the actual sending bit rate is calculated according to the calculated average bit rate, the delay guarantee bit rate and the cache bit rate, and the calculated actual sending bit rate is used as the sending bit rate of subsequent data frames. By using the calculated actual sending bit rate as the sending bit rate of subsequent data frames, not only can the network packet loss problem caused by the impact on the network be avoided, but also it can be ensured that the delay in sending data will not exceed the established delay size, thereby ensuring the timeliness of the service. In addition, it can also be ensured that the cache size will not exceed the established cache size, thereby improving the user experience.

[0035] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0037] Figure 1 A flowchart schematically illustrates the steps of a multimedia code stream shaping method for ensuring delay in an exemplary embodiment of the present disclosure;

[0038] Figure 2 A block diagram schematically shows a multimedia code stream shaping device for ensuring delay in an exemplary embodiment of the present disclosure;

[0039] Figure 3 A schematic diagram of a program product in an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0040] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0041] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0042] This example implementation first provides a multimedia code stream shaping method that guarantees delay. Figure 1 As shown in , the method may include: steps S101 to S104.

[0043] Step S101: Calculate the average bit rate according to the frame length in the current preset time period, the current latest frame sequence number, the number of frames included in a cycle and the frame interval duration; wherein the current preset time period at least includes the entire cycle.

[0044] Step S102: Calculate the delay guarantee code rate according to the current length of the data frame to be sent and the delay to be guaranteed.

[0045] Step S103: Calculate the cache bit rate according to the total length of data in the current cache and the delay to be guaranteed.

[0046] Step S104: in each sending cycle, the actual sending bit rate is calculated according to the average bit rate, the delay guaranteed bit rate and the buffered bit rate.

[0047] Through the above method and device, by calculating the average bit rate, the delay guarantee bit rate and the cache bit rate, then in each sending cycle, the actual sending bit rate is calculated according to the calculated average bit rate, the delay guarantee bit rate and the cache bit rate, and the calculated actual sending bit rate is used as the sending bit rate of subsequent data frames. By using the actual sending bit rate as the sending bit rate of subsequent data frames, not only can the network packet loss problem caused by the impact on the network be avoided, but also it can be ensured that the delay when sending data will not exceed the established delay size, thereby ensuring the timeliness of the service. In addition, it can also be ensured that the cache size will not exceed the established cache size, thereby improving the user experience.

[0048] Next, we will refer to Figure 1 Each step of the above method in this example implementation is described in more detail.

[0049] In step S101, the average bit rate is calculated based on the frame length in the current preset time period, the current latest frame sequence number, the number of frames contained in a cycle and the frame interval duration; wherein the current preset time period at least includes the entire cycle. Specifically, the average bit rate usually refers to the average bit rate of digital music or video. When calculating the average bit rate Ra, it is calculated by taking the average value of the sent data in a relatively long period of time. If the data is periodic, the time period at least includes the entire cycle. The average bit rate is specifically calculated based on the frame length in the current preset time period, the current latest frame sequence number, the number of frames contained in a cycle and the frame interval duration. wherein the period is the GOP period commonly used in video encoding.

[0050] In actual encoding, the bit rate will be adjusted according to the complexity of the image / audio to achieve the best control of file size and quality. Therefore, the final average bit rate cannot be determined before encoding, and the fluctuation range is large. In order to solve this problem to a certain extent, the secondary encoding method was born. It does not perform actual encoding operations during the first encoding, but only calculates the bit rate. The second time, it uses this calculation result as a basis, adjusts according to the set value, and finally encodes. Average bit rate (ABR) refers to such an encoding method in audio encoding: it is basically the same as constant bit rate (CBR), and will be encoded according to the set target bit rate. But when the encoder considers it "appropriate", it will use a value higher than the target bit rate for encoding to ensure better quality.

[0051] In one embodiment, the calculation formula of the average bit rate is:

[0052]

[0053] Wherein, Ra represents the average bit rate, n represents the current latest frame number, p represents the number of frames in a cycle, L represents the frame length in the current preset time period, and Δt represents the frame interval length. Specifically, Ra is called Rate average in English, and the average bit rate Ra is calculated according to the above formula (1).

[0054] In one embodiment, the average bit rate is recalculated after each frame of data is received. Specifically, since the average bit rate of each received frame of data is different, the average bit rate needs to be recalculated after each received frame of data to ensure that the average bit rate value calculated each time is the latest.

[0055] In step S102, the delay guaranteed code rate is calculated according to the current length of the data frame to be sent and the guaranteed delay. Specifically, the delay guaranteed code rate is calculated according to the current length of the data frame to be sent and the guaranteed delay.

[0056] In one embodiment, the calculation formula of the delay guarantee code rate is:

[0057] Rd=Ln / D (2)

[0058] Wherein, Rd represents the delay guarantee code rate, Ln represents the length of the current data frame to be sent, and D represents the delay to be guaranteed. Specifically, Rd is the full name of Rate for delay, Ln is the full name of Length of frame n, and the delay guarantee code rate Rd is calculated by the above formula (2).

[0059] In one embodiment, the delay guarantee code rate is recalculated after each frame of data is sent. Specifically, since the delay guarantee code rate after each frame of data is sent is different, the delay guarantee code rate needs to be recalculated after each frame of data is sent to ensure that the value of the delay guarantee code rate calculated each time is the latest.

[0060] In step S103, the cache code rate is calculated according to the total length of the data in the current cache and the delay to be guaranteed. Specifically, the cache code rate is calculated according to the total length of the data in the current cache and the delay to be guaranteed.

[0061] In one embodiment, the calculation formula of the cache code rate is:

[0062] Rb=B / D (3)

[0063] Wherein, Rb represents the buffer code rate, B represents the total length of data in the current buffer, and D represents the delay to be guaranteed. Specifically, Rb is called Rate for buffer in English, and the buffer code rate Rb is calculated by the above formula (3).

[0064] In one embodiment, the cache code rate is recalculated in each sending cycle. Specifically, since the cache code rate in each sending cycle is different, the cache code rate needs to be recalculated in each sending cycle to ensure that the calculated cache code rate value is the latest in each sending cycle.

[0065] In step S104, in each sending cycle, the actual sending code rate is calculated according to the average code rate, the delay guarantee code rate and the buffer code rate. Specifically, the actual sending code rate in each sending cycle depends on the average code rate, the delay guarantee code rate and the buffer code rate, so when calculating the actual sending code rate in each sending cycle, it can be calculated according to the average code rate, the delay guarantee code rate and the buffer code rate.

[0066] In one embodiment, the actual sending rate is calculated as follows:

[0067] S=max(Ra,Rd,Rb) (4)

[0068] Wherein, S represents the actual sending rate of each sending cycle.

[0069] Specifically, the actual sending rate of each sending cycle can be calculated according to the above formula (4), that is, the actual sending rate of each sending cycle is the maximum value of the average bit rate, the delay guarantee bit rate and the cache bit rate.

[0070] Through the above method, the average bit rate, the delay guarantee bit rate and the cache bit rate are calculated, and then in each sending cycle, the actual sending bit rate is calculated according to the calculated average bit rate, the delay guarantee bit rate and the cache bit rate, and the calculated actual sending bit rate is used as the sending bit rate of subsequent data frames. By using the actual sending bit rate as the sending bit rate of subsequent data frames, not only can the network packet loss problem caused by the impact on the network be avoided, but also it can be ensured that the delay when sending data will not exceed the established delay size, thereby ensuring the timeliness of the service. In addition, it can also be ensured that the cache size will not exceed the established cache size, thereby improving the user experience.

[0071] It should be noted that the predetermined delay size is a preset delay size, that is, a preset delay indicator; the predetermined cache size is a preset cache size, that is, a preset cache indicator.

[0072] It should be noted that, although the steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc. In addition, it is also easy to understand that these steps may be, for example, executed synchronously or asynchronously in multiple modules / processes / threads.

[0073] Furthermore, in this exemplary embodiment, a multimedia code stream shaping device with guaranteed delay is also provided. Figure 2 As shown in , the apparatus 200 may include a first calculation module 201 , a second calculation module 202 , a third calculation module 203 and a fourth calculation module 204 .

[0074] The first calculation module 201 is used to calculate the average bit rate according to the frame length in the current preset time period, the current latest frame sequence number, the number of frames included in a cycle and the frame interval duration; wherein the current preset time period at least includes the entire cycle.

[0075] The second calculation module 202 is used to calculate the delay guarantee code rate according to the current length of the data frame to be sent and the delay to be guaranteed.

[0076] The third calculation module 203 is used to calculate the cache code rate according to the total length of data in the current cache and the delay to be guaranteed.

[0077] The fourth calculation module 204 is used to calculate the actual transmission bit rate according to the average bit rate, the delay guarantee bit rate and the buffer bit rate in each transmission cycle.

[0078] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0079] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of a module or unit described above can be further divided into multiple modules or units for concretization. The components displayed as modules or units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without paying creative work.

[0080] In this example embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a computer program is stored. When the program is executed by, for example, a processor, the steps of the multimedia code stream shaping method for ensuring delay described in any of the above embodiments can be implemented.

[0081] In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary implementations of the present invention described in the above-mentioned cloud desktop sharing method section of this specification.

[0082] refer to Figure 3As shown in , a program product 300 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0083] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0084] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0085] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0086] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A multimedia code stream shaping method for ensuring delay, characterized in that: The method includes: The average bit rate is calculated according to the frame length in the current preset time period, the current latest frame sequence number, the number of frames included in a cycle and the frame interval duration; wherein the current preset time period at least includes the entire cycle; The calculation formula for the average bit rate is: Wherein, Ra represents the average bit rate, n represents the current latest frame number, p represents the number of frames in a cycle, L represents the frame length in the current preset time period, and Δt represents the frame interval duration; Calculate the delay guarantee code rate according to the current length of the data frame to be sent and the delay to be guaranteed; Calculate the cache bit rate based on the total length of data in the current cache and the delay to be guaranteed; In each sending cycle, the actual sending bit rate is calculated according to the average bit rate, the delay guarantee bit rate and the buffer bit rate; The calculation formula of the actual sending rate is: S=max(Ra,Rd,Rb) (4) Wherein, S represents the actual sending rate of each sending cycle, Rd represents the delay guaranteed bit rate, and Rb represents the buffered bit rate.

2. The delay-guaranteed multimedia code stream shaping method according to claim 1, characterized in that: The calculation formula of the delay guarantee bit rate is: Rd=Ln / D (2) Among them, Rd represents the delay guaranteed bit rate, Ln represents the length of the current data frame to be sent, and D represents the delay to be guaranteed.

3. The delay-guaranteed multimedia code stream shaping method according to claim 1, characterized in that: The calculation formula of the cache code rate is: Rb=B / D (3) Among them, Rb represents the cache bit rate, B represents the total length of data in the current cache, and D represents the delay to be guaranteed.

4. The delay-guaranteed multimedia code stream shaping method according to claim 1, characterized in that: The period is a GOP period.

5. The delay-guaranteed multimedia code stream shaping method according to claim 1, characterized in that: The average bit rate is recalculated after each frame of data is received.

6. The delay-guaranteed multimedia code stream shaping method according to claim 1, characterized in that: The delay guarantees that the bit rate is recalculated after each frame of data is sent.

7. The delay-guaranteed multimedia code stream shaping method according to claim 1, characterized in that: The buffer code rate is recalculated in each sending cycle.

8. A multimedia code stream shaping device for ensuring delay, characterized in that: The device includes: A first calculation module, used to calculate the average bit rate according to the frame length in the current preset time period, the current latest frame sequence number, the number of frames included in a cycle and the frame interval duration; wherein the current preset time period at least includes the entire cycle; The calculation formula for the average bit rate is: Wherein, Ra represents the average bit rate, n represents the current latest frame number, p represents the number of frames in a cycle, L represents the frame length in the current preset time period, and Δt represents the frame interval duration; The second calculation module is used to calculate the delay guarantee code rate according to the current length of the data frame to be sent and the delay to be guaranteed; A third calculation module is used to calculate the cache bit rate according to the total length of data in the current cache and the delay to be guaranteed; A fourth calculation module, used to calculate the actual transmission bit rate according to the average bit rate, the delay guarantee bit rate and the cache bit rate in each transmission cycle; The calculation formula of the actual sending rate is: S=max(Ra,Rd,Rb) (4) Wherein, S represents the actual sending rate of each sending cycle, Rd represents the delay guaranteed bit rate, and Rb represents the buffered bit rate.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the delay-guaranteed multimedia code stream shaping method described in any one of claims 1 to 7 are implemented.

Images