Code rate control and video coding method and device, live broadcast system, equipment and medium

Abstract

The invention relates to a code rate control method and device, a video coding control method and device, a live broadcast system, electronic equipment and a computer readable storage medium. The code rate control method comprises the following steps: entering a forward prediction code rate control stage in a process of performing program stream coding on a to-be-coded image frame; calculating an image frame error between a front frame and a rear frame of the to-be-coded image frame in a forward prediction code rate control stage; calculating a frame-level quantization parameter and a block-level quantization parameter according to the image frame error between the front frame and the rear frame in code rate control; updating the coding rate of the image frame to be coded according to the frame-level quantization parameter and the block-level quantization parameter; according to the technical scheme, the complexity of calculating the quantization parameter in the TPL thread is reduced, the time delay and the data calculation amount are reduced, and the code rate control stability is improved.

Description

Technical Field

[0001] The present application relates to the field of video coding technology, and in particular to a rate control method and device, a video coding control method and device, a live broadcast system, an electronic device, and a computer-readable storage medium. Background Art

[0002] In video coding technology, AV1 (AOMedia Video 1) is an open source video coding format that introduces complex compression technology, achieves a higher data compression ratio, and provides a more efficient and high-quality solution for Internet video transmission. It performs well in application scenarios such as video on demand, real-time live encoding and transcoding, has excellent cross-platform compatibility, and shows great potential to become the next generation of mainstream codec.

[0003] Rate control is the process of adjusting the video quality by allocating an appropriate number of bits to each frame of the video during the video encoding process, which has a very important impact on the encoding effect. During conventional AV1 video encoding, the rate control process of VBV (Video Buffering Verifier) ​​is as follows: assign a target number of bits to each GOP (Group of Pictures) and mini-GOP, update the target number of bits for each frame based on feedback and internal buffering, generate frame-level QP (Quantization Parameter), block-level QP and λ function based on TPL (Transport Program Layer) and the target number of bits, and finally re-encode and update parameters through judgment.

[0004] However, the inventors found in actual applications that the above-mentioned AV1 bit rate control scheme has a fixed and lengthy overall process and complex TPL calculation, which increases the time delay and data calculation amount. It is necessary to use secondary encoding to ensure the accuracy of bit rate control, and the bit rate control stability is poor. Summary of the invention

[0005] Based on this, it is necessary to provide a rate control method and device, a video encoding control method and device, a live broadcast system, an electronic device and a computer-readable storage medium to address at least one of the above-mentioned technical defects, so as to reduce the complexity of rate control and improve the accuracy and stability of rate control.

[0006] A rate control method for video image encoding, comprising:

[0007] In the process of program stream encoding of the image frame to be encoded, the forward prediction rate control stage is entered;

[0008] Calculating the image frame error between the previous and next frames of the image frame to be encoded in the forward prediction rate control stage;

[0009] Calculating a frame-level quantization parameter and a block-level quantization parameter according to the image frame error between the preceding and following frames in bit rate control;

[0010] The encoding bit rate of the image frame to be encoded is updated according to the frame-level quantization parameter and the block-level quantization parameter.

[0011] In one embodiment, the bit rate control method further includes:

[0012] During the process of program stream encoding of the image frame to be encoded, it is determined whether to enter the forward prediction rate control stage based on the current encoding efficiency and video image quality. If so, the forward prediction rate control stage is entered; otherwise, the encoding rate of the image frame to be encoded is controlled using the old encoding rate.

[0013] In one embodiment, calculating the image frame error between the previous and next frames of the image frame to be encoded in the forward prediction rate control stage includes:

[0014] Sub-sampling the image frame to be encoded to obtain image frame data;

[0015] According to the image frame data, respectively calculate the intra-frame coding cost of each macroblock when intra-frame coding is performed and calculate the inter-frame coding cost of the difference between the current frame and the reference frame;

[0016] An image frame error between previous and next frames of the image frame to be encoded is obtained according to the intra-frame encoding cost and the inter-frame encoding cost.

[0017] In one embodiment, calculating the frame-level quantization parameter and the block-level quantization parameter according to the image frame error between the previous and next frames in the bit rate control includes:

[0018] In the bit rate control thread, the coding complexity is calculated according to the image frame error between the previous and next frames;

[0019] Calculating frame-level quantization parameters of I frames and P frames according to the coding complexity, and calculating frame-level quantization parameters of B frames by means of a layer offset;

[0020] The block-level quantization parameter is calculated according to the image frame error between the previous and next frames and a reference value.

[0021] In one embodiment, updating the encoding bit rate of the to-be-encoded image frame according to the frame-level quantization parameter and the block-level quantization parameter includes:

[0022] The bit rate parameter and Lambda function of the image frame to be encoded are updated according to the frame-level quantization parameter of the I frame and the P frame, the B frame-level quantization parameter control and the block-level quantization parameter offset value.

[0023] A bit rate control device, used for video image encoding, comprising:

[0024] A forward prediction module, used for entering a forward prediction rate control stage during the process of program stream encoding of the image frame to be encoded;

[0025] An error calculation module, used for calculating the image frame error between the previous and next frames of the image frame to be encoded in the forward prediction rate control stage;

[0026] A parameter calculation module, used for calculating a frame-level quantization parameter and a block-level quantization parameter according to an image frame error between the preceding and following frames in bit rate control;

[0027] The bit rate updating module is used to update the encoding bit rate of the image frame to be encoded according to the frame level quantization parameter and the block level quantization parameter.

[0028] A video encoding method, comprising:

[0029] Reading an image frame of a video image to be encoded;

[0030] Configure the basic encoding parameters and encoding switch of the current encoding;

[0031] When it is necessary to enter a new rate control, the encoding rate of the image frame of the video image to be encoded is obtained based on the rate control method;

[0032] The encoding rate is used to control the encoding process of each image frame and output an encoded code stream.

[0033] A video encoding device, comprising:

[0034] A reading module, used for reading an image frame of a video image to be encoded;

[0035] Configuration module, used to configure the basic encoding parameters and encoding switch of the current encoding;

[0036] An acquisition module, used for acquiring the encoding rate of the image frame of the video image to be encoded based on the rate control method when it is necessary to enter a new rate control;

[0037] The control module is used to control the encoding process of each image frame using the encoding code rate and output the encoding code stream.

[0038] A live broadcast system, characterized in that it comprises: a client and a live broadcast server; wherein the client comprises an anchor terminal and a viewer terminal respectively connected to the live broadcast server through a network;

[0039] The live broadcast server is used to receive the anchor video uploaded by the anchor terminal, encode the anchor video using the video encoding method, and forward it to the audience terminal;

[0040] The anchor terminal is used to collect anchor videos and upload them to the live broadcast server;

[0041] The audience terminal is used for audience users accessing the live broadcast room to play the anchor video sent by the live broadcast server.

[0042] An electronic device, comprising:

[0043] one or more processors;

[0044] Memory;

[0045] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the steps of the bit rate control method or the steps of the video encoding method.

[0046] A computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, wherein the at least one instruction, the at least one program, the code set or the instruction set is loaded by the processor and executes the steps of the bit rate control method or the steps of the video encoding method.

[0047] The technical solution of the present application enters a forward prediction rate control stage during program stream encoding of an image frame to be encoded; in the forward prediction rate control stage, an image frame error between the previous and next frames of the image frame to be encoded is calculated; in rate control, a frame-level quantization parameter and a block-level quantization parameter are calculated according to the image frame error between the previous and next frames; and the encoding rate of the image frame to be encoded is updated according to the frame-level quantization parameter and the block-level quantization parameter; this technical solution introduces rate control based on forward prediction during program stream encoding of an image frame to be encoded, calculates the image frame error in forward prediction and uses it to calculate the frame-level quantization parameter and the block-level quantization parameter, thereby controlling the rate of the AV1 video format, reducing the complexity of calculating the quantization parameter in the TPL thread, reducing the time delay and the amount of data calculation, and improving the stability of rate control. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of a rate control method according to an embodiment;

[0049] Figure 2 is a schematic diagram of an example of the process of calculating the quantization parameter;

[0050] Figure 3 is a schematic diagram of an example bit rate control process;

[0051] Figure 4A schematic diagram of the structure of a bit rate control device according to an embodiment

[0052] Figure 5 is a flow chart of a video encoding method according to an embodiment;

[0053] Figure 6 is a video encoding flow chart of an example AV1 encoder;

[0054] Figure 7 is a schematic structural diagram of a video encoding device according to an embodiment;

[0055] Figure 8 This is a schematic diagram of the live broadcast system structure;

[0056] Fig. 9 is a block diagram of an example electronic device. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] In the embodiments of the present application, the words "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. "at least one" means one or more, and "multiple" means two or more. For example, multiple objects refer to two or more objects. "Include" or "comprises" and similar words mean that the information appearing before "include" or "comprises" covers the information listed after "include" or "comprises" and its equivalents, and does not exclude other information. "And / or" mentioned in the embodiments of the present application indicates that three types of relationships may exist. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0059] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0060] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0061] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0062] The rate control scheme and video encoding scheme of the present application are mainly aimed at the technical defects in AV1 video encoding technology, in which the overall process is fixed and lengthy, the TPL calculation is complex, the time delay and data calculation amount are too large, and secondary encoding is required to ensure the accuracy of rate control, and an improvement plan is proposed.

[0063] refer to Figure 1 As shown, Figure 1 The flowchart of the bit rate control method of an embodiment is applied in the AV1 video encoding scenario, which mainly includes the following steps:

[0064] Step S110, during the process of performing program stream encoding on the image frame to be encoded, entering the forward prediction rate control stage.

[0065] In this step, Lookahead is introduced in the TPL thread of AV1 video encoding to perform bit rate control and enter the forward prediction bit rate control stage; TPL is a way of using program flow for video encoding, and Lookahead is a technology used in video encoding to estimate the encoding consumption or cost of video frames that have not yet completed analysis.

[0066] In one embodiment, the rate control method of the present application, during the process of program stream encoding of the image frame to be encoded in step S110, also determines whether to enter the forward prediction rate control stage based on the current encoding efficiency and video image quality. If so, enter the forward prediction rate control stage; otherwise, use the old encoding rate to control the encoding rate of the image frame to be encoded.

[0067] The scheme of the above-mentioned embodiment introduces a new lookahead-based rate control during the program stream encoding process of the image frame to be encoded. When it is necessary to enter the rate control, Lookahead can be used to perform the rate control. When the rate update is not required, the old encoding rate is used to control the encoding rate of the image frame to be encoded, thereby optimizing the rate control process and ensuring the encoding stability.

[0068] Step S120, calculating the image frame error between the previous and next frames of the image frame to be encoded in the forward prediction rate control stage.

[0069] In this step, in the forward prediction rate control stage, the image frame error between the previous and next frames of the image frame to be encoded is calculated in Lookahead.

[0070] In one embodiment, the method for calculating the image frame error between the previous and next frames of the image frame to be encoded in step S120 may include the following steps:

[0071] S201, performing sub-sampling on the image frame to be encoded to obtain image frame data.

[0072] Specifically, sub-sampling is performed on the image frame to be encoded to reduce the number of image frame data and maintain the representativeness of the image frame data.

[0073] S202, calculating the intra-frame coding cost of each macroblock when performing intra-frame coding according to the image frame data and the inter-frame coding cost of calculating the difference between the current frame and the reference frame.

[0074] Specifically, the intra-frame encoding cost Intracost when each macroblock is intra-frame encoded is calculated according to the image frame data, and the intra-frame encoding cost Intracost is the cost or price when each macroblock (macroblock) is intra-frame encoded (Intra-frame encoding) in the video encoding process; and the inter-frame encoding cost intercost of the difference between the current frame and the reference frame is calculated according to the image frame data; the inter-frame encoding cost intercost generally refers to the cost of inter-frame encoding in video encoding, and is used to measure the difference cost between the current frame and the reference frame in the inter-frame encoding process.

[0075] S203: Obtain an image frame error between previous and next frames of the image frame to be encoded according to the intra-frame encoding cost and the inter-frame encoding cost.

[0076] Specifically, based on the intra-frame coding cost Intracost and the inter-frame coding cost intercost calculated in Lookahead, the image frame error data between the previous and next frames of the image frame to be coded can be obtained.

[0077] Step S130 , calculating a frame-level quantization parameter and a block-level quantization parameter according to the image frame error between the previous and next frames in rate control.

[0078] In this step, during the bit rate control process, a quantization parameter QP (Quantization Parameter) is calculated according to the image frame error between the previous and next frames, including a frame-level QP and a block-level QP.

[0079] In one embodiment, the method for calculating the frame-level quantization parameter and the block-level quantization parameter in step S130 may include the following steps:

[0080] (1) In the bit rate control thread, the coding complexity is calculated according to the image frame error between the previous and next frames.

[0081] (2) Calculate the frame-level quantization parameters of the I frame and the P frame according to the coding complexity, and calculate the frame-level quantization parameter of the B frame by means of a layer offset.

[0082] In one embodiment, for I frames and P frames, the complexity of the image frame is calculated using errors and mapped to the corresponding Q according to the complexity, mapped to the frame-level QP of the AV1 video according to the layer limit, and the B frame is assigned a frame-level QP according to the hierarchy.

[0083] (3) Calculating a block-level quantization parameter based on the image frame error between the previous and next frames and a reference value.

[0084] Specifically, the block-level QP offset is calculated according to the image frame error between the previous and next frames in combination with the coding cost and the reference value.

[0085] Exemplary, reference Figure 2 As shown, Figure 2 This is a schematic diagram of the calculation process of the quantization parameter, which mainly includes the following:

[0086] s101, Calculate encoding cost in Lookahead;

[0087] s102, calculating the offset of the block-level quantization parameter QP;

[0088] s103, calculating the complexity of the image frame;

[0089] s104, calculating the quantization parameter QP of the I frame and the P frame;

[0090] s105, calculating the quantization parameter QP of the B frame through the level offset;

[0091] s106, bit rate control ends.

[0092] As in the solution of the above embodiment, frame-level QP and block-level QP are calculated in rate control, thereby reducing the amount of calculation and ensuring the accuracy and stability of rate control.

[0093] Step S140: updating the encoding bit rate of the to-be-encoded image frame according to the frame-level quantization parameter and the block-level quantization parameter.

[0094] In this step, the encoding bit rate of the image frame to be encoded is updated in the bit rate control according to the frame level quantization parameter and the block level quantization parameter.

[0095] In one embodiment, when updating the encoding bit rate of the image frame to be encoded, the bit rate parameters of the image frame to be encoded and the Lambda function used in the encoding process can be updated according to the frame-level quantization parameters of the I frame and the P frame, the B frame-level quantization parameter control and the block-level quantization parameter offset value.

[0096] refer to Figure 3 As shown, Figure 3 This is a schematic diagram of an example bit rate control process, which mainly includes the following:

[0097] s201, enter the TPL thread;

[0098] s202, determine whether it is necessary to enter the Lookahead bit rate control, if so, enter s103, otherwise use the old encoding bit rate for control, and go to s106;

[0099] s203, calculating the image frame error between the previous and next frames;

[0100] s204, calculating the frame-level quantization parameter QP;

[0101] s205, calculating the block-level quantization parameter QP;

[0102] s106, end updating the bit rate parameters.

[0103] As in the bit rate control method of the above-mentioned embodiments, in the process of program stream encoding of the image frame to be encoded, a bit rate control based on forward prediction is introduced, the image frame error is calculated in the forward prediction and the frame-level quantization parameter and the block-level quantization parameter are calculated based on this, so as to control the bit rate of the AV1 video format, reduce the complexity of calculating the quantization parameter in the TPL thread, reduce the time delay and the amount of data calculation, and improve the stability of the bit rate control.

[0104] An embodiment of a bit rate control device is described below.

[0105] refer to Figure 4 As shown, Figure 4 The structure diagram of a bit rate control device of an embodiment includes:

[0106] A forward prediction module 110 is used to enter a forward prediction rate control stage during the process of program stream encoding of the image frame to be encoded;

[0107] The error calculation module 120 is used to calculate the image frame error between the previous and next frames of the image frame to be encoded in the forward prediction rate control stage.

[0108] The parameter calculation module 130 is used to calculate the frame-level quantization parameter and the block-level quantization parameter according to the image frame error between the previous and next frames in the bit rate control.

[0109] The bit rate updating module 140 is used to update the encoding bit rate of the to-be-encoded image frame according to the frame-level quantization parameter and the block-level quantization parameter.

[0110] For the forward prediction module 110, Lookahead can be introduced in the TPL thread of AV1 video encoding to perform bit rate control and enter the forward prediction bit rate control stage.

[0111] In one embodiment, the forward prediction module 110 can determine whether to enter the forward prediction rate control stage based on the current coding efficiency and video image quality. If so, it enters the forward prediction rate control stage; otherwise, it uses the old coding rate to control the coding rate of the image frame to be encoded.

[0112] For the error calculation module 120, in the forward prediction rate control stage, the image frame error between the previous and next frames of the image frame to be encoded can be calculated in Lookahead.

[0113] In one embodiment, the error calculation module 120 can perform sub-sampling on the image frame to be encoded to obtain image frame data; calculate the intra-frame coding cost of each macroblock when performing intra-frame coding and the inter-frame coding cost of calculating the difference between the current frame and the reference frame based on the image frame data; and obtain the image frame error between the previous and next frames of the image frame to be encoded based on the intra-frame coding cost and the inter-frame coding cost.

[0114] Specifically, sub-sampling is performed on the image frame to be encoded to reduce the number of image frame data and maintain the representativeness of the image frame data. The intra-frame coding cost Intracost when each macroblock is intra-coded is calculated based on the image frame data. The intra-frame coding cost Intracost is the cost or price when each macroblock is intra-coded during the video coding process; and the inter-frame coding cost intercost of the difference between the current frame and the reference frame is calculated based on the image frame data; the inter-frame coding cost intercost usually refers to the cost of inter-frame coding in video coding, which is used to measure the difference cost between the current frame and the reference frame during the inter-frame coding process. Based on the intra-frame coding cost Intracost and the inter-frame coding cost intercost calculated in Lookahead, the image frame error data between the previous and next frames of the image frame to be encoded can be obtained.

[0115] The parameter calculation module 130 may calculate a quantization parameter QP, including a frame-level QP and a block-level QP, according to an image frame error between previous and next frames during the bit rate control process.

[0116] In one embodiment, the parameter calculation module 130 can calculate the coding complexity according to the image frame error between the previous and next frames in the bit rate control thread; calculate the frame-level quantization parameters of the I frame and the P frame according to the coding complexity, and calculate the frame-level quantization parameters of the B frame by means of a hierarchical offset; calculate the block-level quantization parameters according to the image frame error between the previous and next frames and a reference value.

[0117] Specifically, for I frames and P frames, the complexity of the image frame is calculated using the error, and mapped to the corresponding Q according to the complexity, mapped to the frame-level QP of the AV1 video according to the hierarchy restriction, and the B frame is assigned a frame-level QP according to the hierarchy; based on the image frame error between the previous and next frames, the block-level QP offset is calculated in combination with the encoding cost and the reference value.

[0118] For the bit rate update module 140, when updating the encoding bit rate of the image frame to be encoded, the bit rate parameters of the image frame to be encoded and the Lambda function used in the encoding process can be updated according to the frame level quantization parameters of the I frame and the P frame, the B frame level quantization parameter control and the block level quantization parameter offset value.

[0119] The rate control device of this embodiment can execute a rate control method provided by the embodiment of the present application, and its implementation principle is similar. The actions performed by each module in the rate control device in each embodiment of the present application correspond to the steps in the rate control method in each embodiment of the present application. For the detailed functional description of each module of the rate control device, please refer to the description in the corresponding rate control method shown in the previous text, which will not be repeated here.

[0120] An embodiment of a video encoding method is described below.

[0121] refer to Figure 5 As shown, Figure 5 The video encoding method of an embodiment is a flow chart, comprising:

[0122] Step S210, reading an image frame of a video image to be encoded.

[0123] Specifically, each image frame of the video image to be encoded is read to perform AV1 format image encoding.

[0124] Step S220, configuring basic encoding parameters and encoding switches for the current encoding.

[0125] Specifically, basic encoding parameters and encoding switches used for AV1 format image encoding are configured.

[0126] Step S230: when it is necessary to enter a new rate control, the encoding rate of the image frame of the video image to be encoded is obtained based on the rate control method of any of the previous embodiments.

[0127] Specifically, it is determined whether it is necessary to enter the forward prediction rate control stage. If necessary, the encoding rate is calculated using the rate control solution of the above embodiment.

[0128] Step S240: Control the encoding process of each image frame using the encoding bit rate and output an encoded bit stream.

[0129] Specifically, the frame-level quantization parameters and block-level quantization parameters of each image frame are controlled by bit rate, and the normal encoding process is entered to output the encoded bit stream. When it is not necessary to enter the forward prediction bit rate control stage, the old encoding bit rate is used to control the encoding process.

[0130] refer to Figure 6 As shown, Figure 6 This is a sample AV1 encoder video encoding flow chart, which mainly includes the following steps:

[0131] s301, input image frame;

[0132] s302, configure basic encoding parameters and encoding switch;

[0133] s303, determine whether it is necessary to enter Lookahead bitrate control? If yes, enter s304, otherwise use the old encoding bitrate for control and go to s305;

[0134] s304, executing a rate control process based on Lookahead;

[0135] s305, execution mode decision;

[0136] s306, output the encoded code stream.

[0137] As in the video encoding scheme of the above embodiment, in the process of encoding the AV1 video format, the complexity of calculating the quantization parameters in the TPL thread is reduced, the time delay and the amount of data calculation are reduced, and the stability of the bit rate control is improved.

[0138] An embodiment of a video encoding apparatus is described below.

[0139] refer to Figure 7 As shown, Figure 7 The structure diagram of a video encoding device according to an embodiment includes:

[0140] The reading module 210 is used to read the image frame of the video image to be encoded.

[0141] Specifically, each image frame of the video image to be encoded is read to perform AV1 format image encoding.

[0142] The configuration module 220 is used to configure the basic encoding parameters and encoding switches of the current encoding.

[0143] Specifically, basic encoding parameters and encoding switches used for AV1 format image encoding are configured.

[0144] The acquisition module 230 is used to acquire the encoding bit rate of the image frame of the video image to be encoded based on the bit rate control method of any of the above embodiments when it is necessary to enter a new bit rate control.

[0145] Specifically, it is determined whether it is necessary to enter the forward prediction rate control stage. If necessary, the encoding rate is calculated using the rate control solution of the above embodiment.

[0146] The control module 240 is used to control the encoding process of each image frame using the encoding bit rate and output an encoding bit stream.

[0147] Specifically, the frame-level quantization parameters and block-level quantization parameters of each image frame are controlled by bit rate, and the normal encoding process is entered to output the encoded bit stream. When it is not necessary to enter the forward prediction bit rate control stage, the old encoding bit rate is used to control the encoding process.

[0148] The video encoding device of this embodiment can execute a video encoding method provided by the embodiment of the present application, and its implementation principle is similar. The actions performed by each module in the video encoding device in each embodiment of the present application correspond to the steps in the video encoding method in each embodiment of the present application. For the detailed functional description of each module of the video encoding device, please refer to the description in the corresponding video encoding method shown in the previous text, which will not be repeated here.

[0149] An embodiment of the live broadcast system is described below.

[0150] The live broadcast system of this embodiment refers to Figure 8 As shown, Figure 8 This is an example of a live broadcast system structure diagram. The system framework may include a client and a live broadcast server. The client includes an anchor terminal and an audience terminal. The anchor terminal and multiple audience terminals respectively establish communication connections with the live broadcast server through a wired network or a wireless network. The anchor user broadcasts live through the anchor terminal, such as performing a real talent show, and the audience user chooses to enter a virtual live broadcast room through the audience terminal to watch the anchor user broadcast live. The audience terminal and the anchor terminal can enter the live broadcast platform through the client installed on the electronic device. For example, the anchor terminal and the audience terminal can be computer devices, such as PDA, smart phone, tablet computer, desktop computer or laptop computer, etc., which is not limited to this. It can also be a software module of an application. The live broadcast server includes a background server for providing background services for electronic devices, which can be implemented with an independent server or a server cluster composed of multiple servers.

[0151] For the live broadcast server, it is used to receive the anchor video uploaded by the anchor end, encode the anchor video using the video encoding method of any of the above embodiments, and forward it to the audience end; for the anchor end, it is used to collect the anchor video and upload it to the live broadcast server; for the audience end, it is used to access the live broadcast room and play the anchor video sent by the live broadcast server.

[0152] As in the live broadcast system of the above-mentioned embodiment, the live broadcast server can encode the video image using the AV1 video format. During the encoding process of the AV1 video format, the complexity of calculating the quantization parameters in the TPL thread is reduced, the time delay and the amount of data calculation are reduced, and the stability of the bit rate control is improved. Compared with the conventional AV1 video format encoding scheme, the overall encoding speed is increased by more than 10%, the bit rate control is more stable, and the image quality can be improved.

[0153] Embodiments of electronic devices and computer-readable storage media are described below.

[0154] The present application provides a technical solution for an electronic device, which is used to implement a rate control method or video encoding related functions. The electronic device includes: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured for the rate control method or video encoding method of any embodiment.

[0155] The electronic device may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The electronic device may also be a mobile or stationary server.

[0156] like Fig. 9 As shown, Fig. 9It is a block diagram of an example electronic device, which includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the communication interface communicates with a database that stores data via one or more networks. Examples of these networks include a public switched telephone network (PSTN, Public Switched Telephone Network), a local area network (LAN, Local Area Network), a wide area network (WAN, Wide Area Network), a personal area network (PAN, Personal Area Network) or a combination of communication networks such as the Internet. The access device may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE802.11 wireless local area network (WLAN) wireless interface, a Wi-MAX (World Interoperability for Microwave Access) interface, an Ethernet interface, a USB (Universal Serial Bus) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC). When the computer program is executed by a processor, a method for obtaining a steady-state value of a delay estimation is implemented. The display screen of the electronic device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device may be a touch layer covered on the display screen, or a key, a trackball or a touch pad provided on the housing of the electronic device, or an external keyboard, touch pad or mouse, etc.

[0157] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0158] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0159] The present application provides a technical solution of a computer-readable storage medium to implement the related functions of a rate control method and a video encoding method. The computer-readable storage medium, the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and at least one instruction, at least one program, a code set or an instruction set is loaded by a processor and executes the rate control method of any embodiment. In an exemplary embodiment, the computer-readable storage medium may be a non-temporary computer-readable storage medium including instructions, such as a memory including instructions, for example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0160] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0161] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0162] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A rate control method for video image encoding, characterized in that: include: In the process of program stream encoding of the image frame to be encoded, the forward prediction rate control stage is entered; Calculating the image frame error between the previous and next frames of the image frame to be encoded in the forward prediction rate control stage; Calculating a frame-level quantization parameter and a block-level quantization parameter according to the image frame error between the preceding and following frames in bit rate control; The encoding bit rate of the image frame to be encoded is updated according to the frame-level quantization parameter and the block-level quantization parameter.

2. The bit rate control method according to claim 1, characterized in that: Also includes: In the process of program stream encoding of the image frame to be encoded, judging whether to enter the forward prediction rate control stage according to the current encoding efficiency and the video image quality, if so, entering the forward prediction rate control stage; Otherwise, the encoding rate of the image frame to be encoded is controlled by using the old encoding rate.

3. The bit rate control method according to claim 1, characterized in that: In the forward prediction rate control stage, the image frame error between the previous and next frames of the image frame to be encoded is calculated, including: Sub-sampling the image frame to be encoded to obtain image frame data; According to the image frame data, respectively calculate the intra-frame coding cost of each macroblock when intra-frame coding is performed and calculate the inter-frame coding cost of the difference between the current frame and the reference frame; An image frame error between previous and next frames of the image frame to be encoded is obtained according to the intra-frame encoding cost and the inter-frame encoding cost.

4. The bit rate control method according to claim 3, characterized in that: Calculating a frame-level quantization parameter and a block-level quantization parameter according to the image frame error between the previous and next frames in the bit rate control includes: In the bit rate control thread, the coding complexity is calculated according to the image frame error between the previous and next frames; Calculating frame-level quantization parameters of I frames and P frames according to the coding complexity, and calculating frame-level quantization parameters of B frames by means of a layer offset; The block-level quantization parameter is calculated according to the image frame error between the previous and next frames and a reference value.

5. The bit rate control method according to claim 4, characterized in that: Updating the encoding bit rate of the to-be-encoded image frame according to the frame-level quantization parameter and the block-level quantization parameter includes: The bit rate parameter and Lambda function of the image frame to be encoded are updated according to the frame-level quantization parameter of the I frame and the P frame, the B frame-level quantization parameter control and the block-level quantization parameter offset value.

6. A bit rate control device for video image encoding, characterized in that: include: A forward prediction module, used for entering a forward prediction rate control stage during the process of program stream encoding of the image frame to be encoded; An error calculation module, used for calculating the image frame error between the previous and next frames of the image frame to be encoded in the forward prediction rate control stage; A parameter calculation module, used for calculating a frame-level quantization parameter and a block-level quantization parameter according to an image frame error between the preceding and following frames in bit rate control; The bit rate updating module is used to update the encoding bit rate of the image frame to be encoded according to the frame level quantization parameter and the block level quantization parameter.

7. A video encoding method, characterized in that: include: Reading an image frame of a video image to be encoded; Configure the basic encoding parameters and encoding switch of the current encoding; When it is necessary to enter a new rate control, obtaining the encoding rate of the image frame of the video image to be encoded based on the rate control method according to any one of claims 1 to 5; The encoding rate is used to control the encoding process of each image frame and output an encoded code stream.

8. A video encoding device, characterized in that: include: A reading module, used for reading an image frame of a video image to be encoded; Configuration module, used to configure the basic encoding parameters and encoding switch of the current encoding; An acquisition module, used for acquiring the encoding rate of the image frame of the video image to be encoded based on the rate control method according to any one of claims 1 to 5 when it is necessary to enter a new rate control; The control module is used to control the encoding process of each image frame using the encoding code rate and output the encoding code stream.

9. A live broadcast system, characterized in that: include: Client and live broadcast server; wherein the client includes an anchor terminal and a viewer terminal respectively connected to the live broadcast server through a network; The live broadcast server is used to receive the anchor video uploaded by the anchor terminal, encode the anchor video using the video encoding method according to claim 7, and forward it to the audience terminal; The anchor terminal is used to collect anchor videos and upload them to the live broadcast server; The audience terminal is used for audience users accessing the live broadcast room to play the anchor video sent by the live broadcast server.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the steps of the bit rate control method described in any one of claims 1 to 5, or the steps of the video encoding method described in claim 7.

11. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded by the processor and executes the steps of the bit rate control method described in any one of claims 1 to 5, or the steps of the video encoding method described in claim 7.

Images