Video processing method and apparatus, electronic device, and storage medium

Abstract

The present disclosure relates to the technical field of computers. Disclosed are a video processing method and apparatus, and electronic device, and a storage medium. The method provided by the present disclosure comprises: obtaining a current video frame and a current bandwidth corresponding to the current video frame; performing bit rate prediction on the basis of video frame information and a target quantization parameter to obtain a predicted bit rate of the current video frame, wherein the video frame information comprises a content analysis result of the current video frame and / or frame information of a previous output frame, and the previous output frame is an output frame corresponding to a previous video frame; determining a target bit rate of the current video frame on the basis of the predicted bit rate and the current bandwidth; and encoding the current video frame on the basis of the target bit rate, and determining a current output frame. The method realizes dynamic adjustment of a bit rate of an encoder, and avoids bandwidth resource waste when a definition objective is met.

Description

Video processing method, device, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on October 8, 2023, with application number 202311295072.9 and invention name “Video processing method, device, electronic device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to the field of computer technology, and in particular to a video processing method, device, electronic device, and storage medium. Background Art

[0003] In real-time video transmission scenarios, constant bit rate encoding is generally used to encode video frames before sending them. However, constant bit rate encoding methods cannot meet the needs of real-time video transmission scenarios.

[0004] Summary of the Invention

[0005] In view of this, the present disclosure provides a video processing method, apparatus, electronic device, and storage medium to solve the problem of video processing in real-time video transmission scenarios.

[0006] In a first aspect, the present disclosure provides a video processing method, the method comprising:

[0007] Obtaining a current video frame and a current bandwidth corresponding to the current video frame;

[0008] Performing bit rate prediction based on video frame information and a target quantization parameter to obtain a predicted bit rate of the current video frame, wherein the video frame information includes a content analysis result of the current video frame and / or frame information of a previous output frame, where the previous output frame is an output frame corresponding to the previous video frame;

[0009] Determining a target bit rate for the current video frame based on the predicted bit rate and the current bandwidth;

[0010] The current video frame is encoded based on the target bit rate to determine a current output frame.

[0011] In a second aspect, the present disclosure provides a video processing device, the device comprising:

[0012] An acquisition module, configured to acquire a current video frame and a current bandwidth corresponding to the current video frame;

[0013] a bit rate prediction module, configured to perform bit rate prediction based on video frame information and a target quantization parameter to obtain a predicted bit rate of the current video frame, wherein the video frame information includes a content analysis result of the current video frame and / or frame information of a previous output frame, where the previous output frame is an output frame corresponding to the previous video frame;

[0014] A bit rate determination module, configured to determine a target bit rate for the current video frame based on the predicted bit rate and the current bandwidth;

[0015] The encoding module is used to encode the current video frame based on the target bit rate to determine a current output frame.

[0016] In a third aspect, the present disclosure provides an electronic device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the video processing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0017] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the video processing method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] FIG1 is a flow chart of a video processing method according to an embodiment of the present disclosure;

[0020] FIG2 is a flow chart of another video processing method according to an embodiment of the present disclosure;

[0021] FIG3 is a flow chart of another video processing method according to an embodiment of the present disclosure;

[0022] FIG4 is a processing block diagram of a video processing method according to an embodiment of the present disclosure;

[0023] FIG5 is a processing block diagram of another video processing method according to an embodiment of the present disclosure;

[0024] FIG6 is a structural block diagram of a video processing device according to an embodiment of the present disclosure;

[0025] FIG7 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] To make the purpose, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present disclosure.

[0027] In real-time video transmission technologies, constant bitrate encoding is used to encode and transmit video frames. The higher the bitrate, the clearer the encoded video frames are during playback. Too low a bitrate results in unclear image quality, while higher bitrates waste bandwidth. Therefore, constant encoding struggles to strike a balance between clarity and bandwidth—in other words, it's difficult to provide a constant bitrate that guarantees clarity without wasting bandwidth due to excessively high bitrates.

[0028] Based on this, the video processing method provided by the embodiments of the present disclosure predicts the real-time bitrate based on the content analysis results of the current video frame and / or the frame information of the previous output frame to obtain a predicted bitrate. Based on this, the target bitrate for the current video frame is calculated in combination with the current bandwidth. The current video frame is then encoded at the target bitrate to obtain the current output frame.

[0029] It should be noted that the minimum unit for bitrate processing in the disclosed embodiments is the video frame. That is, for each video frame, a target bitrate corresponding to that video frame must be determined. Of course, in some optional implementations, the bitrate can be calculated every preset number of video frames, or the number of video frames required can be selected based on actual needs. This is not limited to any specific setting and is determined based on actual needs. The following detailed description assumes that the target bitrate needs to be determined for each video frame.

[0030] According to an embodiment of the present disclosure, an embodiment of a video processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] In this embodiment, a video processing method is provided, which can be used in electronic devices such as computers, servers, etc. FIG1 is a flow chart of the video processing method according to an embodiment of the present disclosure. As shown in FIG1 , the flow chart includes the following steps:

[0032] Step S101: Obtain a current video frame and a current bandwidth corresponding to the current video frame.

[0033] In real-time video transmission scenarios, such as video calls, video conferences, live broadcasts, etc., these scenarios all set up a video frame publishing end and a video frame player. The video frame publishing end sends the video frame to the server, encodes it in the server, and then sends it to the video frame player. The video frame player decodes the received video frame and plays it.

[0034] After the video publisher publishes a video frame, the server receives it. The video frame currently being processed is called the current video frame. The video frame immediately preceding the current video frame is called the previous video frame. Video frames correspond to encoded output frames. Therefore, the current video frame corresponds to the current output frame, and the previous video frame corresponds to the previous output frame.

[0035] The current bandwidth corresponding to the current video frame represents the bandwidth allocated for the current video frame. Specifically, the bandwidth allocated for the current video frame is determined by real-time monitoring of the bandwidth status and the amount of data currently being transmitted. The current bandwidth is used to set bandwidth constraints for the encoding bitrate.

[0036] Step S102 : performing bit rate prediction based on the video frame information and the target quantization parameter to obtain a predicted bit rate of the current video frame.

[0037] The video frame information includes a content analysis result of the current video frame and / or frame information of a previous output frame, where the previous output frame is an output frame corresponding to the previous video frame.

[0038] The predicted bitrate is determined based on at least one of the content analysis results of the current video frame or the frame information of the previous output frame. The content analysis results of the current video frame include, but are not limited to, the data volume and content complexity of the current video frame. The frame information of the previous output frame includes, but is not limited to, the size and definition of the previous output frame. The specific content of the content analysis results of the current video frame and the frame information of the previous output frame is determined based on actual needs and is not limited herein.

[0039] The target quantization parameter is used to represent the target definition and constrain the predicted bitrate. The content analysis results of the current video frame and the target quantization parameter are used in the bitrate prediction to represent the bitrate requirement of the current video frame. The frame information of the previous output frame is used in the bitrate prediction to represent the corresponding encoding result after the previous frame is encoded.

[0040] In predicting the bit rate of the current video frame, if the content analysis result of the current video frame and the target quantization parameter are used for bit rate prediction, from the perspective of the needs of the current video frame, combined with the constraints of the target quantization parameter, the modification of the target quantization parameter and the current video frame is obtained; if the frame information and target quantization parameter of the previous output frame are used for bit rate prediction, the encoding result of the previous output frame is used as the bit rate reference of the current video frame, and the target quantization parameter is used for constraint, and the predicted bit rate that meets the target quantization parameter and the processing performance of the encoder is obtained; if the content analysis result of the current video frame, the frame information of the previous output frame and the target quantization parameter are used for bit rate prediction, the effects of the above two methods are combined, and the reliability of the obtained predicted bit rate is further improved.

[0041] Step S103: determining a target bit rate for the current video frame based on the predicted bit rate and the current bandwidth.

[0042] The predicted bitrate represents the bitrate requirement for the current video frame, while the current bandwidth represents the currently available bandwidth. These two factors are combined to determine the target bitrate used to encode the current video frame. When determining the target bitrate, the bitrate requirement must be met within the available bandwidth.

[0043] In some optional implementations, the above step S103 includes:

[0044] Step a1: Query the minimum value of the predicted bit rate and the current bandwidth.

[0045] Step a2: determine the minimum value as the target bit rate.

[0046] Compare the predicted bitrate with the current bandwidth and take the minimum of the two. This minimum is used as the target bitrate for encoding the current video frame. The minimum of the predicted bitrate and the current bandwidth is used as the target bitrate to avoid wasting bandwidth.

[0047] Step S104 : Encode the current video frame based on the target bit rate to determine the current output frame.

[0048] After determining the target bit rate, the current video frame is encoded at the target bit rate to obtain the current output frame. When the encoded current output frame is subsequently transmitted, it can be split into multiple data packets for transmission, etc., and no specific limitation is made here.

[0049] The video processing method provided in this embodiment predicts the bitrate based on video frame information and target quantization parameters. Because the video frame information includes the content analysis results of the current video frame and / or the frame information of the previous output frame, the predicted bitrate can meet both the video frame requirements and the target quantization parameters. The target bitrate for the current video frame is determined based on the current bandwidth, further avoiding waste of bandwidth resources. Finally, the target bitrate is used to encode the current video frame, enabling dynamic adjustment of the encoder bitrate, thus avoiding waste of bandwidth resources while still meeting the definition target.

[0050] In this embodiment, a video processing method is provided, which can be used in electronic devices such as computers, servers, etc. FIG2 is a flow chart of the video processing method according to an embodiment of the present disclosure. As shown in FIG2 , the flow chart includes the following steps:

[0051] Step S201: Obtain the current video frame and the current bandwidth corresponding to the current video frame. Please refer to the description of step S101 of the embodiment shown in FIG1 for details, which will not be repeated here.

[0052] Step S202 : performing bit rate prediction based on the video frame information and the target quantization parameter to obtain a predicted bit rate of the current video frame.

[0053] The video frame information includes a content analysis result of the current video frame and / or frame information of a previous output frame, where the previous output frame is an output frame corresponding to the previous video frame.

[0054] Specifically, the above step S202 includes:

[0055] Step S2021: Obtain the frame information of the last output frame.

[0056] The frame information of the last output frame includes the quantization parameter and size of the last output frame.

[0057] The Quantizer Parameter (QP) is used to characterize the compression of spatial detail. For example, a smaller QP indicates finer quantization, higher bitrates, and higher image quality. A larger QP indicates loss of detail, lower bitrates, and lower image quality. The quantization parameter and size of the previous output frame can be obtained based on the encoding results of the previous frame.

[0058] Step S2022: performing complexity analysis based on the content of the current video frame to obtain a content analysis result of the current video frame.

[0059] Complexity analysis of the content of the current video frame includes, but is not limited to, texture complexity or motion complexity of the video frame. Complexity calculation can be performed by dividing the current video frame into blocks and obtaining the complexity by comparing the differences between the blocks. If texture complexity needs to be determined, the texture differences between the blocks are compared; if motion complexity needs to be determined, the differences in moving objects between the blocks are compared. Of course, the above is only an example of calculating complexity and does not limit the scope of protection of the present disclosure.

[0060] The content analysis result of the current video frame is represented by the complexity analysis result. The higher the complexity, the higher the requirement for image clarity. Correspondingly, the higher the bit rate requirement, so that higher image quality can be displayed when the output frame is subsequently played back.

[0061] Step S2023 , performing bit rate prediction based on the content analysis result of the current video frame, the frame information of the previous output frame, and the target quantization parameter to obtain a predicted bit rate.

[0062] As described above, the content analysis results of the current video frame are used to characterize the bitrate requirement for the current video frame. The frame information of the previous output frame is used to characterize the encoder's encoding rate or compression efficiency. The target quantization parameter is used to characterize the definition constraint. Using the content analysis results of the current video frame, the frame information of the previous output frame, and the target quantization parameter for bitrate prediction ensures the reliability of the predicted bitrate.

[0063] Bitrate prediction can be achieved using a bitrate prediction model. This prediction model is based on a neural network. Its input is the content analysis results of the current sample frame, the frame information of the previous sample output frame, and the sample quantization parameters. The output is the sample bitrate. Alternatively, other methods can be used for bitrate prediction, including but not limited to curve fitting.

[0064] In some optional implementations, the bit rate prediction is implemented based on a bit rate prediction model. Based on this, the bit rate prediction model is determined in the following ways:

[0065] Step b1, obtaining the content analysis result of the current sample frame, the frame information of the previous sample output frame, the sample quantization parameter and the sample bit rate, where the previous sample output frame is the output frame corresponding to the previous sample frame.

[0066] Step b2: performing fitting based on the content analysis result of the current sample frame, the frame information of the previous sample output frame, the sample quantization parameter, and the sample bit rate to obtain a bit rate prediction model.

[0067] Specifically, multiple sets of sample data are collected, each including the content analysis results of the current sample frame, the frame information of the previous sample frame, the sample quantization parameter, and the sample bitrate. The first three parameters are used as independent variables, and the sample bitrate is used as the dependent variable. A curve fitting method is used to determine the relationship between the content analysis results of the current sample frame, the frame information of the previous sample frame, the sample quantization parameter, and the sample bitrate. This relationship is characterized by a bitrate prediction model.

[0068] The bit rate prediction is performed through a bit rate prediction model, and the bit rate prediction model is obtained through offline fitting. On the basis of ensuring the prediction accuracy, the method of determining the bit rate prediction model is simplified.

[0069] Step S203: Determine the target bit rate of the current video frame based on the predicted bit rate and the current bandwidth. Please refer to the detailed description of step S103 in the embodiment shown in FIG1 for details, which will not be repeated here.

[0070] Step S204: Encode the current video frame based on the target bit rate to determine the current output frame. For details, please refer to the detailed description of step S104 in the embodiment shown in FIG1 , which will not be repeated here.

[0071] The video processing method provided in this embodiment performs bit rate prediction based on the complexity of the current video frame and the quantization parameter and size of the previous output frame, wherein the content analysis result and target quantization parameter of the current video frame represent the bit rate requirement of the current video frame, and the frame information of the previous output frame represents the current compression efficiency of the encoder. The predicted bit rate is obtained by combining the requirement and the current compression efficiency, thereby improving the reliability of the obtained predicted bit rate.

[0072] In this embodiment, a video processing method is provided, which can be used in electronic devices such as computers, servers, etc. FIG3 is a flow chart of the video processing method according to an embodiment of the present disclosure. As shown in FIG3 , the flow chart includes the following steps:

[0073] Step S301: Obtain the current video frame and the current bandwidth corresponding to the current video frame. Please refer to the detailed description of step S101 in the embodiment shown in FIG1 for details, which will not be repeated here.

[0074] Step S302 : performing bit rate prediction based on the video frame information and the target quantization parameter to obtain a predicted bit rate of the current video frame.

[0075] The video frame information includes a content analysis result of the current video frame and / or frame information of a previous output frame, where the previous output frame is an output frame corresponding to the previous video frame.

[0076] Specifically, the above step S302 includes:

[0077] Step S3021: Obtain the frame information of the last output frame and the historical bit rate corresponding to the last output frame.

[0078] The frame information of the last output frame includes the definition of the last output frame.

[0079] The frame information of the last output frame includes the definition of the last output frame. The historical bitrate corresponding to the last output frame is the bitrate used to encode the last video frame. The definition of the last output frame may be determined by, but is not limited to, Peak Signal-to-Noise Ratio (PSNR). Other methods may also be used to determine the definition, which are not limited herein.

[0080] Step S3022 : Based on the definition of the previous output frame and the target definition corresponding to the target quantization parameter, the historical bit rate is adjusted to obtain a predicted bit rate.

[0081] The clarity of the previous output frame represents the encoding result of the previous video frame, and the historical bitrate is represented by the clarity. The target quantization parameter represents the target clarity. The clarity of the previous output frame is compared with the target clarity represented by the target quantization parameter to determine whether the historical bitrate needs to be adjusted, thereby obtaining the predicted bitrate.

[0082] In some optional implementations, the above step S3022 includes:

[0083] Step c1: determining a bit rate adjustment amount based on a relationship between the definition of the previous output frame and the target definition.

[0084] Step c2: adjusting the historical bit rate based on the bit rate adjustment amount to obtain a predicted bit rate.

[0085] The resolution of the previous output frame is compared with the target resolution, and the bitrate adjustment amount is first determined. The historical bitrate is then adjusted based on the bitrate adjustment amount to obtain the predicted bitrate. The bitrate adjustment amount can be a fixed value or a value set according to actual needs. For example, if the resolution of the previous output frame is greater than the target resolution, it means that the current output frame resolution needs to be reduced to avoid bandwidth waste. Therefore, the corresponding bitrate adjustment amount needs to be less than zero. If the resolution of the previous output frame is less than the target resolution, it means that the current output frame resolution needs to be increased to meet the resolution requirement. Therefore, the corresponding bitrate adjustment amount needs to be greater than zero.

[0086] After obtaining the corresponding bit rate adjustment amount, the historical bit rate is adjusted using the bit rate adjustment amount to obtain the predicted bit rate.

[0087] Based on the relationship between the clarity of the previous output frame and the target clarity, a rate adjustment amount is obtained, and the historical rate is adjusted using the rate adjustment amount. The adjustment method is simple and the efficiency of determining the predicted rate is improved.

[0088] In some optional implementations, the above step b1 includes:

[0089] Step b11: determining a difference level corresponding to the difference based on the difference between the definition of the previous output frame and the target definition.

[0090] Step b12: determining a bit rate adjustment amount based on the difference level.

[0091] Calculate the difference between the definition of the previous output frame and the target definition, compare the calculated difference with each difference level, and obtain the difference level corresponding to the difference. For example, the difference corresponding to difference level 1 is (0, 2], the difference corresponding to difference level 2 is (2, 5], and the difference corresponding to difference level 3 is (5, 10]. If the current calculated difference is 3, the corresponding difference level is 2.

[0092] Different difference levels correspond to different bitrate adjustments. A larger difference corresponds to a higher bitrate adjustment, and a higher difference level results in a higher bitrate adjustment. This allows for hierarchical bitrate adjustment, minimizing the difference between adjacent output frames.

[0093] In some optional implementations, if the definition of the last output frame is greater than the target definition, the bit rate adjustment amount is determined to be less than zero. If the definition of the last output frame is less than the target definition, the bit rate adjustment amount is determined to be greater than zero.

[0094] When the clarity of the previous output frame is greater than the target clarity, the historical bit rate needs to be lowered to reduce bandwidth waste, so the determined bit rate adjustment amount is less than zero; when the clarity of the previous output frame is less than the target clarity, the historical bit rate needs to be increased to meet the target clarity requirements, so the determined bit rate adjustment amount is greater than zero.

[0095] In some optional implementations, a method for determining the target clarity corresponding to the target quantization parameter includes:

[0096] Step d1, obtaining the type of the current encoder.

[0097] Step d2: determining a target quantization parameter based on the type of the current encoder to obtain a target definition.

[0098] The target quantization parameter can be fixed or set based on the current usage environment. Alternatively, different encoders may use different values ​​in the same usage environment. The target quantization parameter is determined by obtaining the current encoder type. There is a correspondence between encoder types and quantization parameters. After obtaining the current encoder type, the target quantization parameter can be obtained by querying this correspondence, thereby obtaining the target definition.

[0099] The corresponding target quantization parameter is obtained according to the type of the current encoder, thereby ensuring that the target quantization parameter corresponds to the target definition and the type of the time domain encoder, so as to maximize the encoding performance of the current encoder.

[0100] Step S303: Determine the target bit rate of the current video frame based on the predicted bit rate and the current bandwidth. Please refer to the detailed description of step S103 in the embodiment shown in FIG1 for details, which will not be repeated here.

[0101] Step S304: Encode the current video frame based on the target bit rate to determine the current output frame. For details, please refer to the detailed description of step S104 in the embodiment shown in FIG1 , which will not be repeated here.

[0102] The video processing method provided in this embodiment uses the cleanliness and target clarity of the previous output frame to adjust the historical bit rate of the previous output frame to obtain a predicted bit rate. Since the difference between the clarity of the previous output frame and the target clarity can represent the space for bit rate adjustment, the historical bit rate is adjusted based on this, ensuring the reliability of the obtained predicted bit rate.

[0103] As a specific application embodiment of the embodiment of the present disclosure, as shown in FIG4 , in the process of real-time video communication, an initial bit rate is set to start encoding the obtained first video frame to obtain a first output frame. After obtaining the first output frame, the bit rate is predicted by the bit rate prediction model based on the complexity calculation result of the second video frame, the quantization parameter of the first output frame and the size of the first output frame, combined with the target quantization parameter, to obtain a predicted bit rate. The current bandwidth is compared with the predicted bit rate, and the minimum value is taken as the target bit rate. The second video frame is encoded at the target bit rate to obtain a second output frame. By analogy, that is, for the target bit rate of the current video frame, the predicted bit rate is determined by combining the quantization parameter of the previous output frame and the size of the output frame, the complexity calculation result of the current video frame, and the target quantization parameter, and then the bandwidth constraint is performed based on the current bandwidth corresponding to each current video frame, thereby obtaining the target bit rate of the current video frame.

[0104] As another specific application embodiment of the embodiment of the present disclosure, as shown in FIG5 , for the first video frame, the initial bit rate is used to encode it to obtain the first output frame. For the target bit rate of the second video frame, the clarity of the first output frame is first calculated, and the calculated clarity is compared with the target clarity. If the calculated clarity is higher than the target clarity, the historical bit rate corresponding to the first video frame is lowered; if the calculated clarity is higher than the target clarity, the historical bit rate corresponding to the first video frame is increased. By adjusting the historical bit rate, the target bit rate corresponding to the second video frame is obtained, and the second video frame is encoded using the target bit rate to obtain the second output frame. And so on, taking the video frame as the minimum processing unit, the target bit rate corresponding to each video frame is determined.

[0105] In this embodiment, a video processing device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0106] This embodiment provides a video processing device, as shown in FIG6 , including:

[0107] The acquisition module 601 is configured to acquire a current video frame and a current bandwidth corresponding to the current video frame.

[0108] The bit rate prediction module 602 is used to perform bit rate prediction based on video frame information and target quantization parameters to obtain the predicted bit rate of the current video frame. The video frame information includes the content analysis result of the current video frame and / or the frame information of the previous output frame, and the previous output frame is the output frame corresponding to the previous video frame.

[0109] The bit rate determination module 603 is configured to determine a target bit rate for the current video frame based on the predicted bit rate and the current bandwidth.

[0110] The encoding module 604 is configured to encode the current video frame based on a target bit rate and determine a current output frame.

[0111] In some optional implementations, the bit rate prediction module 602 includes:

[0112] The frame information acquisition unit is used to acquire the frame information of the last output frame, where the frame information of the last output frame includes the quantization parameter and size of the last output frame.

[0113] The complexity analysis unit is used to perform complexity analysis based on the content of the current video frame to obtain a content analysis result of the current video frame.

[0114] The bit rate prediction unit is used to perform bit rate prediction based on the content analysis result of the current video frame, the frame information of the previous output frame and the target quantization parameter to obtain the predicted bit rate.

[0115] In some optional implementations, the bit rate prediction is implemented based on a bit rate prediction model, and a determination unit of the bit rate prediction model includes:

[0116] The acquisition subunit is used to obtain the content analysis result of the current sample frame, the frame information of the previous sample output frame, the sample quantization parameter and the sample bit rate, where the previous sample output frame is the output frame corresponding to the previous sample frame.

[0117] The fitting subunit is used to perform fitting based on the content analysis result of the current sample frame, the frame information of the previous sample output frame, the sample quantization parameter and the sample bit rate to obtain a bit rate prediction model.

[0118] In some optional implementations, the bit rate prediction module 602 includes:

[0119] The acquisition unit is used to acquire the frame information of the last output frame and the historical bit rate corresponding to the last output frame, wherein the frame information of the last output frame includes the definition of the last output frame.

[0120] The historical bit rate adjustment unit is used to adjust the historical bit rate based on the definition of the previous output frame and the target definition corresponding to the target quantization parameter to obtain a predicted bit rate.

[0121] In some optional implementations, the historical bit rate adjustment unit includes:

[0122] The bit rate adjustment amount determining subunit is used to determine the bit rate adjustment amount based on the relationship between the definition of the previous output frame and the target definition.

[0123] The bit rate adjustment subunit is used to adjust the historical bit rate based on the bit rate adjustment amount to obtain the predicted bit rate.

[0124] In some optional implementations, the bit rate adjustment amount determination subunit includes:

[0125] The difference level determination subunit is used to determine the difference level corresponding to the difference based on the difference between the definition of the previous output frame and the target definition.

[0126] The bit rate adjustment amount determining subunit is configured to determine the bit rate adjustment amount based on the difference level.

[0127] In some optional implementations, the bit rate adjustment amount determination subunit includes:

[0128] The first bit rate adjustment amount determining subunit is configured to determine that the bit rate adjustment amount is less than zero if the definition of the previous output frame is greater than the target definition.

[0129] The second bit rate adjustment amount determining subunit is configured to determine that the bit rate adjustment amount is greater than zero if the definition of the previous output frame is less than the target definition.

[0130] In some optional implementations, the unit for determining the target clarity corresponding to the target quantization parameter includes:

[0131] The type acquisition subunit is used to obtain the type of the current encoder.

[0132] The target definition determination subunit is configured to determine a target quantization parameter based on the type of the current encoder to obtain a target definition.

[0133] In some optional implementations, the bit rate determination module 603 includes:

[0134] The minimum value query unit is used to query the minimum value of the predicted bit rate and the current bandwidth.

[0135] The target bit rate determining unit is configured to determine the minimum value as the target bit rate.

[0136] The video processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0137] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0138] An embodiment of the present disclosure further provides an electronic device having the video processing device shown in FIG. 6 .

[0139] Please refer to Figure 7, which is a structural diagram of an electronic device provided by an optional embodiment of the present disclosure. As shown in Figure 7, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 takes a processor 10 as an example.

[0140] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0141] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0142] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0143] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0144] The electronic device further includes a communication interface 30 for the electronic device to communicate with other devices or a communication network.

[0145] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0146] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0147] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0148] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0149] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0150] Although the embodiments of the present disclosure have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A video processing method, comprising: Obtain a current video frame and a current bandwidth corresponding to the current video frame; Performing bit rate prediction based on video frame information and a target quantization parameter to obtain a predicted bit rate of the current video frame, wherein the video frame information includes a content analysis result of the current video frame and / or frame information of a previous output frame, where the previous output frame is an output frame corresponding to the previous video frame; Determining a target bit rate of the current video frame based on the predicted bit rate and the current bandwidth; The current video frame is encoded based on the target bit rate to determine a current output frame.

2. The method according to claim 1, wherein the bit rate prediction based on the video frame information and the target quantization parameter to obtain the predicted bit rate of the current video frame comprises: Acquire frame information of the last output frame, where the frame information of the last output frame includes a quantization parameter and a size of the last output frame; Performing complexity analysis based on the content of the current video frame to obtain a content analysis result of the current video frame; The bit rate is predicted based on the content analysis result of the current video frame, the frame information of the previous output frame and the target quantization parameter to obtain the predicted bit rate.

3. The method according to claim 2, wherein the bit rate prediction is implemented based on a bit rate prediction model, and the determination method of the bit rate prediction model includes: Obtaining a content analysis result of a current sample frame, frame information of a previous sample output frame, a sample quantization parameter, and a sample bit rate, wherein the previous sample output frame is an output frame corresponding to the previous sample frame; The bit rate prediction model is obtained by performing fitting based on the content analysis result of the current sample frame, the frame information of the previous sample output frame, the sample quantization parameter and the sample bit rate.

4. The method according to claim 1, wherein the bit rate prediction based on the video frame information and the target quantization parameter to obtain the predicted bit rate of the current video frame comprises: Acquire frame information of the last output frame and a historical bit rate corresponding to the last output frame, wherein the frame information of the last output frame includes the definition of the last output frame; Based on the definition of the previous output frame and the target definition corresponding to the target quantization parameter, the historical bit rate is adjusted to obtain the predicted bit rate.

5. The method according to claim 4, wherein the adjusting the historical bit rate based on the definition of the previous output frame and the target definition corresponding to the target quantization parameter to obtain the predicted bit rate comprises: Determining a bit rate adjustment amount based on a relationship between the definition of the previous output frame and the target definition; The historical bit rate is adjusted based on the bit rate adjustment amount to obtain the predicted bit rate.

6. The method according to claim 5, wherein the determining the bit rate adjustment amount based on the size relationship between the definition of the previous output frame and the target definition corresponding to the target quantization parameter comprises: Based on the difference between the definition of the previous output frame and the target definition, determining a difference level corresponding to the difference; The bit rate adjustment amount is determined based on the difference level.

7. The method according to claim 5, wherein the determining the bit rate adjustment amount based on the relationship between the definition of the previous output frame and the target definition comprises: If the definition of the previous output frame is greater than the target definition, determining that the bit rate adjustment amount is less than zero; If the definition of the previous output frame is less than the target definition, it is determined that the bit rate adjustment amount is greater than zero.

8. The method according to claim 4, wherein the target definition corresponding to the target quantization parameter is determined by: Get the type of the current encoder; The target quantization parameter is determined based on the type of the current encoder to obtain the target definition.

9. The method according to any one of claims 1 to 8, wherein determining the target bit rate of the current video frame based on the predicted bit rate and the current bandwidth comprises: Querying a minimum value of the predicted bit rate and the current bandwidth; The minimum value is determined as the target bit rate.

10. A video processing device, comprising: An acquisition module, used to acquire a current video frame and a current bandwidth corresponding to the current video frame; A bit rate prediction module, configured to perform bit rate prediction based on video frame information and a target quantization parameter to obtain a predicted bit rate of the current video frame, wherein the video frame information includes a content analysis result of the current video frame and / or frame information of a previous output frame, wherein the previous output frame is an output frame corresponding to the previous video frame; A bit rate determination module, configured to determine a target bit rate of the current video frame based on the predicted bit rate and the current bandwidth; The encoding module is used to encode the current video frame based on the target bit rate to determine a current output frame.

11. An electronic device, comprising: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the video processing method according to any one of claims 1 to 9 by executing the computer instructions. 12 . A computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, wherein the computer instructions are used to enable a computer to execute the video processing method according to claim 1 .

Images