Content-adaptive video encoding method, device, equipment and storage medium

Abstract

The embodiment of the present invention discloses a content-adaptive video encoding method, device, equipment and storage medium, the method comprising: obtaining video data to be encoded, dividing the video data into a plurality of image sets including continuous frame images; determining encoding features of the image set, inputting the encoding features and set video picture evaluation parameters into a pre-trained machine learning model to output a bit rate control parameter; encoding the image set according to the encoding features and the bit rate control parameter. This solution improves the video encoding efficiency and is also applicable to real-time video scenarios.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video processing technology, and in particular to a content-adaptive video encoding method, apparatus, device, and storage medium. Background Art

[0002] With the rapid development of mobile Internet technology, video has become the mainstream media used by users. Video live broadcast, on-demand, short videos and video chat have become part of people's lives. However, compared with text and pictures, the amount of video data is very large, and the transmission and storage of videos also face huge challenges. Video coding and decoding technology is to achieve the highest possible compression ratio and the highest possible video reconstruction quality within the available computing resources to meet the requirements of storage capacity and bandwidth. Early video service providers usually use a pre-determined universal encoding configuration to process almost all video content. In this way, there may be insufficient bitrate for highly moving videos, resulting in low encoding quality, and there may be a problem of bitrate waste for slow-moving videos. Content adaptive coding sets different encoding configurations for different videos according to the video content, and finds the minimum bitrate that meets the requirements of clarity and subjective sensitivity for each video or video clip, thereby saving bandwidth.

[0003] When encoding a video, the encoded data is extracted as features by pre-encoding the training video data, and the machine learning model is trained in combination with the corresponding constant bit rate coefficient value. In the production environment, the model is used to predict the encoding parameters based on the video features, and then the predicted values ​​are used for encoding, so as to strike a balance between the encoding bit rate and the encoding quality, and improve the viewing experience of most viewers. However, this encoding method extracts features by encoding the entire video, and then uses the machine learning model to predict the encoding constant bit rate coefficient value of the entire video. For long videos containing complex and mixed content, this method will result in poor encoding quality of the complex part of the video and waste of bit rate of the simple part. At the same time, in the encoding process, firstly encode the entire video to extract features and predict the constant bit rate coefficient value, and then encode according to the predicted value, which takes a lot of time and is not suitable for live broadcast scenarios. Summary of the invention

[0004] The embodiments of the present invention provide a content-adaptive video encoding method, apparatus, device and storage medium, which solve the problem in the prior art that video encoding has unsatisfactory encoding effect in complex scenes, improve the video encoding efficiency, and are suitable for real-time video scenes.

[0005] In a first aspect, an embodiment of the present invention provides a content-adaptive video encoding method, the method comprising:

[0006] Acquire video data to be encoded, and divide the video data into a plurality of image sets including continuous frame images;

[0007] Determine the coding features of the image set, and input the coding features and the set video picture evaluation parameters into a pre-trained machine learning model to output a bit rate control parameter;

[0008] The image set is encoded according to the encoding feature and the rate control parameter.

[0009] In a second aspect, an embodiment of the present invention further provides a content adaptive video encoding device, including:

[0010] An image set determination module is used to obtain video data to be encoded and divide the video data into a plurality of image sets including continuous frame images;

[0011] A bit rate parameter determination module, used to determine the coding features of the image set, and input the coding features and the set video picture evaluation parameters into a pre-trained machine learning model to output a bit rate control parameter;

[0012] The encoding module is used to encode the image set according to the encoding characteristics and the bit rate control parameters.

[0013] In a third aspect, an embodiment of the present invention further provides a content adaptive video encoding device, the device comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the content adaptive video encoding method described in the embodiment of the present invention.

[0017] In a fourth aspect, an embodiment of the present invention further provides a storage medium storing computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute the content adaptive video encoding method described in the embodiment of the present invention.

[0018] In an embodiment of the present invention, by acquiring video data to be encoded, the video data is divided into multiple image sets including continuous frame images, the encoding features of the image sets are determined, the encoding features and the set video picture evaluation parameters are input into a pre-trained machine learning model to output a bit rate control parameter, and the image set is encoded according to the encoding features and the bit rate control parameter. This solves the problem in the prior art that the encoding effect of video encoding in complex scenes is not ideal, improves the video encoding efficiency, and is suitable for real-time video scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a content-adaptive video encoding method provided by an embodiment of the present invention;

[0020] Figure 2 A flowchart of a method for performing secondary encoding based on a primary encoding result provided by an embodiment of the present invention;

[0021] Figure 3 A flowchart of another content-adaptive video encoding method provided by an embodiment of the present invention;

[0022] Figure 4 A flowchart of another content-adaptive video encoding method provided by an embodiment of the present invention;

[0023] Figure 5 A structural block diagram of a content adaptive video encoding device provided by an embodiment of the present invention;

[0024] Figure 6 A schematic structural diagram of a content adaptive video encoding device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention, rather than to limit the embodiments of the present invention. It is also necessary to explain that, for ease of description, only parts related to the embodiments of the present invention are shown in the accompanying drawings, rather than all structures.

[0026] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0027] Figure 1 A flowchart of a content-adaptive video encoding method provided by an embodiment of the present invention can be applied to encoding video data. The method can be executed by a computing device such as a notebook, a desktop, a smart phone, a server, and a tablet computer, and specifically includes the following steps:

[0028] Step S101: Obtain video data to be encoded, and divide the video data into a plurality of image sets including continuous frame images.

[0029] The video data to be encoded includes recorded video data and real-time generated video data that needs to be transmitted and displayed, such as live video data.

[0030] In one embodiment, when encoding video data, for a segment of video data, the video data is first divided into a plurality of image sets containing continuous frame images. That is, when video encoding is performed, separate video encoding is performed for each subdivided image set. Exemplarily, the video data may be divided into a plurality of continuous GOPs (Group of pictures), each GOP representing a group of continuous pictures in a coded video stream. For example, each GOP contains 15 or 20 frames of images, that is, the video data to be encoded is divided into a plurality of continuous image sets, each image set contains 15 to 20 frames of images, that is, the video data is encoded using GOP as the encoding unit.

[0031] Step S102: determine the coding features of the image set, and input the coding features and the set video picture evaluation parameters into the pre-trained machine learning model to output the bit rate control parameters.

[0032] In one embodiment, the coding feature of the image set may be determined by pre-coding to obtain the coding feature of the image set, such as encoding the image set using an encoder to obtain the corresponding coding feature.

[0033] In one embodiment, the encoding features of the image set are obtained by extracting and analyzing the features of each frame of the image set. Optionally, the encoding features include motion vector features, distortion parameters, complexity parameters, etc. for describing each frame of the image set. Among them, the motion vector feature is used to characterize the degree of change of the image, wherein the more drastic the change between each frame of the image, the larger the motion vector is relatively, and vice versa if each frame of the image describes a static picture, the motion vector is relatively smaller; the distortion parameter is used to characterize the degree of distortion of the image, the greater the degree of distortion of the image, the higher the parameter value is, and vice versa if the degree of distortion of the image is low, the corresponding parameter value is also relatively low; the complexity parameter is used to characterize the complexity of the image, such as the image contains multiple different objects, the greater the pixel difference between each object, the higher its complexity is. Optionally, the identification and determination of the above encoding features can be realized by existing encoder modules and image processing algorithms.

[0034] Among them, the video picture evaluation parameter is a comprehensive evaluation index used to characterize the image quality. Optionally, the video picture evaluation parameter can be characterized by VMAF (Video Multimethod Assessment Fusion). Among them, VMAF is an objective evaluation index proposed by Netflix that combines human visual modeling and machine learning. VMAF uses a large amount of subjective data as a training set, and integrates algorithms of different evaluation dimensions through machine learning. It is currently a more mainstream objective evaluation index. It can generally be considered that the higher the VMAF score, the better the video quality. However, from the perspective of human eye perception, when the VMAF score of the same video is increased to a certain threshold, the human eye cannot perceive the improvement in picture quality. Therefore, different VMAF values ​​can be designed for different videos to save encoding bit rate without changing the subjective quality of the video.

[0035] In one embodiment, the coding features of the determined image set and the set video picture evaluation parameters are input into the pre-trained machine learning model to output the bit rate control parameters, wherein the set video picture evaluation parameters can be customized according to different picture quality requirements, different playback devices, etc., and the set values ​​can also be adjusted. The input machine learning model is a pre-trained neural network model, which can output the corresponding bit rate control parameters based on the coding features of the image set and the set video picture evaluation parameter input. Optionally, the bit rate control parameter can be CRF (Constant Rate Factor, constant bit rate coefficient) or CQF (Constant Quality Factor, constant quality factor). CRF is a type of bit rate control. The smaller the CRF value, the higher the video quality, and the file size will also increase; the larger the CRF value, the higher the video compression rate, but the lower the video quality. Optionally, different CRF values ​​correspond to different bit rates, and different CRF values ​​and corresponding encoding bit rates can be recorded in a mapping table, or the relationship between CRF and bit rate can be characterized in a function curve.

[0036] Step S103: Encode the image set according to the encoding feature and the bit rate control parameter.

[0037] In one embodiment, after the rate control parameters are obtained through the machine learning model, the image set is finally re-encoded based on the rate control parameters and the coding features determined in step S101 to output bitstream data.

[0038] Specifically, Figure 2 A flowchart of a method for performing secondary encoding based on a primary encoding result provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, specifically including:

[0039] Step S1031: Determine frame type information and scene information according to the coding feature.

[0040] The coding features record the frame type of each frame, such as different frame types divided into I frame, P frame and B frame. Different frame type information requires different quality coding compression due to their different reference reference relationships. The I frame represents a key frame, which is a completely retained frame. When decoding, only the frame data is needed to complete the image decoding without referring to other frames; the P frame represents the difference between the frame and the previous key frame or P frame. When decoding, the previous cached picture needs to be superimposed on the difference defined by the current frame to generate the final picture; the B frame represents a two-way difference frame, that is, the B frame records the difference between the current frame and the previous and next frames. When decoding the B frame image, not only the previous cached picture needs to be obtained, but also the picture after decoding, and the final picture is obtained by superimposing the previous and next pictures with the current frame data.

[0041] The scene information can be exemplarily divided into moving scenes and static scenes. It can be determined by an integrated scene discrimination module based on coding features. The coding features record the image features related to the motion displacement change, such as the motion vector and motion compensation of each frame of the image, and the scene information of the image can be determined by analyzing the motion vector, motion compensation and other data.

[0042] Step S1032: performing prediction analysis according to the frame type information, the scene information and the bit rate control parameter to obtain encoding parameters.

[0043] Here, the coding parameter takes HEVC (High Efficiency Video Coding) as an example, and its corresponding quantization parameter QP (quantization parameter). The quantization parameter QP is the serial number of the quantization step Qstep. For brightness coding, the quantization step Qstep has 52 values, and the QP value ranges from 0 to 51. For chrominance coding, the QP value ranges from 0 to 39.

[0044] Taking the quantization parameter QP as an example, this coding parameter reflects the compression of spatial details. The smaller the coding parameter value, the finer the quantization, the higher the image quality, and the longer the generated bitstream; if the quantization parameter QP value is small, most of the details in the image will be retained. When the quantization parameter QP value increases, some details in the image will be lost and the bit rate will be reduced. Taking the above QP value of 0 to 51 as an example, when QP takes the minimum value of 0, it means the finest quantization; on the contrary, when QP takes the maximum value of 51, it means the coarsest quantization. Quantization is to reduce the image coding length without reducing the visual effect, and reduce unnecessary information in visual restoration.

[0045] Specifically, the process of predicting and analyzing the encoding parameters based on the frame type information, scene information and bit rate control parameters can be implemented using the integrated encoder module of HEVC high-efficiency video coding as an example. That is, different frame type information (I frame, B frame, P frame), scene information (static scene, dynamic scene), and bit rate control parameter (CRF) jointly determine the final encoding parameter (frame-level QP). Exemplarily, the higher the frame type is a key frame, the scene information is a dynamic scene, and the value of the bit rate control parameter is, the lower the frame-level QP value is determined to be.

[0046] Step S1033: Encode the image set based on the encoding parameters.

[0047] In one embodiment, after the encoding parameters are obtained, HEVC high-efficiency video encoding is performed by taking the frame-level QP parameter in HEVC high-efficiency video encoding as an example to achieve code stream output.

[0048] In another embodiment, in order to improve the accuracy of secondary encoding, the above-mentioned process of performing predictive analysis to obtain encoding parameters and encoding the image set based on the encoding parameters includes: performing predictive analysis to obtain a first encoding parameter; determining a second encoding parameter based on the first encoding parameter, encoding feedback information, cache information, frame type information, and scene information; adjusting the quantization offset parameter according to the first encoding parameter; encoding the image set according to the second encoding parameter and the adjusted quantization offset parameter to output code stream data. Wherein, taking HEVC encoding as an example, the first encoding parameter can be understood as the base QP information (base QP), which determines the frame-level QP information according to the first encoding parameter, encoding feedback information, cache information, frame type information, and scene information. Wherein, the cache information represents the parameters of the buffer memory during the video encoding process. The larger the cache occupancy, the larger the corresponding QP value is to reduce the amount of computation and storage of video encoding. The encoding feedback information can be information obtained during the pre-encoding process or feedback after the previous round of encoding of this image set or video, such as the degree of distortion. If the degree of distortion is higher, the QP value needs to be reduced accordingly to improve the encoding quality. While determining the second encoding parameter according to the first encoding parameter, the quantization offset parameter is further adjusted according to the first encoding parameter. Taking HEVC video encoding as an example, the quantization offset parameter can be characterized by Cutree intensity, which represents the quantization offset adjustment according to the degree to which the current block is referenced. Specifically, if the current block is referenced, it is further determined whether a certain number of blocks after the current block refer to the current block. If the number of blocks referenced by subsequent image blocks is greater, it indicates that the current block belongs to a slowly changing scene, and the QP value is correspondingly lowered to improve the image quality. Finally, the determined second encoding parameter and the determined quantization offset parameter are used to comprehensively encode the image set to output the code stream data to ensure the optimal balance between image quality and compression rate in the encoding effect.

[0049] It can be seen from the above scheme that when performing video encoding, the video is first divided into an image set, which is encoded once to obtain the encoding features, and then the trained machine learning model is used to output accurate bit rate control parameters. Then, based on the bit rate control parameters and the encoding features obtained during the first encoding process, the image set is encoded twice to finally obtain the video encoding result. This method uses the video live content adaptive coding technology of two encodings and machine learning. By utilizing HEVC's multiple encodings and machine learning prediction models, the encoding configuration is dynamically adjusted according to the complexity of the video content, content adaptive coding is achieved, and the video smoothness and clarity are better balanced. It can be applied to real-time live video scenarios, and the video encoding effect is good.

[0050] Figure 3A flowchart of another content-adaptive video encoding method provided by an embodiment of the present invention provides a method for determining encoding features of an image set, such as Figure 3 As shown, specifically including:

[0051] Step S201: Obtain video data to be encoded, and divide the video data into a plurality of image sets including continuous frame images.

[0052] Step S202: obtaining a preset number of frame images in the image set, encoding the preset number of frame images to obtain encoding features, and determining the encoding features as encoding features of the image set.

[0053] In one embodiment, taking the image set as a GOP image as an example, the preset number of frame images may be miniGOP images in a GOP image, that is, taking the image set as a GOP image with 15 frames as an example, the preset number of frame images may be 5 frame images therein. The process of encoding the preset number of frame images may be pre-encoding by an encoder to obtain encoding features. The encoding features of the preset number of frame images are then determined as the encoding features of the image set.

[0054] Step S203: input the encoding features and the set video picture evaluation parameters into the pre-trained machine learning model to output the bit rate control parameters.

[0055] Step S204: Encode the image set according to the encoding feature and the bit rate control parameter.

[0056] It can be seen from the above scheme that the video live content adaptive coding technology using two-time encoding and machine learning is used in the video encoding process, and the coding configuration is dynamically adjusted according to the complexity of the video content. Among them, by obtaining a preset number of frame images in an image set, encoding the preset number of frame images to obtain coding features, and determining the coding features as the coding features of the image set, the encoding speed can be significantly improved, and the video encoding effect is outstanding for real-time requirements. At the same time, the amount of data calculation is reduced, content adaptive coding is realized, and the video smoothness and clarity are better balanced. It can be applied to real-time live video scenarios, and the video encoding effect is good.

[0057] Figure 4 A flowchart of another content-adaptive video encoding method provided by an embodiment of the present invention provides a specific method for outputting a bit rate control parameter through a machine learning model, wherein the machine learning model includes a joint model composed of a first training model and a second training model, such as Figure 4 As shown, specifically including:

[0058] Step S301: Obtain video data to be encoded, and divide the video data into a plurality of image sets including continuous frame images.

[0059] Step S302: determine the coding features of the image set, input the coding features and the set video picture evaluation parameters into the first training model and the second training model respectively, and obtain the first bit rate control parameter output by the first training model and the second bit rate control parameter output by the second training model.

[0060] In one embodiment, the first training model is an XGBoost model, and the second training model is a LightGBM model, both of which are machine learning algorithms based on decision trees. Exemplarily, the first rate control parameter output by the first training model is recorded as CRF1, and the second rate control parameter output by the second training model is recorded as CRF2.

[0061] Step S303: Perform weighted average calculation on the first rate control parameter and the second rate control parameter to obtain a rate control parameter.

[0062] The finally calculated bit rate control parameter is recorded as CRF3, which is optionally calculated by the formula CRF3 = λ1*CRF1+λ2*CRF2. Where λ1+λ2=1, λ1∈[0,1], λ2∈[0,1].

[0063] Step S304: Encode the image set according to the encoding feature and the bit rate control parameter.

[0064] From the above, it can be seen that when the bit rate control parameters are output through the machine learning model, two different decision tree-based models are used to output the corresponding bit rate control parameters and then weighted average is performed to obtain the final bit rate control parameters, so that the obtained bit rate control parameters are more accurate and the final video encoding effect is better.

[0065] In one embodiment, before the coding features and the set video picture evaluation parameters are input into the first training model and the second training model respectively, it also includes: obtaining video sample data of different scene types and corresponding to different resolutions; dividing the video sample data into training set samples, test set samples and validation set samples, and inputting them into the first training model and the second training model for training respectively. During the model training process, this solution first distinguishes the scene types of the video pictures, such as dynamic scenes and static scenes, and at the same time, trains them separately based on video pictures of different resolutions as sample data. During the training process, the video sample data is divided into training set samples, test set samples and validation set samples to obtain the final training model with good prediction effect.

[0066] Figure 5 This is a structural block diagram of a content adaptive video encoding device provided by an embodiment of the present invention. The device is used to execute the content adaptive video encoding method provided by the above embodiment, and has the corresponding functional modules and beneficial effects of the execution method. Figure 5 As shown, the device specifically includes: an image set determination module 101, a bit rate parameter determination module 102 and an encoding module 103, wherein:

[0067] The image set determination module 101 is used to obtain video data to be encoded and divide the video data into a plurality of image sets including continuous frame images;

[0068] A bit rate parameter determination module 102 is used to determine the coding features of the image set, and input the coding features and the set video picture evaluation parameters into a pre-trained machine learning model to output a bit rate control parameter;

[0069] The encoding module 103 is used to encode the image set according to the encoding feature and the bit rate control parameter.

[0070] It can be seen from the above scheme that when performing video encoding, the video is first divided into an image set, which is encoded once to obtain the encoding features, and then the trained machine learning model is used to output accurate bit rate control parameters. Then, based on the bit rate control parameters and the encoding features obtained during the first encoding process, the image set is encoded twice to finally obtain the video encoding result. This method uses the video live content adaptive coding technology of two encodings and machine learning. By utilizing HEVC's multiple encodings and machine learning prediction models, the encoding configuration is dynamically adjusted according to the complexity of the video content, content adaptive coding is achieved, and the video smoothness and clarity are better balanced. It can be applied to real-time live video scenarios, and the video encoding effect is good.

[0071] In a possible embodiment, the bit rate parameter determination module 102 is specifically configured to:

[0072] Acquire a preset number of frame images from the image set;

[0073] The preset number of frame images are encoded to obtain encoding features, and the encoding features are determined as encoding features of the image set.

[0074] In a possible embodiment, the machine learning model includes a joint model consisting of a first training model and a second training model, and the bit rate parameter determination module 102 is specifically used to:

[0075] Inputting the coding features and the set video picture evaluation parameters into the first training model and the second training model respectively, to obtain a first rate control parameter output by the first training model and a second rate control parameter output by the second training model;

[0076] A rate control parameter is obtained by performing weighted average calculation on the first rate control parameter and the second rate control parameter.

[0077] In a possible embodiment, the bit rate parameter determination module 102 is further configured to:

[0078] Before inputting the coding features and the set video picture evaluation parameters into the first training model and the second training model respectively, obtaining video sample data of different scene types and corresponding to different resolutions;

[0079] The video sample data is divided into training set samples, test set samples and validation set samples, and are respectively input into the first training model and the second training model for training.

[0080] In a possible embodiment, the encoding module 103 is specifically used for:

[0081] Determining frame type information and scene information according to the coding features;

[0082] Performing prediction analysis according to the frame type information, the scene information and the bit rate control parameter to obtain a coding parameter;

[0083] The set of images is encoded based on the encoding parameters.

[0084] In a possible embodiment, the encoding module 103 is specifically used for:

[0085] Performing a prediction analysis to obtain a first encoding parameter;

[0086] Determine a second encoding parameter based on the first encoding parameter, encoding feedback information, cache information, the frame type information, and the scene information.

[0087] In a possible embodiment, the encoding module 103 is specifically used for:

[0088] Adjusting a quantization offset parameter according to the first encoding parameter;

[0089] The image set is encoded according to the second encoding parameter and the adjusted quantization offset parameter to output code stream data.

[0090] Figure 6 A structural diagram of a content adaptive video encoding device provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the device includes a processor 201, a memory 202, an input device 203 and an output device 204; the number of processors 201 in the device can be one or more. Figure 6 A processor 201 is taken as an example; the processor 201, memory 202, input device 203 and output device 204 in the device can be connected by a bus or other means. Figure 6 The example of connection via a bus is taken. The memory 202, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the content adaptive video encoding method in the embodiment of the present invention. The processor 201 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 202, that is, implements the above-mentioned content adaptive video encoding method. The input device 203 can be used to receive input digital or character information, and generate key signal input related to user settings and function control of the device. The output device 204 may include a display device such as a display screen.

[0091] An embodiment of the present invention further provides a storage medium containing computer executable instructions, wherein the computer executable instructions are used to execute a content adaptive video encoding method described in the above embodiment when executed by a computer processor, specifically including:

[0092] Acquire video data to be encoded, and divide the video data into a plurality of image sets including continuous frame images;

[0093] Determine the coding features of the image set, and input the coding features and the set video picture evaluation parameters into a pre-trained machine learning model to output a bit rate control parameter;

[0094] The image set is encoded according to the encoding feature and the rate control parameter.

[0095] It is worth noting that in the embodiment of the above-mentioned content adaptive video encoding device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present invention.

[0096] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the embodiments of the present invention are not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the embodiments of the present invention. Therefore, although the embodiments of the present invention are described in more detail through the above embodiments, the embodiments of the present invention are not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the embodiments of the present invention, and the scope of the embodiments of the present invention is determined by the scope of the appended claims.

Claims

1. A content-adaptive video encoding method, characterized in that: include: Acquire video data to be encoded, and divide the video data into a plurality of image sets including continuous frame images; Determine the coding features of the image set, and input the coding features and the set video picture evaluation parameters into a pre-trained machine learning model to output a bit rate control parameter; Frame type information and scene information are determined according to the encoding features, encoding parameters are obtained by performing predictive analysis according to the frame type information, the scene information and the bit rate control parameters, and the image set is encoded based on the encoding parameters, wherein the encoding parameters include a first encoding parameter obtained when performing predictive analysis, and a second encoding parameter is determined based on the first encoding parameter, encoding feedback information, cache information, the frame type information, and the scene information.

2. The content adaptive video encoding method according to claim 1, characterized in that: The determining of the encoding features of the image set comprises: Acquire a preset number of frame images from the image set; The preset number of frame images are encoded to obtain encoding features, and the encoding features are determined as encoding features of the image set.

3. The content adaptive video encoding method according to claim 1, characterized in that: The machine learning model includes a joint model composed of a first training model and a second training model, and the encoding features and the set video picture evaluation parameters are input into the pre-trained machine learning model to output the bit rate control parameters, including: Inputting the coding features and the set video picture evaluation parameters into the first training model and the second training model respectively, to obtain a first rate control parameter output by the first training model and a second rate control parameter output by the second training model; A rate control parameter is obtained by performing weighted average calculation on the first rate control parameter and the second rate control parameter.

4. The content adaptive video encoding method according to claim 3, characterized in that: Before the coding features and the set video picture evaluation parameters are input into the first training model and the second training model respectively, the method further includes: Obtain video sample data of different scene types and corresponding resolutions; The video sample data is divided into training set samples, test set samples and validation set samples, and are respectively input into the first training model and the second training model for training.

5. The content adaptive video encoding method according to claim 1, characterized in that: The encoding of the image set based on the encoding parameters comprises: Adjusting a quantization offset parameter according to the first encoding parameter; The image set is encoded according to the second encoding parameter and the adjusted quantization offset parameter to output code stream data.

6. A content-adaptive video encoding device, characterized in that: include: An image set determination module is used to obtain video data to be encoded and divide the video data into a plurality of image sets including continuous frame images; A bit rate parameter determination module, used to determine the coding features of the image set, and input the coding features and the set video picture evaluation parameters into a pre-trained machine learning model to output a bit rate control parameter; A coding module, used to determine frame type information and scene information according to the coding features, perform predictive analysis based on the frame type information, the scene information and the bit rate control parameters to obtain coding parameters, and encode the image set based on the coding parameters, wherein the coding parameters include a first coding parameter obtained when performing predictive analysis, and a second coding parameter determined based on the first coding parameter, coding feedback information, cache information, the frame type information, and the scene information.

7. A content adaptive video encoding device, the device comprising: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the content adaptive video encoding method as described in any one of claims 1-5.

8. A storage medium storing computer executable instructions, wherein the computer executable instructions are used to perform the content adaptive video encoding method according to any one of claims 1 to 5 when executed by a computer processor.

Images