A video data processing method, device, computer equipment and storage medium

By training the neural network model to generate a code rate prediction model, the target code rate of video is determined, which solves the problem of video encoding code rate control in the prior art, and improves video quality and resource utilization efficiency.

CN117729335BActive Publication Date: 2024-11-19SHUXING TECH (BEIJING) CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202310267004.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-14
Publication Date
2024-11-19
Estimated Expiration
2043-03-14

AI Technical Summary

Technical Problem

In the existing video encoding technology, it is difficult to flexibly and accurately determine the target encoding rate of the video, which makes it difficult to balance the video quality after encoding, network bandwidth and storage costs.

Method used

By acquiring the video data set, the neural network model is trained to generate a code rate prediction model, input the model based on the data information of the video to be encoded to determine the target code rate, and use this code rate to perform video encoding.

Benefits of technology

It realizes more accurate and flexible determination of the target encoding rate of the video, improves the encoded video quality, and optimizes network bandwidth and storage costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117729335B_ABST
    Figure CN117729335B_ABST
Patent Text Reader

Abstract

The embodiment of the present application discloses a video data processing method, device, computer equipment and storage medium, the method comprising: obtaining a video data set, the video data set including multiple videos, and obtaining video data information of each video in the video data set; training a preset neural network model according to the video data information of each video to obtain a bit rate prediction model; obtaining a video to be encoded, and obtaining the video data information of the video to be encoded, and inputting the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded; encoding the video to be encoded using the target encoding bit rate to obtain video encoding data corresponding to the video to be encoded. In this way, the target encoding bit rate of the video can be flexibly and accurately determined to improve the quality of the encoded video.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a video data processing method, device, computer equipment and storage medium. Background Art

[0002] In the short video consumption scenario, in order to ensure that users have a better video consumption experience, video encoding usually uses a constant rate factor (CRF) bit rate control strategy with a fixed video quality, that is, a preset CRF value is set before the compression encoder, and the preset CRF value is used to encode the video. Among them, the CRF parameter value range allowed in open source video encoders such as x264 and x265 is 0-51 (also supports floating point numbers). However, the CRF value should be strongly related to the content and picture complexity of each video. For video content with more detailed textures and complex motion, a smaller CRF value should be set before encoding to ensure that the video has a higher video quality after encoding. For video content with simple textures and fewer motion areas, a larger CRF value should be set for compression encoding. This can ensure that the encoded video has a better picture quality while not occupying more network bandwidth and storage costs. Therefore, how to better determine the bit rate of video encoding is a difficult problem in the bit rate control of video encoders. Summary of the invention

[0003] The embodiments of the present application provide a video data processing method, apparatus, computer equipment, and storage medium, which can flexibly and accurately determine the target encoding bit rate of a video to improve the quality of the encoded video.

[0004] In a first aspect, an embodiment of the present application provides a video data processing method, including:

[0005] Acquire a video data set, the video data set including a plurality of videos, and acquire video data information of each video in the video data set;

[0006] Training a preset neural network model according to the video data information of each video to obtain a bit rate prediction model;

[0007] Acquire a video to be encoded, acquire video data information of the video to be encoded, and input the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded;

[0008] The to-be-encoded video is encoded using the target encoding bit rate to obtain video encoding data corresponding to the to-be-encoded video.

[0009] In a second aspect, an embodiment of the present application provides a video data processing device, including:

[0010] An acquisition unit, configured to acquire a video data set, wherein the video data set includes a plurality of videos, and acquire video data information of each video in the video data set;

[0011] A training unit, used to train a preset neural network model according to the video data information of each video to obtain a bit rate prediction model;

[0012] A prediction unit, configured to obtain a video to be encoded, obtain video data information of the video to be encoded, and input the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded;

[0013] The encoding unit is used to encode the video to be encoded using the target encoding bit rate to obtain video encoding data corresponding to the video to be encoded.

[0014] In a third aspect, an embodiment of the present application provides a computer device, comprising: a processor and a memory, wherein the processor is used to execute the method described in the first aspect above.

[0015] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which program instructions are stored, and when the program instructions are executed, the method described in the first aspect above is implemented.

[0016] The embodiment of the present application obtains a video data set and video data information of each video in the video data set; trains a preset neural network model according to the video data information of each video to obtain a bit rate prediction model; obtains a video to be encoded and video data information of the video to be encoded, and inputs the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded; and encodes the video to be encoded using the target encoding bit rate to obtain video encoding data corresponding to the video to be encoded. By training the bit rate prediction model according to the video data information, the target encoding bit rate of the video to be encoded can be determined more accurately and flexibly through the bit rate prediction model, which helps to improve the quality of the encoded video to be encoded. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a schematic diagram of non-I frame predictive coding;

[0019] Figure 2 It is a flowchart of a video data processing method provided in an embodiment of the present application;

[0020] Figure 3 It is a flowchart of another video data processing method provided in an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of coding mode distribution of a coding unit provided in an embodiment of the present application;

[0022] Figure 5 is a schematic diagram of calculating a video frame evaluation index provided by an embodiment of the present application;

[0023] Figure 6 is a structural schematic diagram of a video data processing device provided in an embodiment of the present application;

[0024] Figure 7 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The present application proposes a video data processing scheme, which obtains a video data set and video data information of each video in the video data set; trains a preset neural network model according to the video data information of each video to obtain a bit rate prediction model; obtains a video to be encoded, obtains the video data information of the video to be encoded, and inputs the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded; and encodes the video to be encoded using the target encoding bit rate to obtain video encoding data corresponding to the video to be encoded. By training the bit rate prediction model according to the video data information, the target encoding bit rate of the video to be encoded can be determined more accurately and flexibly through the bit rate prediction model, which helps to improve the quality of the encoded video to be encoded.

[0027] The encoder on the short video transcoding server needs to adaptively decide the most appropriate bitrate for different videos based on the video data information of the original video before encoding and compression. Considering that the compression characteristics of I (Intra) frames encoded within the frame and non-I frames encoded between frames are very different during video encoding, the bitrate of non-I frames needs to be processed differently, such as Figure 1 As shown, Figure 1 It is a schematic diagram of non-I frame prediction coding, such as Figure 1 As shown, the non-I frame 11 is decoded to obtain a decoded frame 12, a predicted frame 13 is predicted based on the decoded frame 12, and the predicted frame 13 is encoded to obtain a current frame 14. Therefore, the present application proposes to perform machine learning network training based on the video data information of the original video based on the inter-frame coded frames (P frames and B frames), and then intelligently decide the bit rates of the inter-frame coded frames of different videos based on the bit rate model obtained by the training.

[0028] The video data processing method provided in the present application can be applied to any scenario where video encoding is required, such as video transmission, video storage, etc.

[0029] The video data processing method provided in the embodiments of the present application can be applied to a video data processing device, which can be set in a computer device. In some embodiments, the computer device may include but is not limited to smart phones, tablet computers, laptop computers, desktop computers, in-vehicle smart terminals, smart watches and other smart terminal devices or servers.

[0030] The video data processing method provided in the embodiment of the present application is schematically described below with reference to the accompanying drawings.

[0031] For details, please see Figure 2 , Figure 2 It is a flow chart of a video data processing method provided in an embodiment of the present application. The video data processing method in the embodiment of the present application can be executed by a video data processing device, wherein the video data processing device can be set in a computer device.

[0032] S201: Obtain a video data set, where the video data set includes multiple videos, and obtain video data information of each video in the video data set.

[0033] In an embodiment of the present application, a computer device may obtain a video data set, where the video data set includes multiple videos, and obtain video data information of each video in the video data set.

[0034] In one embodiment, when a computer device obtains video data information of each video in the video data set, it can use a target encoder to encode each video in the video data set to obtain encoded data of each video in the video data set; determine video feature information and target encoding data of each video in the video data set based on the encoded data of each video in the video data set, and determine that the video feature information and target encoding data of each video in the video data set are the video data information of each video in the video data set.

[0035] Furthermore, the target encoder may be an encoder supporting CRF rate control (such as open source x264); when the computer device uses the target encoder to encode each video in the video data set to obtain the encoded data of each video in the video data set, the computer device may use the encoder supporting CRF rate control to encode each video in the video data set to obtain the encoded data of each video in the video data set. The computer device may encode each video in the video data set offline using the encoder supporting CRF rate control.

[0036] The embodiment of the present application obtains video feature information and target coding data from video data information, which helps to train a more reliable bit rate prediction model, so as to determine the target coding bit rate suitable for different video contents through the bit rate prediction model, thereby improving the accuracy and flexibility of video coding.

[0037] S202: Training a preset neural network model according to the video data information of each video to obtain a bit rate prediction model.

[0038] In the embodiment of the present application, the computer device can train the preset neural network model according to the video data information of each video to obtain a bit rate prediction model. The preset neural network model may include but is not limited to a convolutional neural network model, which includes a feature extractor, a fully connected layer and an output layer, wherein the fully connected layer is connected to the feature extractor and the output layer, the feature extractor extracts the features of each video through the convolutional neural network model, the fully connected layer uses the statistical video features to predict the CRF value, and finally the output layer outputs the predicted CRF value.

[0039] In one embodiment, when the computer device trains the preset neural network model according to the video data information of each video to obtain the bit rate prediction model, the video feature information and target coding data of each video can be input into the preset neural network model to obtain the predicted bit rate, and the loss function value can be determined according to the predicted bit rate; when the loss function value does not meet the preset conditions, the model parameters of the neural network model are adjusted according to the loss function value; the video feature information and target coding data of each video are input into the neural network model after the model parameters are adjusted to retrain, and when the loss function value obtained by retraining meets the preset conditions, it is determined that the bit rate prediction model is obtained. Wherein, the loss function value meeting the preset conditions is used to indicate that the loss function is less than the preset threshold.

[0040] In the embodiment of the present application, the video feature information and target coding data of each video are input into a neural network model for training to obtain a bit rate prediction model, which can accurately predict the target bit rate of video coding.

[0041] S203: Obtain a video to be encoded, obtain video data information of the video to be encoded, and input the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded.

[0042] In an embodiment of the present application, a computer device can obtain a video to be encoded, obtain video data information of the video to be encoded, and input the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded.

[0043] In one embodiment, when a computer device obtains the video data information of the video to be encoded, it can use a target encoder to encode the video to be encoded to obtain the encoded data of the video to be encoded; based on the encoded data of the video to be encoded, determine the video feature information and target encoding data of the video to be encoded, and determine that the video feature information and target encoding data of the video to be encoded are the video data information of the video to be encoded.

[0044] Furthermore, the target encoder may be an encoder supporting CRF rate control (such as open source x264); when the computer device uses the target encoder to encode the video to be encoded to obtain the encoded data of the video to be encoded, the computer device may use the encoder supporting CRF rate control to encode the video to be encoded to obtain the encoded data of the video to be encoded. The computer device may encode the video to be encoded offline using the encoder supporting CRF rate control.

[0045] In one embodiment, when a computer device determines the video feature information and target coding data of a video to be encoded based on the coding data of the video to be encoded, it can extract the video feature information of each video in the video data set from the coding data of the video to be encoded; obtain a first evaluation index of the video to be encoded before the coding process and obtain a second evaluation index of the video to be encoded after the coding process from the coding data, and determine the first evaluation index and the second evaluation index as the target coding data of the video to be encoded.

[0046] In an optional embodiment, the video feature information includes texture complexity; when the computer device extracts the video feature information of the video to be encoded from the encoded data of the video to be encoded, the computer device can obtain the original pixels and reconstructed pixels of each frame of the video image to be encoded; calculate the pixel difference between the original pixels and the reconstructed pixels of each frame of the video image to be encoded, and determine the pixel difference as the texture complexity of each video image frame of the video to be encoded, and determine the sum of the texture complexities of all video image frames of the video to be encoded as the texture complexity of each video.

[0047] In an optional implementation, when obtaining the first evaluation index of each video in the video data set before encoding, the computer device may obtain the peak signal-to-noise ratio of each video frame of the video to be encoded before encoding, and determine the average of the peak signal-to-noise ratios of all video frames in each video as the first evaluation index of each video. The peak signal-to-noise ratio may be Peak Signal to Noise Ratio (PSNR).

[0048] In an optional implementation, when the computer device obtains the second evaluation index of the video to be encoded after the encoding process from the encoded data, it can obtain the quality score of each video frame of the video to be encoded after the encoding process, and determine the average value of the quality scores of all video frames in each video as the second evaluation index of each video. The quality score can be a value obtained by the Visual Multimethod Assessment Fusion (VMAF) video quality multi-method evaluation fusion calculation.

[0049] S204: Encode the video to be encoded using the target encoding bit rate to obtain video encoding data corresponding to the video to be encoded.

[0050] In the embodiment of the present application, the computer device can encode the video to be encoded using the target encoding bit rate to obtain video encoding data corresponding to the video to be encoded. By accurately determining the target encoding bit rate of the video to be encoded and encoding the video to be encoded using the target encoding bit rate, it is helpful to improve the video quality of the video to be encoded after encoding.

[0051] The embodiment of the present application obtains a video data set and video data information of each video in the video data set; trains a preset neural network model according to the video data information of each video to obtain a bit rate prediction model; obtains a video to be encoded and video data information of the video to be encoded, and inputs the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded; and encodes the video to be encoded using the target encoding bit rate to obtain video encoding data corresponding to the video to be encoded. By training the bit rate prediction model according to the video data information, the target encoding bit rate of the video to be encoded can be determined more accurately and flexibly through the bit rate prediction model, which helps to improve the quality of the encoded video to be encoded.

[0052] For details, please see Figure 3 , Figure 3 1 is a flow chart of another video data processing method provided by an embodiment of the present application. The video data processing method of the embodiment of the present application can be executed by a video data processing device, wherein the video data processing device is arranged in a computer device, wherein the specific explanation of the computer device is as described above. The embodiment of the present application is an explanation of how to obtain video feature information and target coding data. Specifically, the method of the embodiment of the present application includes the following steps.

[0053] S301: Obtain video feature information of each video in a video dataset.

[0054] In the embodiment of the present application, the computer device can determine the video feature information and target encoding data of each video in the video data set according to the encoding data of each video in the video data set.

[0055] In one embodiment, when a computer device determines the video feature information and target coding data of each video in the video data set based on the coding data of each video in the video data set, it can extract the video feature information of each video in the video data set from the coding data of each video in the video data set, and the video feature information includes one or more of the proportion of coding units of multiple non-I frames, texture complexity, and motion vectors.

[0056] In an optional embodiment, the video feature information includes the proportion of coding units of multiple non-I frames, and the data blocks of each non-I frame include I blocks, P blocks and B blocks, and the non-I frames are inter-frame coded frames of P frames and B frames; when the computer device extracts the video feature information of each video in the video data set from the encoded data of each video in the video data set, it can extract the I block proportion, P block proportion and B block proportion of the inter-frame coded frames of each P frame and B frame of each video in the video data set from the encoded data of each video in the video data set.

[0057] Specific can Figure 4 Take this as an example to illustrate: Figure 4 is a schematic diagram of the distribution of coding modes of a coding unit provided in an embodiment of the present application, such as Figure 4 As shown, the proportion of coding modes of 16 coding units (i.e., macroblocks or (coding tree units, CTU) tree coding units, CTU can contain one coding unit or be cut into multiple smaller coding units), including 13 B blocks and 3 intra blocks. Therefore, the proportion of intra blocks is 3 / 16, the proportion of B blocks is 13 / 16, and the proportion of P blocks is 0.

[0058] In an optional embodiment, the video feature information includes texture complexity; when the computer device extracts the video feature information of each video in the video data set from the encoded data of each video in the video data set, the computer device can obtain the original pixels and reconstructed pixels of each frame of video image of each video in the video data set; calculate the pixel difference between the original pixels and the reconstructed pixels of each frame of video image of each video in the video data set, and determine the pixel difference as the texture complexity of each video image frame of each video in the video data set, and determine the sum of the texture complexities of all video image frames of each video in the video data set as the texture complexity of each video.

[0059] Furthermore, the calculation formula of the texture complexity of each frame of video image is shown in the following formula (1):

[0060]

[0061] Wherein, n represents the number of 8x8 blocks into which the current video frame is divided, and the sum of absolute transformed differences (SATD) is the pixel difference between the original pixel and the reconstructed pixel (i.e., original pixel - reconstructed pixel).

[0062] In an optional embodiment, for an Intra mode block in a P frame or a B frame, its SATD_Cost is the pixel difference between the original pixel and the reconstructed pixel in the best mode determined by the luminance component (the two chrominance components do not need to be calculated) in all intra-frame prediction angle modes (9 for H.264 and 35 for H.265), that is, the SATD value.

[0063] In an optional implementation, for an Inter mode block in a P frame or a B frame, its SATD_Cost is the pixel difference between the original pixel and the reconstructed pixel under the best motion vector obtained after the luminance component performs motion estimation integer pixel search and sub-pixel search, i.e., the SATD value.

[0064] In one embodiment, the video feature information includes a motion vector; when the computer device extracts the video feature information of each video in the video data set from the encoded data of each video in the video data set, the computer device can calculate the average, maximum and minimum values ​​of the motion vectors (Motion Vector, MV) of all motion vector blocks after encoding each frame of the video image.

[0065] S302: Obtain target coding data for each video in the video data set.

[0066] In an embodiment of the present application, a computer device can obtain a first evaluation index of each video in the video data set before encoding processing and obtain a second evaluation index of each video in the video data set after encoding processing from the encoded data, and determine the first evaluation index and the second evaluation index as target encoding data for each video in the video data set.

[0067] In an optional implementation, when obtaining the first evaluation index of each video in the video data set before encoding, the computer device may obtain the peak signal-to-noise ratio of each video frame of each video in the video data set before encoding, and determine the average of the peak signal-to-noise ratios of all video frames in each video as the first evaluation index of each video. The peak signal-to-noise ratio may be Peak Signal to Noise Ratio (PSNR).

[0068] In an optional implementation, when the computer device obtains the second evaluation index of each video in the video data set after the encoding process from the encoded data, it can obtain the quality score of each video frame of each video in the video data set after the encoding process, and determine the average value of the quality scores of all video frames in each video as the second evaluation index of each video. The quality score can be a value obtained by the fusion calculation of the Visual Multimethod Assessment Fusion (VMAF) video quality multi-method evaluation.

[0069] Specific can Figure 5 Take this as an example to illustrate: Figure 5 is a schematic diagram of calculating a video frame evaluation index provided by an embodiment of the present application, such as Figure 5 As shown, an optional implementation of calculating the PSNR / VMAF of a video frame is to obtain the original YUV of a video frame, and calculate the PSNR / VMAF of the video frame through the original YUV of the video frame. Another optional implementation of calculating the PSNR / VMAF of a video frame is to encode a video frame to obtain a compressed bitstream, decode the compressed bitstream to obtain a distorted YUV, and further calculate the PSNR / VMAF of the video frame through the distorted YUV.

[0070] Furthermore, the computer device may calculate the PSNR or VMAF of each video. The PSNR or VMAF of each video is the cumulative average of the PSNR or VMAF of each frame of the video. The calculation method is shown in the following formula (2) and formula (3):

[0071]

[0072]

[0073] Here, n is used to indicate the number of video frames.

[0074] S303: Training a preset neural network model according to the video data information of each video to obtain a bit rate prediction model.

[0075] S304: Obtain a video to be encoded, obtain video data information of the video to be encoded, and input the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded.

[0076] S305: Encode the video to be encoded using the target encoding bit rate to obtain video encoding data corresponding to the video to be encoded.

[0077] The embodiment of the present application obtains a video data set and video data information of each video in the video data set; trains a preset neural network model according to the video data information of each video to obtain a bit rate prediction model; obtains a video to be encoded and video data information of the video to be encoded, and inputs the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded; and encodes the video to be encoded using the target encoding bit rate to obtain video encoding data corresponding to the video to be encoded. By training the bit rate prediction model according to the video data information, the target encoding bit rate of the video to be encoded can be determined more accurately and flexibly through the bit rate prediction model, which helps to improve the quality of the encoded video to be encoded.

[0078] See also Figure 6 , Figure 6 60 is a schematic diagram of the structure of a video data processing device provided in an embodiment of the present application. Specifically, the device is set in a computer device, and the device includes: an acquisition unit 601, a training unit 602, a prediction unit 603 and an encoding unit 604;

[0079] An acquisition unit 601 is configured to acquire a video data set, wherein the video data set includes a plurality of videos, and acquire video data information of each video in the video data set;

[0080] A training unit 602, configured to train a preset neural network model according to the video data information of each video to obtain a bit rate prediction model;

[0081] The prediction unit 603 is used to obtain a video to be encoded, obtain video data information of the video to be encoded, and input the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded;

[0082] The encoding unit 604 is configured to encode the video to be encoded using the target encoding bit rate to obtain video encoding data corresponding to the video to be encoded.

[0083] Furthermore, when the acquisition unit 601 acquires the video data information of each video in the video data set, it is specifically used to:

[0084] Using a target encoder to encode each video in the video data set to obtain encoded data of each video in the video data set;

[0085] According to the encoding data of each video in the video data set, the video feature information and target encoding data of each video in the video data set are determined, and the video feature information and target encoding data of each video in the video data set are determined to be the video data information of each video in the video data set.

[0086] Further, when the acquisition unit 601 determines the video feature information and target encoding data of each video in the video data set according to the encoding data of each video in the video data set, it is specifically used to:

[0087] Extracting video feature information of each video in the video data set from the encoded data of each video in the video data set, wherein the video feature information includes one or more of a proportion of coding units of multiple non-I frames, texture complexity, and motion vectors;

[0088] Obtain a first evaluation index of each video in the video data set before encoding processing and obtain a second evaluation index of each video in the video data set after encoding processing from the encoded data, and determine the first evaluation index and the second evaluation index as target encoding data for each video in the video data set.

[0089] Further, the video feature information includes a proportion of coding units of multiple non-I frames, each data block of the non-I frame includes an I block, a P block and a B block, and the non-I frame is an inter-frame coding frame of a P frame and a B frame; when the acquisition unit 601 extracts the video feature information of each video in the video data set from the coded data of each video in the video data set, it is specifically used to:

[0090] The I block ratio, P block ratio and B block ratio of the inter-frame coded frame of each P frame and B frame of each video in the video data set are extracted from the coded data of each video in the video data set.

[0091] Further, the video feature information includes texture complexity; when the acquisition unit 601 extracts the video feature information of each video in the video data set from the encoded data of each video in the video data set, it is specifically used to:

[0092] Obtaining original pixels and reconstructed pixels of each frame of video image of each video in the video data set;

[0093] Calculate the pixel difference between the original pixel and the reconstructed pixel of each frame of the video image of each video in the video data set, and determine the pixel difference as the texture complexity of each video image frame of each video in the video data set, and determine the sum of the texture complexities of all video image frames of each video in the video data set as the texture complexity of each video.

[0094] Further, when the acquisition unit 601 acquires the first evaluation index of each video in the video data set before the encoding process and acquires the second evaluation index of each video in the video data set after the encoding process from the encoded data, it is specifically used to:

[0095] Obtaining a peak signal-to-noise ratio of each video frame of each video in the video data set before encoding processing, and determining an average value of the peak signal-to-noise ratios of all video frames in each video as a first evaluation indicator of each video; and

[0096] The quality score of each video frame of each video in the video data set after encoding is obtained, and an average value of the quality scores of all video frames in each video is determined as a second evaluation index for each video.

[0097] Furthermore, the training unit 602 trains a preset neural network model according to the video data information of each video to obtain a bit rate prediction model, which is specifically used to:

[0098] Inputting the video feature information and target coding data of each video into a preset neural network model to obtain a predicted bit rate, and determining a loss function value according to the predicted bit rate;

[0099] When the loss function value does not meet the preset conditions, adjusting the model parameters of the neural network model according to the loss function value;

[0100] The video feature information and target coding data of each video are input into the neural network model after adjusting the model parameters for retraining. When the loss function value obtained by the retraining meets the preset conditions, it is determined that the bit rate prediction model is obtained.

[0101] The embodiment of the present application obtains a video data set and video data information of each video in the video data set; trains a preset neural network model according to the video data information of each video to obtain a bit rate prediction model; obtains a video to be encoded and video data information of the video to be encoded, and inputs the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded; and encodes the video to be encoded using the target encoding bit rate to obtain video encoding data corresponding to the video to be encoded. By training the bit rate prediction model according to the video data information, the target encoding bit rate of the video to be encoded can be determined more accurately and flexibly through the bit rate prediction model, which helps to improve the quality of the encoded video to be encoded.

[0102] See also Figure 7 , Figure 7701 is a schematic diagram of a computer device provided in an embodiment of the present application. Specifically, the computer device includes: a memory 701 and a processor 702.

[0103] In one embodiment, the computer device further includes a data interface 703, and the data interface 703 is used to transfer data information between the computer device and other devices.

[0104] The memory 701 may include a volatile memory; the memory 701 may also include a non-volatile memory; the memory 701 may also include a combination of the above-mentioned types of memories. The processor 702 may be a central processing unit (CPU). The processor 702 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA) or any combination thereof.

[0105] The memory 701 is used to store programs, and the processor 702 can call the program stored in the memory 701 to perform the following steps:

[0106] Acquire a video data set, the video data set including a plurality of videos, and acquire video data information of each video in the video data set;

[0107] Training a preset neural network model according to the video data information of each video to obtain a bit rate prediction model;

[0108] Acquire a video to be encoded, acquire video data information of the video to be encoded, and input the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded;

[0109] The to-be-encoded video is encoded using the target encoding bit rate to obtain video encoding data corresponding to the to-be-encoded video.

[0110] Further, when the processor 702 obtains the video data information of each video in the video data set, it is specifically used to:

[0111] Using a target encoder to encode each video in the video data set to obtain encoded data of each video in the video data set;

[0112] According to the encoding data of each video in the video data set, the video feature information and target encoding data of each video in the video data set are determined, and the video feature information and target encoding data of each video in the video data set are determined to be the video data information of each video in the video data set.

[0113] Further, when the processor 702 determines the video feature information and the target coding data of each video in the video data set according to the coding data of each video in the video data set, it is specifically used to:

[0114] Extracting video feature information of each video in the video data set from the encoded data of each video in the video data set, wherein the video feature information includes one or more of a proportion of coding units of multiple non-I frames, texture complexity, and motion vectors;

[0115] Obtain a first evaluation index of each video in the video data set before encoding processing and obtain a second evaluation index of each video in the video data set after encoding processing from the encoded data, and determine the first evaluation index and the second evaluation index as target encoding data for each video in the video data set.

[0116] Further, the video feature information includes a proportion of coding units of multiple non-I frames, each data block of the non-I frame includes an I block, a P block and a B block, and the non-I frame is an inter-frame coded frame of a P frame and a B frame; when the processor 702 extracts the video feature information of each video in the video data set from the coded data of each video in the video data set, it is specifically used to:

[0117] The I block ratio, P block ratio and B block ratio of the inter-frame coded frame of each P frame and B frame of each video in the video data set are extracted from the coded data of each video in the video data set.

[0118] Further, the video feature information includes texture complexity; when the processor 702 extracts the video feature information of each video in the video data set from the encoded data of each video in the video data set, it is specifically used to:

[0119] Obtaining original pixels and reconstructed pixels of each frame of video image of each video in the video data set;

[0120] Calculate the pixel difference between the original pixel and the reconstructed pixel of each frame of the video image of each video in the video data set, and determine the pixel difference as the texture complexity of each video image frame of each video in the video data set, and determine the sum of the texture complexities of all video image frames of each video in the video data set as the texture complexity of each video.

[0121] Further, when the processor 702 obtains the first evaluation index of each video in the video data set before the encoding process and obtains the second evaluation index of each video in the video data set after the encoding process from the encoded data, it is specifically used to:

[0122] Obtaining a peak signal-to-noise ratio of each video frame of each video in the video data set before encoding processing, and determining an average value of the peak signal-to-noise ratios of all video frames in each video as a first evaluation indicator of each video; and

[0123] The quality score of each video frame of each video in the video data set after encoding is obtained, and an average value of the quality scores of all video frames in each video is determined as a second evaluation index for each video.

[0124] Furthermore, the processor 702 trains a preset neural network model according to the video data information of each video to obtain a bit rate prediction model, which is specifically used to:

[0125] Inputting the video feature information and target coding data of each video into a preset neural network model to obtain a predicted bit rate, and determining a loss function value according to the predicted bit rate;

[0126] When the loss function value does not meet the preset conditions, adjusting the model parameters of the neural network model according to the loss function value;

[0127] The video feature information and target coding data of each video are input into the neural network model after adjusting the model parameters for retraining. When the loss function value obtained by the retraining meets the preset conditions, it is determined that the bit rate prediction model is obtained.

[0128] The embodiment of the present application obtains a video data set and video data information of each video in the video data set; trains a preset neural network model according to the video data information of each video to obtain a bit rate prediction model; obtains a video to be encoded and video data information of the video to be encoded, and inputs the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded; and encodes the video to be encoded using the target encoding bit rate to obtain video encoding data corresponding to the video to be encoded. By training the bit rate prediction model according to the video data information, the target encoding bit rate of the video to be encoded can be determined more accurately and flexibly through the bit rate prediction model, which helps to improve the quality of the encoded video to be encoded.

[0129] The embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the present application is implemented. Figure 1 or Figure 3 The method described in the corresponding embodiment can also be implemented Figure 6 The device corresponding to the embodiment of the present application will not be described in detail here.

[0130] The computer-readable storage medium may be an internal storage unit of the device described in any of the foregoing embodiments, such as a hard disk or memory of the device. The computer-readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0131] The embodiments of the present application also provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above various embodiments.

[0132] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0133] The above disclosure is only part of the embodiments of the present application, which certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and making equivalent changes according to the claims of this application are still within the scope of the present invention.

Claims

1. A video data processing method, characterized in that: include: Acquire a video data set, wherein the video data set includes a plurality of videos, and each video in the video data set includes at least one inter-frame coding frame; Obtaining video data information of each video in the video data set, the video data information of any video including video feature information of each inter-coded frame in the any video and target coding data of the any video, the video feature information of any inter-coded frame including an I-block ratio, a P-block ratio, and a B-block ratio of the any inter-coded frame; Training a preset neural network model according to the I block ratio, P block ratio and B block ratio of each inter-frame coding frame of each video and the target coding data of each video to obtain a bit rate prediction model; Obtaining a video to be encoded, and obtaining video data information of the video to be encoded, and inputting the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded, wherein the video to be encoded includes at least one inter-frame coded frame to be encoded, and the video data information of the video to be encoded includes video feature information of each inter-frame coded frame to be encoded in the video to be encoded and target encoding data of the video to be encoded, and the video feature information of any inter-frame coded frame to be encoded includes an I-block ratio, a P-block ratio, and a B-block ratio of any inter-frame coded frame to be encoded; The to-be-encoded video is encoded using the target encoding bit rate to obtain video encoding data corresponding to the to-be-encoded video.

2. The method according to claim 1, characterized in that The obtaining of video data information of each video in the video data set includes: Using a target encoder to encode each video in the video data set to obtain encoded data of each video in the video data set; According to the encoding data of each video in the video data set, the video feature information and target encoding data of each video in the video data set are determined, and the video feature information and target encoding data of each video in the video data set are determined to be the video data information of each video in the video data set.

3. The method according to claim 2, characterized in that The step of determining the video feature information and target coding data of each video in the video data set according to the coding data of each video in the video data set includes: Extracting video feature information of each video in the video data set from the encoded data of each video in the video data set, wherein the video feature information further includes one or more of texture complexity and motion vector; Obtain a first evaluation index of each video in the video data set before encoding processing and obtain a second evaluation index of each video in the video data set after encoding processing from the encoded data, and determine the first evaluation index and the second evaluation index as target encoding data for each video in the video data set.

4. The method according to claim 3, characterized in that The extracting the video feature information of each video in the video data set from the encoded data of each video in the video data set includes: The I block ratio, P block ratio and B block ratio of each P frame or B frame of each video in the video data set are extracted from the encoded data of each video in the video data set.

5. The method according to claim 3, characterized in that: The video feature information includes texture complexity; extracting the video feature information of each video in the video data set from the encoded data of each video in the video data set includes: Obtaining original pixels and reconstructed pixels of each frame of video image of each video in the video data set; Calculate the pixel difference between the original pixel and the reconstructed pixel of each frame of the video image of each video in the video data set, and determine the pixel difference as the texture complexity of each video image frame of each video in the video data set, and determine the sum of the texture complexities of all video image frames of each video in the video data set as the texture complexity of each video.

6. The method according to claim 3, characterized in that The obtaining of a first evaluation index of each video in the video data set before encoding and obtaining a second evaluation index of each video in the video data set after encoding from the encoded data include: Obtaining a peak signal-to-noise ratio of each video frame of each video in the video data set before encoding processing, and determining an average value of the peak signal-to-noise ratios of all video frames in each video as a first evaluation indicator of each video; and The quality score of each video frame of each video in the video data set after encoding is obtained, and an average value of the quality scores of all video frames in each video is determined as a second evaluation index for each video.

7. The method according to claim 3, characterized in that The step of training a preset neural network model according to the video data information of each video to obtain a bit rate prediction model includes: Inputting the video feature information and target coding data of each video into a preset neural network model to obtain a predicted bit rate, and determining a loss function value according to the predicted bit rate; When the loss function value does not meet the preset conditions, adjusting the model parameters of the neural network model according to the loss function value; The video feature information and target coding data of each video are input into the neural network model after adjusting the model parameters for retraining. When the loss function value obtained by the retraining meets the preset conditions, it is determined that the bit rate prediction model is obtained.

8. A video data processing device, characterized in that: include: an acquisition unit, configured to acquire a video data set, wherein the video data set includes a plurality of videos, and acquire video data information of each video in the video data set, wherein each video in the video data set includes at least one inter-coded frame, and the video data information of any video includes video feature information of each inter-coded frame in the any video and target coding data of the any video, and the video feature information of any inter-coded frame includes an I-block ratio, a P-block ratio, and a B-block ratio of the any inter-coded frame; A training unit, configured to train a preset neural network model according to the I block ratio, P block ratio and B block ratio of each inter-frame coding frame of each video and the target coding data of each video to obtain a bit rate prediction model; A prediction unit is used to obtain a video to be encoded, obtain video data information of the video to be encoded, and input the video data information of the video to be encoded into the bit rate prediction model to obtain a target encoding bit rate corresponding to the video to be encoded, wherein the video to be encoded includes at least one inter-frame coded frame to be encoded, and the video data information of the video to be encoded includes video feature information of each inter-frame coded frame to be encoded in the video to be encoded and target encoding data of the video to be encoded, and the video feature information of any inter-frame coded frame to be encoded includes an I block ratio, a P block ratio, and a B block ratio of any inter-frame coded frame to be encoded; The encoding unit is used to encode the video to be encoded using the target encoding bit rate to obtain video encoding data corresponding to the video to be encoded.

9. A computer device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are connected to each other, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, which, when executed, implement the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product comprises a computer program, which is suitable for being loaded by a processor and executing the method according to any one of claims 1-7.

Citation Information

Patent Citations

  • Video frame encoding method, video frame encoding equipment and device with storage function

    CN110198444A

  • Video coding rate determination method and device, electronic equipment and storage medium

    CN111083483A

  • Content adaptive video coding method and device, equipment and storage medium

    CN114554211A

  • Video coding method and device

    CN114885167A

  • Video coding parameter acquisition method and device, network model and electronic equipment

    CN115209150A