Video encoding method and device

Abstract

The present application provides a video encoding method and apparatus, wherein the video encoding method comprises: obtaining a video segment to be encoded; decoding the video segment to be encoded to obtain a set of video frames to be encoded; extracting spatiotemporal feature information and encoding feature information of the video segment to be encoded based on the set of video frames to be encoded; determining a rate factor of the video segment to be encoded based on the spatiotemporal feature information and the encoding feature information; encoding the video segment to be encoded based on a preset video encoding score and the rate factor to obtain a target video segment. By using the present method, spatiotemporal feature information and encoding feature information are extracted based on the set of video frames to be encoded, and the rate factor for video encoding is then predicted. By using the method of predicted rate factor and constant video encoding score, the accuracy of video encoding is effectively improved, that is, the video quality is guaranteed and the size of the video compressed file is significantly reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more particularly to a video encoding method, a video encoding apparatus, a computing device, and a computer-readable storage medium. Background Art

[0002] With the development of technology, video platforms have also developed rapidly. More and more users are sharing and watching videos. For the platform, as the number of users continues to grow, the bandwidth cost for video transmission is also increasing. In order to reduce this cost, a transcoding system with better compression is needed. While ensuring the video quality, it can significantly reduce the size of the compressed video file, thereby reducing the network traffic required for transmission. Summary of the Invention

[0003] In view of this, embodiments of the present application provide a video encoding method. This application also relates to a video encoding apparatus, a computing device, and a computer-readable storage medium to address the prior art issues of unstable encoding quality and low encoding accuracy during video encoding in a constant bit rate mode.

[0004] According to a first aspect of an embodiment of the present application, a video encoding method is provided, including:

[0005] Get the video clip to be encoded;

[0006] Decoding the video segment to be encoded to obtain a set of video frames to be encoded;

[0007] Extracting spatiotemporal feature information and coding feature information of the video segment to be encoded based on the set of video frames to be encoded;

[0008] Determining a bit rate factor of the to-be-encoded video segment according to the spatiotemporal feature information and the encoding feature information;

[0009] The to-be-encoded video segment is encoded according to the preset video encoding score and the bit rate factor to obtain a target video segment.

[0010] According to a second aspect of the embodiments of the present application, a video encoding apparatus is provided, including:

[0011] An acquisition module, configured to acquire a video segment to be encoded;

[0012] A decoding module is configured to decode the video segment to be encoded to obtain a set of video frames to be encoded;

[0013] an extraction module configured to extract spatiotemporal feature information and coding feature information of the video segment to be encoded based on the set of video frames to be encoded;

[0014] a determination module, configured to determine a bit rate factor of the to-be-encoded video segment according to the spatiotemporal feature information and the encoding feature information;

[0015] The encoding module is configured to encode the to-be-encoded video segment according to a preset video encoding score and the bit rate factor to obtain a target video segment.

[0016] According to a third aspect of an embodiment of the present application, a computing device is provided, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor implements the steps of the video encoding method when executing the computer instructions.

[0017] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, which stores computer instructions, and when the computer instructions are executed by a processor, the steps of the video encoding method are implemented.

[0018] The video encoding method provided in the present application obtains a video segment to be encoded; decodes the video segment to be encoded to obtain a set of video frames to be encoded; extracts spatiotemporal feature information and encoding feature information of the video segment to be encoded based on the set of video frames to be encoded; determines a rate factor of the video segment to be encoded based on the spatiotemporal feature information and the encoding feature information; and encodes the video segment to be encoded based on a preset video encoding score and the rate factor to obtain a target video segment.

[0019] An embodiment of the present application realizes the extraction of spatiotemporal feature information and coding feature information based on a set of video frames to be encoded, and predicts the bit rate factor for video encoding based on the spatiotemporal feature information and coding feature information, and encodes the video based on a preset video encoding score and bit rate factor, thereby obtaining a target video clip. Through the method of predicted bit rate factor and constant video encoding score, the accuracy of video encoding is effectively improved, that is, the video quality is guaranteed, and the size of the video compression file is significantly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flowchart of a video encoding method provided by an embodiment of the present application;

[0021] Figure 2 This is a schematic diagram of the structure of a rate factor prediction model provided by an embodiment of the present application;

[0022] Figure 3 is a processing flow chart of a video encoding method applied to video T encoding provided by an embodiment of the present application;

[0023] Figure 4 This is a structural diagram of a video encoding device provided by an embodiment of the present application;

[0024] Figure 5 This is a structural block diagram of a computing device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0025] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

[0026] The terms used in one or more embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to any or all possible combinations of one or more associated listed items.

[0027] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0028] First, the terms involved in one or more embodiments of the present application are explained.

[0029] CRF: Constant Rate Factor mode is a rate control method for video encoding.

[0030] RF: Rate Factor, an encoding parameter of the CRF encoding mode, which can control the bit rate and quality of the encoded file.

[0031] VMAF: Video Multimethod Assessment Fusion, a full-reference evaluation method for measuring the quality of encoded videos.

[0032] GLCM: Gray Level Co-occurrence Matrix gray level co-occurrence information is a feature that can describe the complexity of video spatial texture.

[0033] NCC: Normalized Correlation Coefficient normalized information is a feature that can describe the temporal complexity of the video.

[0034] H.264: A video coding standard.

[0035] With the development of technology, video platforms have also developed rapidly. More and more users are sharing and watching videos. For video platforms, as the number of users continues to grow, the bandwidth cost for video transmission is also increasing. In order to reduce this cost, a transcoding system with better compression is needed. While ensuring video quality, it can significantly reduce the size of video compressed files, thereby reducing the network traffic required for transmission.

[0036] Currently common encoding modes include fixed quantization parameter mode, constant bit rate mode, and variable bit rate mode. The most commonly used one is the constant bit rate mode. In this mode, the output bit stream has a stable bit rate, which can improve the problem of playback stuttering. However, this mode determines a constant bit rate and uses the same bit rate for encoding all videos. For videos with rich scene content, the picture quality experience of all clips cannot be guaranteed, and bit rate waste cannot be avoided. For example, for complex scenes, using the overall average bit rate for encoding will result in a significant decrease in picture quality due to insufficient bit rate. For simple scenes, the average bit rate is too high, resulting in unnecessary bit rate waste.

[0037] In order to ensure the picture quality experience and avoid bit rate waste at the same time, a constant quality encoding mode can also be used. Specifically, a target quality can be set, and then the encoder can be used to encode the video to obtain a stream file that meets the quality. The goal of this encoding mode is to make the quality of the encoded video as close as possible to the set target quality. The higher the accuracy, the better the encoding effect. Otherwise, when the actual quality is lower than the target quality, the picture quality experience will be damaged; when the quality exceeds the target quality, it will cause a waste of bit rate.

[0038] However, in this method, before encoding, the relationship between the encoding rate factor and the quality of the final encoding result is unknown, and the accurate rate factor cannot be directly obtained by setting the target quality. For this, one strategy is to perform multiple encodings and use a posteriori method to search for the optimal rate factor, but this method requires huge computing resources; another strategy is to predict the rate factor. The rate factor can be predicted based on the spatiotemporal characteristics of the video before encoding through machine learning to obtain the desired encoding result, but this method is usually a one-time prediction and encoding method, that is, the rate factor is predicted based on the spatiotemporal characteristics of the video according to the preset target, and then the video is encoded based on the predicted rate factor. This method requires encoding prediction for the entire video, and cannot achieve constant quality in each scene. Its prediction accuracy is low, and for popular videos, it is not enough to stably guarantee the picture quality experience and avoid rate waste.

[0039] Based on this, in this application, a video encoding method is provided. This application also involves a video encoding device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0040] Figure 1 A flowchart of a video encoding method according to an embodiment of the present application is shown, which specifically includes the following steps:

[0041] Step 102: Obtain the video segment to be encoded.

[0042] The video clip to be encoded specifically refers to the video clip encoded based on the constant quality video encoding method in this application. In practical applications, the video clip to be encoded specifically refers to a single-shot clip, that is, a video clip without transition shots. Generally, the content displayed in a single-shot clip is usually similar and coherent. Therefore, when encoding single-shot clips, using the same bit rate factor will make the video encoding more reasonable.

[0043] Specifically, obtaining the video clip to be encoded includes:

[0044] Get the video to be encoded;

[0045] Performing scene detection on the video to be encoded, and splitting the video to be encoded into multiple single-shot video clips;

[0046] Each single-shot video segment is determined to be a video segment to be encoded.

[0047] Among them, the video to be encoded specifically refers to a complete video file uploaded by the user. The video to be encoded can be a video file recorded by the user himself, or a video file captured by the user from other video files, or a video file created by the user based on multiple video files. In this application, the specific content of the video to be encoded is not limited.

[0048] After acquiring the video to be encoded, scene detection is performed on the video to identify single-shot video segments within the video. The video to be encoded is then segmented based on the single-shot video segments. Each segmented single-shot video segment can be used as a video segment to be encoded. In practical applications, each video segment to be encoded needs to be encoded separately. The embodiments provided herein use the encoding process of a single video segment to be encoded as an example for explanation.

[0049] In a specific implementation provided in the present application, taking video V as an example, a video V uploaded by a user is received, and video V is the video to be encoded. The open source encoder x264 is called to encode video V, and the scene detection switch is turned on to obtain the corresponding scene switching detection result. According to the detection result, the video V is divided into t independent single-shot video segments (V1, V2...Vt), and each single-shot video segment is a video segment to be encoded.

[0050] Step 104: Decode the video segment to be encoded to obtain a set of video frames to be encoded.

[0051] After obtaining the video segment to be encoded, the video segment to be encoded may be further processed, that is, the video segment to be encoded may be decoded and converted into a corresponding set of video frames to be encoded.

[0052] In a specific implementation provided in the present application, taking the video segment V1 to be encoded as an example, the video segment V1 to be encoded is decoded, and each video frame to be encoded in the video segment to be encoded is read to form a video frame set to be encoded (f1, f2, ... fn) consisting of n video frames to be encoded.

[0053] Step 106: extracting spatiotemporal feature information and coding feature information of the video segment to be encoded based on the set of video frames to be encoded.

[0054] After obtaining the set of video frames to be encoded, the spatiotemporal domain feature information and coding feature information of the video segment to be encoded can be extracted based on the set of video frames to be encoded, wherein the spatiotemporal domain feature information is used to express the spatiotemporal complexity of the video segment to be processed, and the coding feature information is used to express the feature information of the video segment to be processed.

[0055] The step of extracting the spatiotemporal feature information of the video segment to be encoded based on the set of video frames to be encoded includes:

[0056] Extracting grayscale co-occurrence information and normalization information of the to-be-encoded video frame set based on the to-be-encoded video frame set;

[0057] The spatiotemporal domain feature information of the video segment to be encoded is generated by splicing the grayscale symbiosis information and the normalization information.

[0058] Gray Level Co-occurrence Matrix (GLCM) is a feature used to describe the complexity of video spatial texture.

[0059] In practice, for each frame in the set of frames to be encoded, a two-dimensional m*m GLCM matrix is ​​calculated. Each element in the GLCM matrix ranges from 0 to 1. This matrix is ​​then used to calculate five eigenvalues: entropy, contrast, inverse moment correlation, energy, and autocorrelation. Four statistical values, including mean, variance, skewness, and kurtosis, are then calculated for the corresponding eigenvalues ​​of all frames to be encoded. For example, the mean, variance, skewness, and kurtosis of the entropy of each frame to be encoded are calculated. Ultimately, a total of 20 (5*4) eigenvalues ​​are obtained. To account for the multi-scale nature of video, each frame to be encoded is scaled down by a factor of 4 and 16, respectively, and GLCM feature extraction is performed twice. This results in a total of 60 eigenvalues ​​extracted for the set of frames to be encoded.

[0060] Normalized Correlation Coefficient (NCC) is a feature used to describe time domain complexity.

[0061] In practical applications, NCC is calculated for each pair of adjacent to-be-processed video frames in the to-be-processed video frame set to obtain a two-dimensional correlation coefficient matrix with the same length and width as the original to-be-processed video frame. The value of each element in the matrix is ​​0-1, indicating the time domain correlation between the corresponding positions of two adjacent to-be-processed video frames. Multiple correlation coefficient matrices are obtained. A to-be-processed video frame set with n to-be-processed video frames will have n-1 adjacent video frames, and n-1 correlation coefficient matrices will be obtained. The mean, variance, kurtosis, skewness and information entropy of the n-1 correlation coefficient matrices, totaling 5 eigenvalues, are then calculated. For each eigenvalue, the mean and variance are calculated from the time domain, and finally 10 (5*2) eigenvalues ​​can be obtained, i.e., 10 NCC eigenvalues.

[0062] After obtaining the GLCM eigenvalues ​​and the GLCM eigenvalues, the two are spliced ​​to obtain 70 eigenvalues, which constitute the spatiotemporal domain feature information of the video segment to be encoded. The spatiotemporal domain feature information is represented in the form of a vector.

[0063] Accordingly, in addition to extracting the spatiotemporal feature information of the video segment to be encoded, it is also necessary to extract the coding feature information of the video segment to be encoded. Specifically, extracting the coding feature information of the video segment to be encoded based on the set of video frames to be encoded includes:

[0064] Generating a set of video frame pairs to be encoded according to the set of video frames to be encoded;

[0065] Intra-frame coding and inter-frame coding are performed on each video frame pair in the set of video frame pairs to be coded, to obtain coding feature information of the video segment to be coded.

[0066] In practical applications, after obtaining a set of video frames to be encoded, it is also necessary to perform video frame preprocessing on the set of video frames to be encoded. The other video frames to be encoded in the set of video frames to be encoded, excluding the first frame and the last frame, are copied, and the copied video frames to be encoded are placed after each original video frame to be encoded to form a set of video frame pairs to be encoded. For example, the set of video frames to be encoded is (f1, f2, ... fn). In addition to f1 and fn, each video frame from f2 to fn-1 is copied to obtain a set of video frame pairs to be encoded (f1, f2, f2, f3, f3 ... fn-2, fn-2, fn-1, fn-1, fn), where (f1, f2), (f2, f3) ... (fn-2, fn-1), (fn-1, fn) are video frame pairs to be encoded.

[0067] After obtaining the set of video frame pairs to be encoded, the hardware H.264 encoder built into the graphics card is used to pre-encode the set of video frame pairs to be encoded, using a fixed quantization parameter mode. Preferably, the quantization parameter is 29, and the encoding frame structure adopts an IP structure, wherein I refers to intra-frame coding and P refers to inter-frame coding. Under this encoding method, except for the first frame and the last frame, each video frame to be encoded is encoded twice. Intra-frame coding can measure the spatial coding complexity of the video frame to be encoded, and inter-frame coding can measure the coding complexity of the video frame to be encoded. After IP encoding is performed on each video frame pair to be encoded, the encoding feature information of the video segment to be encoded can be obtained.

[0068] Specifically, performing intra-frame coding and inter-frame coding on each video frame pair in the set of video frame pairs to be coded to obtain coding feature information of the video segment to be coded includes:

[0069] Performing intra-frame coding on the first video frame of each video frame pair in the set of video frame pairs to be coded to obtain intra-frame coding information;

[0070] Performing inter-frame coding on each video frame pair in the set of video frame pairs to be coded to obtain inter-frame coding information;

[0071] The intra-frame coding information and the inter-frame coding information are concatenated to obtain coding feature information.

[0072] Among them, there are two video frames in each video frame pair to be encoded, the earlier video frame is the first video frame, and the later video frame is the second video frame. For example, for the video frame pair to be encoded (f1, f2), f1 is the first video frame and f2 is the second video frame; for the video frame pair to be encoded (f2, f3), f2 is the first video frame and f3 is the second video frame, and so on. The first video frame in each video frame pair to be encoded is intra-frame encoded. Intra-frame encoding is used to measure the spatial coding complexity of the video frame, and the characteristic values ​​of each intra-frame encoding are extracted, such as the number of bits, the proportion of various prediction modes in the frame, etc., totaling 34 characteristic values. Then, the corresponding characteristic values ​​of all intra-frame encodings are calculated, including 5 statistical values ​​such as mean, variance, skewness, kurtosis and information entropy, and finally 170 (34*5) inter-frame coding information are obtained.

[0073] For each video frame pair to be encoded, it is also necessary to extract the inter-frame coding feature information of each video frame pair. For example, for the video frame pair to be encoded (f1, f2), the inter-frame coding feature information of f1 and f2 is calculated; for the video frame pair to be encoded (f2, f3), the inter-frame coding feature information of f2 and f3 is calculated, and so on. Inter-frame coding is performed on each video frame pair to be encoded. Inter-frame coding is used to measure the complexity of video time domain coding. A total of 26 feature values ​​such as the number of coding bits and the amplitude distribution of inter-frame motion vectors of each video frame pair to be encoded are extracted. Then, the corresponding feature values ​​of all inter-frame coding are calculated, including 5 statistical values ​​such as mean, variance, skewness, kurtosis and information entropy, to obtain 130 (26*5) inter-frame coding information.

[0074] The 170 inter-frame coding information and the 130 inter-frame coding information are concatenated to obtain the coding feature information of 300 video segments to be coded. The coding feature information is also represented in vector form.

[0075] Step 108: Determine a bit rate factor of the to-be-encoded video segment according to the spatiotemporal feature information and the encoding feature information.

[0076] After obtaining the spatiotemporal feature information and coding feature information, the rate factor (RF) used for video coding can be predicted based on these two feature information. The rate factor refers to the coding parameter of the constant rate factor coding mode, which is used to control the bit rate and quality of the encoded file. The value of the rate factor is usually in the range of 0-50. The larger the value of the rate factor, the lower the bit rate and the worse the encoding quality. Conversely, the smaller the value of the rate factor, the higher the bit rate and the better the encoding quality.

[0077] In the method provided in this application, the rate factor is obtained by jointly predicting the spatiotemporal feature information and the coding feature information. Specifically, determining the rate factor of the video segment to be encoded based on the spatiotemporal feature information and the coding feature information includes:

[0078] Splicing the spatiotemporal feature information and the coded feature information to generate first feature information to be processed;

[0079] Inputting the first feature information to be processed into a first rate factor prediction model, wherein the first rate factor prediction model is trained to generate a predicted rate factor based on the first feature information to be processed;

[0080] A first rate factor generated by the first rate factor prediction model according to the first feature information to be processed is received.

[0081] Among them, the first feature information to be processed specifically refers to the feature information generated by splicing the spatiotemporal domain feature information and the coding feature information, which is used to predict the bit rate factor. In the above steps, it can be seen that the spatiotemporal domain feature information is a vector with 70 eigenvalues, and the coding feature information is a vector with 300 eigenvalues. After splicing the two vectors, the first feature information to be processed with 370 eigenvalues ​​can be obtained. The first feature information to be processed is used to express the spatiotemporal characteristics of a video clip.

[0082] After obtaining the first feature information to be processed, the first feature information to be processed is input into the first rate factor prediction model for processing. The first rate factor prediction model is a pre-trained deep neural network model. The structure of the neural network model is as follows: Figure 2 As shown, Figure 2 FIG. 1 shows a schematic diagram of the structure of a rate factor prediction model provided by an embodiment of the present application. Figure 2 As shown in the figure, the rate factor prediction model consists of a batch normalization module, an attention mechanism module, a residual module, a fully connected module, an attention mechanism module, and a fully connected module.

[0083] The batch normalization module normalizes the raw input features into a Gaussian distribution with a mean of 0 and a variance of 1, thereby eliminating the impact of feature differences. The normalized features are then passed through the attention mechanism module, which calculates a weight for each feature, assigning greater weights to features with higher recognition and smaller weights to features with lower information content. This automatically filters features and improves prediction accuracy. After filtering, the features are passed through a series of residual modules, attention mechanism modules, and fully connected modules to output the predicted rate factor parameter value.

[0084] The model parameters in the rate factor prediction model need to be learned and trained from data in advance. In practical applications, the rate factor prediction model is obtained through the following training method:

[0085] Obtaining sample characteristic information and a sample rate factor corresponding to the sample characteristic information;

[0086] Inputting the sample feature information into a rate factor prediction model for processing, and receiving a predicted rate factor output by the rate factor prediction model;

[0087] Calculating a model loss value according to the predicted bit rate factor and the sample bit rate factor;

[0088] Adjust the model parameters of the rate factor prediction model according to the model loss value, and continue to train the rate factor prediction model until a model training stop condition is reached.

[0089] The sample feature information and sample rate factor are sample training pairs used to train the rate factor prediction model. The sample feature information is obtained using the method for obtaining the first feature information to be processed, and the sample rate factor is the true rate factor. The true rate factor is obtained by repeatedly searching the sample feature information, repeatedly attempting to encode the training video using different rate factor parameters, and ultimately determining the optimal rate factor.

[0090] The sample feature information is input into the rate factor prediction model for processing to obtain the predicted rate factor output by the rate factor prediction model, and then the model loss value is calculated based on the predicted rate factor and the sample rate factor. The loss function for calculating the model loss value can be a cross entropy loss function, a maximum loss function, an average loss function, a 0-1 loss function, etc. In this application, the specific choice of the loss function is not limited and is subject to actual application.

[0091] After calculating the model loss value, the model loss value is back-propagated, and the model parameters of the rate factor prediction model are adjusted. At the same time, the rate factor prediction model is continuously trained until the model training stopping condition is reached. Specifically, the model training stopping condition may be that the model loss value is lower than a preset threshold value and / or the number of model training rounds reaches a preset number. The specific setting of the model training stopping condition is not limited in this application and is subject to actual application.

[0092] Step 110: Encode the to-be-encoded video segment according to a preset video encoding score and the bit rate factor to obtain a target video segment.

[0093] After obtaining the bitrate factor, the transcoding system's corresponding encoder can be used to encode the target video clip using that bitrate factor, resulting in an encoded result. At this point, the encoding score for the result, known as the VMAF (Video Multimethod Assessment Fusion) value, must be calculated. VMAF is a reference score used to measure the quality of the encoded video. If the VMAF value meets the preset video encoding score, the target video clip is obtained. If the VMAF value does not meet the preset video encoding score, re-encoding is required.

[0094] Specifically, encoding the to-be-encoded video segment according to the preset video encoding score and the bit rate factor to obtain a target video segment includes:

[0095] Performing a first encoding on the to-be-encoded video segment based on the first bit rate factor to obtain an initial video segment, and determining a first encoding score for the initial video segment;

[0096] Determining a video coding score interval according to the preset video coding score;

[0097] When the first coding score is within the video coding score interval, determining the initial video segment as a target video segment;

[0098] When the first coding score is outside the video coding score range, the initial video segment is encoded based on the spatiotemporal feature information, the coding feature information, the first bit rate factor, and the first coding score to obtain a target video segment.

[0099] In practical applications, after obtaining the first bit rate factor, the video clip to be encoded is encoded for the first time according to the first bit rate factor through the encoder corresponding to the transcoding system. After the encoding is completed, an initial video clip is obtained, wherein the initial video clip specifically refers to the video clip obtained after the video clip to be encoded is encoded using the first bit rate factor. At the same time, a first encoding score of the initial video clip is calculated, and the first encoding score specifically refers to the video quality score after the first encoding.

[0100] The preset video encoding score specifically refers to a pre-set constant encoding quality score, for example, 91 points. If the encoding score of the video reaches 91 points, the video encoding quality can be considered qualified. In actual applications, an error within a certain range is allowed. For example, the preset video encoding score can fluctuate up or down by 1 point to form a video encoding score range. Still taking the preset video encoding score of 91 points as an example, the fluctuation range is 1 point, then the preset video encoding score can be determined to be 90-92 points.

[0101] If the first encoding score is within the preset video encoding score range (including the endpoints of the range), it can be directly determined that the initial video segment obtained after the first encoding is a video segment that meets the requirements, that is, the initial video segment is determined to be the target video segment.

[0102] If the first coding score is outside the preset video coding score range (excluding the endpoints of the range), the initial video segment needs to be further encoded. Specifically, encoding the initial video segment based on the spatiotemporal feature information, the coding feature information, the first bit rate factor, and the first coding score includes:

[0103] Determining a second rate factor based on the spatiotemporal feature information, the coding feature information, the first rate factor, and the first coding score;

[0104] The initial video segment is encoded according to the second rate factor.

[0105] In practical applications, if the first coding score obtained by the first video coding calculation is outside the video coding score range, it is necessary to re-predict the bit rate factor and encode the initial video clip again. Specifically, in the second coding process, in addition to the above-mentioned spatiotemporal domain feature information and coding feature information, it is also necessary to add the first bit rate factor and the first coding score to determine the second bit rate factor, where the second bit rate factor specifically refers to the bit rate factor used for the second video coding.

[0106] Specifically, determining the second rate factor based on the spatiotemporal feature information, the coding feature information, the first rate factor, and the first coding score includes:

[0107] Splicing the spatiotemporal feature information, the coding feature information, the first rate factor, and the first coding score to generate second feature information to be processed;

[0108] inputting the second feature information to be processed into a second rate factor prediction model, wherein the second rate factor prediction model is trained to generate a predicted rate factor based on the second feature information to be processed;

[0109] Receive a second rate factor generated by the second rate factor prediction model according to the second feature information to be processed.

[0110] Similar to determining the first feature information to be processed in the above steps, the second feature information to be processed is based on the first feature information to be processed, with the first rate factor and the first coding score added. That is, the first feature information to be processed has 370 feature values, and the first rate factor and the first coding score are added to form the second feature information to be processed with 372 feature values.

[0111] The second feature information to be processed is input into the second rate factor prediction model. The model structure of the second rate factor prediction model refers to the above-mentioned rate factor prediction model. The training method of the second rate factor prediction model also refers to the above-mentioned training method of the rate factor prediction model. The difference between the second rate factor prediction model and the first rate factor prediction model is that the input of the second rate factor prediction model has two more feature values ​​than the input of the first rate factor prediction model, namely, the first rate factor and the first coding score.

[0112] The second rate factor prediction model determines the second rate factor based on the second feature information to be processed. After determining the second rate factor for the second video encoding, the initial video clip is encoded according to the second rate factor. The initial video clip is then re-encoded using the encoder used by the transcoding system. Multiple tests have shown that after the second video encoding, the encoding score obtained has a 99% probability of falling within a preset video encoding score range. Therefore, the result of the second video encoding can be directly trusted.

[0113] After encoding each to-be-encoded video segment, the method further includes:

[0114] The target video segments corresponding to each to-be-encoded video segment are spliced ​​based on the order of each to-be-encoded video segment in the to-be-encoded video to obtain the target video corresponding to the to-be-encoded video.

[0115] After performing the above encoding operation on each video segment to be encoded in a video to be encoded, the target video segment corresponding to each video segment to be encoded can be obtained, the encoding result of each video segment to be encoded is written into the final video code stream file, and then each target video segment is spliced ​​according to the order of each video segment to be encoded in the video to be encoded, and finally the target video corresponding to the video to be encoded is obtained, and the encoding process of the video to be encoded is finally completed.

[0116] The method provided in one embodiment of the present application realizes the extraction of spatiotemporal feature information and coding feature information based on a set of video frames to be encoded, and predicts the bit rate factor used for video encoding based on the spatiotemporal feature information and coding feature information, and encodes the video based on a preset video encoding score and bit rate factor, thereby obtaining a target video clip. Through the method of predicted bit rate factor and constant video encoding score, the accuracy of video encoding is effectively improved, that is, the video quality is guaranteed and the size of the video compressed file is significantly reduced.

[0117] In the process of extracting the coding feature information of the video to be encoded, the set of video frames to be encoded is converted into a set of video frame pairs to be encoded, and through the methods of intra-frame coding and inter-frame coding, the spatial coding complexity and temporal coding complexity of the video to be encoded are fully extracted, providing rich reference basis for the subsequent prediction of the bit rate factor.

[0118] A large amount of experimental data shows that after the first prediction and encoding, the accuracy of the video is 40%. Therefore, only 60% of the video clips will undergo a second prediction and encoding. Therefore, the average encoding complexity of the method provided in this application is 1.6 encodings. Under such encoding complexity, the method can output a video stream with constant quality for the video clip, which can stably guarantee the picture quality experience while avoiding bit rate waste.

[0119] The following combined Figure 3 , taking the application of the video encoding method provided by this application in encoding video T as an example, the video encoding method is further described. Figure 3 A processing flow chart of a video encoding method applied to video T encoding provided by an embodiment of the present application is shown, which specifically includes the following steps:

[0120] Step 302: Obtain a video T, perform scene detection on the video T, and divide the video T into multiple video segments to be encoded.

[0121] Step 304: Determine a video segment to be encoded, and decode the video segment to be encoded to obtain a set of video frames to be encoded.

[0122] Step 306: extracting grayscale co-occurrence information and normalization information of the video frame set to be encoded based on the video frame set to be encoded.

[0123] Step 308: Generate spatiotemporal feature information of the video segment to be encoded based on the grayscale symbiosis information and the normalization information.

[0124] Step 310: Generate a set of to-be-encoded video frame pairs according to the to-be-encoded video frame set.

[0125] Step 312: performing intra-frame coding and inter-frame coding on each video frame pair in the set of video frame pairs to be coded, and obtaining coding feature information of the video segment to be coded.

[0126] Step 314: Concatenate the spatiotemporal feature information and the coded feature information to generate first feature information to be processed.

[0127] Step 316: Input the first feature information to be processed into a first rate factor prediction model to obtain a first rate factor output by the first rate factor prediction model.

[0128] Step 318: Encode the to-be-encoded video segment for the first time based on the first bit rate factor to obtain an initial video segment, and determine a first encoding score for the initial video segment.

[0129] Step 320: Determine a video coding score range according to the preset video coding score.

[0130] Step 322: Determine whether the first coding score is within the video coding score range. If so, execute step 324; if not, execute steps 326-330.

[0131] Step 324: Determine that the initial video segment is a target video segment.

[0132] Step 326: Concatenate the spatiotemporal feature information, the coding feature information, the first rate factor, and the first coding score to generate second feature information to be processed.

[0133] Step 328: Input the second feature information to be processed into a second rate factor prediction model to obtain a second rate factor output by the second rate factor prediction model.

[0134] Step 330: Encode the initial video segment according to the second bit rate factor to obtain a target video segment.

[0135] Step 332: splicing the target video segments corresponding to each to-be-encoded video segment based on the order of each to-be-encoded video segment in the video T to obtain the target video corresponding to the video T.

[0136] The method provided in one embodiment of the present application realizes the extraction of spatiotemporal feature information and coding feature information based on a set of video frames to be encoded, and predicts the bit rate factor used for video encoding based on the spatiotemporal feature information and coding feature information, and encodes the video based on a preset video encoding score and bit rate factor, thereby obtaining a target video clip. Through the method of predicted bit rate factor and constant video encoding score, the accuracy of video encoding is effectively improved, that is, the video quality is guaranteed and the size of the video compressed file is significantly reduced.

[0137] In the process of extracting the coding feature information of the video to be encoded, the set of video frames to be encoded is converted into a set of video frame pairs to be encoded, and through the methods of intra-frame coding and inter-frame coding, the spatial coding complexity and temporal coding complexity of the video to be encoded are fully extracted, providing rich reference basis for the subsequent prediction of the bit rate factor.

[0138] Corresponding to the above-mentioned video encoding method embodiment, the present application also provides a video encoding device embodiment, Figure 4 FIG. 1 shows a schematic diagram of the structure of a video encoding device provided by an embodiment of the present application. Figure 4 As shown, the device includes:

[0139] The acquisition module 402 is configured to acquire a video segment to be encoded;

[0140] The decoding module 404 is configured to decode the video segment to be encoded to obtain a set of video frames to be encoded;

[0141] An extraction module 406 is configured to extract spatiotemporal feature information and coding feature information of the video segment to be encoded based on the set of video frames to be encoded;

[0142] a determination module 408 configured to determine a bit rate factor of the to-be-encoded video segment based on the spatiotemporal feature information and the encoding feature information;

[0143] The encoding module 410 is configured to encode the to-be-encoded video segment according to a preset video encoding score and the bit rate factor to obtain a target video segment.

[0144] Optionally, the extraction module 406 is further configured to:

[0145] Extracting grayscale co-occurrence information and normalization information of the to-be-encoded video frame set based on the to-be-encoded video frame set;

[0146] The spatiotemporal domain feature information of the video segment to be encoded is generated by splicing the grayscale symbiosis information and the normalization information.

[0147] Optionally, the extraction module 406 is further configured to:

[0148] Generating a set of video frame pairs to be encoded according to the set of video frames to be encoded;

[0149] Intra-frame coding and inter-frame coding are performed on each video frame pair in the set of video frame pairs to be coded, to obtain coding feature information of the video segment to be coded.

[0150] Optionally, the extraction module 406 is further configured to:

[0151] Performing intra-frame coding on the first video frame of each video frame pair in the set of video frame pairs to be coded to obtain intra-frame coding information;

[0152] Performing inter-frame coding on each video frame pair in the set of video frame pairs to be coded to obtain inter-frame coding information;

[0153] The intra-frame coding information and the inter-frame coding information are concatenated to obtain coding feature information.

[0154] Optionally, the determining module 408 is further configured to:

[0155] Splicing the spatiotemporal feature information and the coded feature information to generate first feature information to be processed;

[0156] Inputting the first feature information to be processed into a first rate factor prediction model, wherein the first rate factor prediction model is trained to generate a predicted rate factor based on the first feature information to be processed;

[0157] A first rate factor generated by the first rate factor prediction model according to the first feature information to be processed is received.

[0158] Optionally, the encoding module 410 is further configured to:

[0159] Performing a first encoding on the to-be-encoded video segment based on the first bit rate factor to obtain an initial video segment, and determining a first encoding score for the initial video segment;

[0160] Determining a video coding score interval according to the preset video coding score;

[0161] When the first coding score is within the video coding score interval, determining the initial video segment as a target video segment;

[0162] When the first coding score is outside the video coding score range, the initial video segment is encoded based on the spatiotemporal feature information, the coding feature information, the first bit rate factor, and the first coding score to obtain a target video segment.

[0163] Optionally, the encoding module 410 is further configured to:

[0164] Determining a second rate factor based on the spatiotemporal feature information, the coding feature information, the first rate factor, and the first coding score;

[0165] The initial video segment is encoded according to the second rate factor.

[0166] Optionally, the encoding module 410 is further configured to:

[0167] Splicing the spatiotemporal feature information, the coding feature information, the first rate factor, and the first coding score to generate second feature information to be processed;

[0168] inputting the second feature information to be processed into a second rate factor prediction model, wherein the second rate factor prediction model is trained to generate a predicted rate factor based on the second feature information to be processed;

[0169] Receive a second rate factor generated by the second rate factor prediction model according to the second feature information to be processed.

[0170] Optionally, the apparatus further includes a training module configured to: obtain sample feature information and a sample bit rate factor corresponding to the sample feature information;

[0171] Inputting the sample feature information into a rate factor prediction model for processing, and receiving a predicted rate factor output by the rate factor prediction model;

[0172] Calculating a model loss value according to the predicted bit rate factor and the sample bit rate factor;

[0173] Adjust the model parameters of the rate factor prediction model according to the model loss value, and continue to train the rate factor prediction model until a model training stop condition is reached.

[0174] Optionally, the device further includes:

[0175] The video acquisition module 402 is configured to acquire the video to be encoded;

[0176] a detection module configured to perform scene detection on the video to be encoded and split the video to be encoded into multiple single-shot video segments;

[0177] The video determination module is configured to determine each single-shot video segment as a video segment to be encoded.

[0178] Optionally, the device further includes:

[0179] The splicing module is configured to splice the target video segments corresponding to each to-be-encoded video segment based on the order of each to-be-encoded video segment in the to-be-encoded video to obtain the target video corresponding to the to-be-encoded video.

[0180] The device provided in one embodiment of the present application realizes the extraction of spatiotemporal feature information and coding feature information based on a set of video frames to be encoded, and predicts the bit rate factor used for video encoding based on the spatiotemporal feature information and coding feature information, and encodes the video based on a preset video encoding score and bit rate factor, thereby obtaining a target video clip. Through the method of predicted bit rate factor and constant video encoding score, the accuracy of video encoding is effectively improved, that is, the video quality is guaranteed and the size of the video compressed file is significantly reduced.

[0181] In the process of extracting the coding feature information of the video to be encoded, the set of video frames to be encoded is converted into a set of video frame pairs to be encoded, and through the methods of intra-frame coding and inter-frame coding, the spatial coding complexity and temporal coding complexity of the video to be encoded are fully extracted, providing rich reference basis for the subsequent prediction of the bit rate factor.

[0182] A large amount of experimental data shows that after the first prediction and encoding, the accuracy of the video is 40%. Therefore, only 60% of the video clips will undergo a second prediction and encoding. Therefore, the average encoding complexity of the method provided in this application is 1.6 encodings. Under such encoding complexity, the method can output a video stream with constant quality for the video clip, which can stably guarantee the picture quality experience while avoiding bit rate waste.

[0183] The above is a schematic scheme of a video encoding device of this embodiment. It should be noted that the technical scheme of the video encoding device and the technical scheme of the above-mentioned video encoding method are based on the same concept. For details not described in detail in the technical scheme of the video encoding device, please refer to the description of the technical scheme of the above-mentioned video encoding method.

[0184] Figure 5 The block diagram shows a structure of a computing device 500 according to an embodiment of the present application. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0185] The computing device 500 also includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)), whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0186] In one embodiment of the present application, the above components of the computing device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 5 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0187] Computing device 500 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. Computing device 500 can also be a mobile or stationary server.

[0188] The processor 520 implements the steps of the video encoding method when executing the computer instructions.

[0189] The above is a schematic solution of a computing device of this embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above-mentioned video encoding method are based on the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above-mentioned video encoding method.

[0190] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the video encoding method as described above.

[0191] The above is an illustrative embodiment of a computer-readable storage medium. It should be noted that the technical solution of the storage medium and the technical solution of the video encoding method described above are based on the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the video encoding method described above.

[0192] The foregoing description describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0193] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0194] It should be noted that for the aforementioned method embodiments, for ease of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0195] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0196] The preferred embodiments of the present application disclosed above are intended only to help illustrate the present application. The optional embodiments do not describe all details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of this application. This application selects and describes these embodiments in detail in order to better explain the principles and practical applications of this application, so that those skilled in the art can better understand and utilize this application. This application is limited only by the claims and their full scope and equivalents.

Claims

1. A video encoding method, characterized in that: include: Get the video clip to be encoded; Decoding the video segment to be encoded to obtain a set of video frames to be encoded; Extracting spatiotemporal feature information and coding feature information of the video segment to be encoded based on the set of video frames to be encoded; Determining a bit rate factor of the to-be-encoded video segment according to the spatiotemporal feature information and the encoding feature information; encoding the to-be-encoded video segment according to a preset video encoding score and the bit rate factor to obtain a target video segment; The step of determining the bit rate factor of the video segment to be encoded according to the spatiotemporal feature information and the encoding feature information includes: Splicing the spatiotemporal feature information and the coded feature information to generate first feature information to be processed; Inputting the first feature information to be processed into a first rate factor prediction model, wherein the first rate factor prediction model is trained to generate a predicted rate factor based on the first feature information to be processed; receiving a first rate factor generated by the first rate factor prediction model according to the first feature information to be processed; The step of encoding the to-be-encoded video segment according to the preset video encoding score and the bit rate factor to obtain a target video segment includes: Performing a first encoding on the to-be-encoded video segment based on the first bit rate factor to obtain an initial video segment, and determining a first encoding score for the initial video segment; Determining a video coding score interval according to the preset video coding score; When the first coding score is outside the video coding score range, the spatiotemporal feature information, the coding feature information, the first rate factor, and the first coding score are concatenated to generate second feature information to be processed; the second feature information to be processed is input into a second rate factor prediction model to determine a second rate factor; and the initial video clip is encoded according to the second rate factor.

2. The method according to claim 1, wherein Extracting spatiotemporal feature information of the to-be-encoded video segment based on the to-be-encoded video frame set includes: Extracting grayscale symbiosis information and normalization information of the to-be-encoded video frame set based on the to-be-encoded video frame set, wherein the grayscale symbiosis information is used to describe characteristics of spatial texture complexity of the to-be-encoded video segment, and the normalization information is used to describe characteristics of temporal complexity of the to-be-encoded video segment; The spatiotemporal domain feature information of the video segment to be encoded is generated by splicing the grayscale symbiosis information and the normalization information.

3. The method according to claim 1, wherein Extracting encoding feature information of the to-be-encoded video segment based on the to-be-encoded video frame set includes: Generating a set of video frame pairs to be encoded according to the set of video frames to be encoded; Intra-frame coding and inter-frame coding are performed on each video frame pair in the set of video frame pairs to be coded, to obtain coding feature information of the video segment to be coded.

4. The method according to claim 3, wherein Performing intra-frame coding and inter-frame coding on each video frame pair in the set of video frame pairs to be coded to obtain coding feature information of the video segment to be coded, including: Performing intra-frame coding on the first video frame of each video frame pair in the set of video frame pairs to be coded to obtain intra-frame coding information; Performing inter-frame coding on each video frame pair in the set of video frame pairs to be coded to obtain inter-frame coding information; The intra-frame coding information and the inter-frame coding information are concatenated to obtain coding feature information.

5. The method according to claim 1, wherein Encoding the to-be-encoded video segment according to the preset video encoding score and the bit rate factor to obtain a target video segment includes: When the first encoding score is within the video encoding score range, the initial video segment is determined to be the target video segment.

6. The method according to claim 1, wherein The second rate factor prediction model is trained to generate a predicted rate factor according to the second feature information to be processed.

7. The method according to claim 1 or 6, wherein: The rate factor prediction model is obtained through the following training method: Obtaining sample characteristic information and a sample rate factor corresponding to the sample characteristic information; Inputting the sample feature information into a rate factor prediction model for processing, and receiving a predicted rate factor output by the rate factor prediction model; Calculating a model loss value according to the predicted bit rate factor and the sample bit rate factor; Adjust the model parameters of the rate factor prediction model according to the model loss value, and continue to train the rate factor prediction model until a model training stop condition is reached.

8. The method according to claim 1, wherein Get the video clip to be encoded, including: Get the video to be encoded; Performing scene detection on the video to be encoded, and splitting the video to be encoded into multiple single-shot video clips; Each single-shot video segment is determined to be a video segment to be encoded.

9. The method according to claim 8, wherein The method further comprises: The target video segments corresponding to each to-be-encoded video segment are spliced ​​based on the order of each to-be-encoded video segment in the to-be-encoded video to obtain the target video corresponding to the to-be-encoded video.

10. A video encoding device, characterized in that: include: An acquisition module, configured to acquire a video segment to be encoded; A decoding module is configured to decode the video segment to be encoded to obtain a set of video frames to be encoded; an extraction module configured to extract spatiotemporal feature information and coding feature information of the video segment to be encoded based on the set of video frames to be encoded; a determination module, configured to determine a bit rate factor of the to-be-encoded video segment according to the spatiotemporal feature information and the encoding feature information; an encoding module configured to encode the to-be-encoded video segment according to a preset video encoding score and the bit rate factor to obtain a target video segment; Wherein, the determining module is further configured to: Splicing the spatiotemporal feature information and the coded feature information to generate first feature information to be processed; Inputting the first feature information to be processed into a first rate factor prediction model, wherein the first rate factor prediction model is trained to generate a predicted rate factor based on the first feature information to be processed; receiving a first rate factor generated by the first rate factor prediction model according to the first feature information to be processed; Wherein, the encoding module is further configured to: Performing a first encoding on the to-be-encoded video segment based on the first bit rate factor to obtain an initial video segment, and determining a first encoding score for the initial video segment; Determining a video coding score interval according to the preset video coding score; When the first coding score is outside the video coding score range, the spatiotemporal feature information, the coding feature information, the first rate factor, and the first coding score are concatenated to generate second feature information to be processed; the second feature information to be processed is input into a second rate factor prediction model to determine a second rate factor; and the initial video clip is encoded according to the second rate factor.

11. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein: When the processor executes the computer instructions, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images