Video encoding method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a video encoding method and device, electronic equipment and storage medium. The method comprises: obtaining a to-be-encoded video segment, the to-be-encoded video segment comprising one or more video frames; calculating spatio-temporal domain feature information of the to-be-encoded video segment based on the one or more video frames; determining a plurality of predicted code rate factors of the to-be-encoded video segment under a plurality of preset video encoding standards based on the spatio-temporal domain feature information; determining a plurality of target code rate factors of the to-be-encoded video segment under the plurality of preset video encoding standards based on a first predicted code rate factor corresponding to a first preset video encoding standard in the plurality of preset video encoding standards and a preset video encoding score interval, wherein the time for video encoding using the first preset video encoding standard is the shortest among the plurality of preset video encoding standards; and encoding the to-be-encoded video segment according to the plurality of target code rate factors to obtain a plurality of target video segments.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to a video encoding method and apparatus, electronic equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of internet technology, video platforms have also experienced rapid growth, with more and more users sharing and watching videos. For these platforms, as the number of users continues to increase, the bandwidth costs for video transmission are also rising. To reduce these costs, a transcoding system with better compression performance is needed to significantly reduce the size of the compressed video while maintaining video quality, thereby reducing the network traffic required for transmission.

[0003] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0004] This disclosure provides a video encoding method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of this disclosure, a video encoding method is provided, comprising: acquiring a video segment to be encoded, the video segment to be encoded including one or more video frames; calculating spatiotemporal feature information of the video segment to be encoded based on the one or more video frames; determining multiple predicted bitrate factors of the video segment to be encoded under multiple preset video encoding standards based on the spatiotemporal feature information; determining multiple target bitrate factors of the video segment to be encoded under multiple preset video encoding standards based on a first predicted bitrate factor corresponding to a first preset video encoding standard among the multiple preset video encoding standards and a preset video encoding score interval, wherein, among the multiple preset video encoding standards, the video encoding time using the first preset video encoding standard is the shortest; and encoding the video segment to be encoded according to the multiple target bitrate factors respectively to obtain multiple target video segments.

[0006] According to another aspect of this disclosure, a video encoding apparatus is also provided, comprising: an acquisition module configured to acquire a video segment to be encoded, the video segment to be encoded including one or more video frames; a calculation module configured to calculate spatiotemporal feature information of the video segment to be encoded based on the one or more video frames; a prediction bitrate factor determination module configured to determine multiple prediction bitrate factors of the video segment to be encoded under multiple preset video encoding standards based on the spatiotemporal feature information; a target bitrate factor determination module configured to determine multiple target bitrate factors of the video segment to be encoded under multiple preset video encoding standards based on a first prediction bitrate factor corresponding to a first preset video encoding standard among the multiple preset video encoding standards and a preset video encoding score range, wherein, among the multiple preset video encoding standards, the video encoding time using the first preset video encoding standard is the shortest; and an encoding module configured to encode the video segment to be encoded according to the multiple target bitrate factors respectively to obtain multiple target video segments.

[0007] According to another aspect of this disclosure, an electronic device is also provided, comprising: at least one processor; and at least one memory communicatively connected to the at least one processor, wherein the at least one memory stores a computer program that, when executed by the at least one processor, implements the video encoding method described above.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium storing a computer program is also provided, wherein the computer program implements the above-described video encoding method when executed by a processor.

[0009] According to another aspect of this disclosure, a computer program product is also provided, comprising a computer program, wherein the computer program implements the above-described video encoding method when executed by a processor.

[0010] According to one or more embodiments of this disclosure, a first prediction of the bitrate factor under multiple preset video coding standards is performed based on the spatiotemporal feature information of the video segment to be encoded. A second prediction of the bitrate factor under multiple preset video coding standards is then performed based on the predicted bitrate factor and preset video coding scores. This improves the accuracy of the bitrate factor, thereby effectively enhancing the accuracy of video coding and ensuring video quality. Furthermore, since the bitrate factor under multiple preset video coding standards can be predicted simultaneously, the encoding system can simultaneously output multiple encoded videos with different video coding standards, all meeting video quality requirements. This avoids repeatedly performing prediction operations for multiple video coding standards, thus saving time. Further, the second prediction of the bitrate factor is based on the bitrate factor corresponding to the video coding standard with the shortest encoding time in the first prediction result. Since the video coding standard with the shortest encoding time has the lowest computational complexity, the bitrate factor under this standard can, to some extent, reflect the accuracy of the bitrate factor under other video coding standards with longer encoding times, thereby linking different video coding standards and further reducing the computational load required for predicting the bitrate factor.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0013] Figure 1 A flowchart of a video encoding method according to some embodiments of the present disclosure is shown;

[0014] Figure 2 A flowchart is shown illustrating a method for training a first code rate factor prediction model according to some embodiments of the present disclosure;

[0015] Figure 3 A structural block diagram of a first rate factor prediction model according to some embodiments of the present disclosure is shown;

[0016] Figure 4 A flowchart illustrating the determination of multiple target bitrate factors for a video segment to be encoded under multiple preset video coding standards according to some embodiments of the present disclosure is shown.

[0017] Figure 5A structural block diagram of a video encoding apparatus according to an embodiment of the present disclosure is shown;

[0018] Figure 6 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0019] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0020] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0021] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0022] With the development of internet technology, video platforms have also experienced rapid growth, with more and more users sharing and watching videos. For these platforms, as the number of users continues to increase, the bandwidth costs for video transmission are also rising. To reduce these costs, a transcoding system with better compression performance is needed to significantly reduce the size of the compressed video while maintaining video quality, thereby reducing the network traffic required for transmission.

[0023] Currently, common encoding modes in transcoding systems include fixed quantization parameter mode, constant bitrate mode, and variable bitrate mode, among which constant bitrate mode is the most commonly used. While this mode outputs a stable bitrate, improving playback stuttering, it also suffers from limitations. Because it encodes all videos using the same bitrate factor, it cannot guarantee consistent image quality for videos with rich scene content, and it also cannot avoid bitrate waste. For example, encoding videos with complex scenes using an average bitrate factor will result in insufficient bitrate and a significant drop in image quality, while encoding videos with simple scenes may lead to unnecessary bitrate waste.

[0024] To ensure a high-quality viewing experience and avoid wasted bitrate, a constant quality encoding mode can be used. Specifically, a target quality can be set, and an encoder can be used to encode the video to obtain a video that meets that target quality. The goal of this encoding mode is to make the quality of the encoded video as close as possible to the set target quality; the closer the quality of the encoded video is to the set target quality, the better the encoding effect. When the actual quality is lower than the target quality, it may reduce the video's viewing experience, while when the actual quality exceeds the target quality, it may result in wasted bitrate.

[0025] However, before encoding video using this encoding mode, the relationship between the bitrate factor used for encoding and the actual quality of the encoded result is unknown; that is, the accurate bitrate factor cannot be directly obtained from a set target quality. A common solution is:

[0026] (1) Encode the video multiple times and obtain the optimal bitrate factor through a post-hoc method by searching, but this method requires huge computing resources;

[0027] (2) Predicting the bitrate factor. For example, machine learning can be used to predict the bitrate factor based on the spatiotemporal complexity information of the video before encoding. However, this solution is usually a one-time prediction and encoding approach, that is, the bitrate factor is predicted based on the spatiotemporal complexity information of the video according to a preset goal, and the video is encoded based on the predicted bitrate factor. This may result in low accuracy of the bitrate factor prediction, especially for videos with complex scenes or popular videos, which may not be able to reliably ensure the image quality experience while avoiding bitrate waste.

[0028] Furthermore, the inventors discovered that, in order to adapt to various terminal players, the transcoding system needs to output video streams that meet different video encoding standards. However, as mentioned above, existing technologies typically employ a one-time prediction and encoding approach, which means that for each video encoding standard, the transcoding system needs to perform bitrate factor prediction and video encoding once, thereby increasing computational load and time costs.

[0029] In view of this, embodiments of the present disclosure provide a video coding method. This method performs a first prediction of bitrate factors under multiple preset video coding standards based on the spatiotemporal feature information of the video segment to be encoded, and then performs a second prediction of bitrate factors under multiple preset video coding standards based on the predicted bitrate factors and preset video coding scores. This improves the accuracy of the bitrate factors, thereby effectively enhancing the accuracy of video coding and ensuring video quality. Furthermore, since bitrate factors under multiple preset video coding standards can be predicted simultaneously, the coding system can simultaneously output multiple encoded videos with different video coding standards, all of which meet video quality requirements. This avoids repeatedly performing prediction operations for multiple video coding standards, thus saving time costs.

[0030] Furthermore, in this method, the second prediction of the bitrate factor is based on the bitrate factor corresponding to the video coding standard with the shortest coding time in the first prediction result. Since the video coding standard with the shortest coding time has the lowest computational complexity, the bitrate factor under this video coding standard can, to some extent, reflect the accuracy of the bitrate factor under other video coding standards with longer coding times, thereby linking different video coding standards and helping to further reduce the amount of computation required to predict the bitrate factor.

[0031] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0032] Figure 1 A flowchart of a video encoding method 100 according to some embodiments of the present disclosure is shown. For example... Figure 1As shown, method 100 may include: step S110, obtaining a video segment to be encoded, the video segment to be encoded including one or more video frames; step S120, calculating spatiotemporal feature information of the video segment to be encoded based on the one or more video frames; step S130, determining multiple predicted bitrate factors of the video segment to be encoded under multiple preset video coding standards based on the spatiotemporal feature information; step S140, determining multiple target bitrate factors of the video segment to be encoded under multiple preset video coding standards based on the first predicted bitrate factor corresponding to the first preset video coding standard among the multiple preset video coding standards and the preset video coding score interval, wherein, among the multiple preset video coding standards, the video encoding time using the first preset video coding standard is the shortest; and step S150, encoding the video segment to be encoded according to the multiple target bitrate factors respectively to obtain multiple target video segments.

[0033] By utilizing the spatiotemporal features of the video segment to be encoded to determine the predicted bitrate factor under multiple preset video coding standards, and then using the predicted bitrate factor and preset video coding scores to determine the target bitrate factor under these standards, the bitrate factor can be corrected. This improves the accuracy of the bitrate factor and, consequently, the accuracy of video coding, ensuring video quality. Furthermore, since the bitrate factor can be predicted simultaneously for multiple preset video coding standards, the coding system can output multiple encoded videos with different coding standards that all meet video quality requirements, avoiding repeated prediction operations for multiple standards and saving time. Further, the bitrate factor correction is based on the bitrate factor corresponding to the shortest coding time in the first prediction result. Since the shortest coding time has the lowest computational complexity, its bitrate factor can reflect the accuracy of bitrate factors under other, longer coding times, thus linking different video coding standards and achieving joint prediction of bitrate factors for different standards, further reducing the computational load required for bitrate factor prediction.

[0034] According to some embodiments of this disclosure, step S110, obtaining the video segment to be encoded, may include: obtaining the video to be encoded; performing scene detection on the video to be encoded; dividing the video to be encoded into one or more sub-video segments based on the scene detection results; and identifying each video sub-segment in the one or more video segments as the video segment to be encoded.

[0035] According to some embodiments, the stored or cached video to be encoded can be read from a suitable storage device (local and / or remote). Alternatively, the video to be encoded can also be received from other external devices via a wired or wireless communication link. The video to be encoded can refer to any complete video file. For example, it can be a video file recorded by the user, a video file extracted by the user from other video files, or a video file created by the user based on multiple video files; the scope of the subject matter claimed in this disclosure is not limited in this respect.

[0036] After obtaining the video to be encoded, scene detection can be performed on the video to be encoded. Based on the detection results, the video to be encoded can be segmented, and each segmented video segment can be used as a video segment to be encoded.

[0037] According to some embodiments, scene detection can be performed on the video to be encoded based on an open-source encoder. For example, in a scenario where video V is determined to be the video to be encoded in response to receiving a user-uploaded video V, the open-source encoder x264 can be invoked to encode video V, and the scene detection switch can be turned on to obtain the corresponding scene transition detection result. Based on the detection result, video V can be divided into t independent single-shot video segments (V1, V2...Vt), and each single-shot video segment can be used as a video segment to be encoded.

[0038] According to other embodiments, the video to be encoded can be segmented based on frame segmentation. Continuing with the example of the video to be encoded being V, the video to be encoded V can be segmented into multiple video frames. Then, for each video frame, the similarity between that video frame and its adjacent video frames is determined, and based on the calculated similarity, it is determined whether that video frame is a boundary video frame used to distinguish adjacent shots in the video to be processed. It should be understood that any suitable decoder or other technology (e.g., OpenCV, FFmpeg, etc.) can be used for frame segmentation, and any suitable calculation method can also be used to determine the similarity between adjacent video frames (e.g., calculating frame difference, grayscale histogram, etc.). The scope of the subject matter claimed in this disclosure is not limited in this respect.

[0039] Compared to a complete video file, the content shown in a single-shot video clip is usually similar and more coherent. Therefore, when encoding the video, it is more reasonable to use a specific, same bitrate factor for each single-shot video clip, thereby ensuring a stable picture quality experience in various scenarios.

[0040] According to some embodiments of this disclosure, the spatiotemporal domain feature information may include spatiotemporal complexity information, and wherein step S120, calculating the spatiotemporal domain feature information of the video segment to be encoded based on one or more video frames, may include: calculating gray-level co-occurrence information and normalized information of the video segment to be encoded as spatiotemporal complexity information based on the one or more video frames, wherein the gray-level co-occurrence information is used to describe the spatial complexity of the video segment to be encoded, and the normalized information is used to describe the temporal complexity of the video segment to be encoded; and combining the gray-level co-occurrence information and the normalized information to generate the spatiotemporal complexity information of the video segment to be encoded.

[0041] Gray-level co-occurrence information (also known as the gray-level co-occurrence matrix, GLCM) is a feature used to describe the spatial texture complexity of a video. In some embodiments, for each video frame in one or more video frames of a video segment to be encoded, an m×m two-dimensional GLCM matrix can be calculated, where each element in the GLCM matrix takes values ​​ranging from 0 to 1. Then, five eigenvalues—entropy, contrast, inverse different moment, energy, and correlation—can be further calculated for this matrix, and four statistical values—mean, variance, skewness, and kurtosis—can be calculated for these eigenvalues ​​corresponding to all video frames. For example, the mean, variance, skewness, and kurtosis of the entropy of each video frame to be processed can be calculated. A total of 20 (5×4) eigenvalues ​​will be obtained. Furthermore, to consider the multi-scale characteristics of the video, each video frame can be scaled down by a factor of 4 and 16, and the above steps can be repeated (i.e., GLCM feature extraction can be performed twice). Therefore, the video frames of the video segment to be encoded will eventually obtain 60 feature values, namely 60 GLCM feature values.

[0042] Normalized information (also known as normalized correlation coefficient, NCC) is a feature used to describe temporal complexity. In some embodiments, for each pair of adjacent video frames in one or more video frames of a video segment to be encoded, the NCC can be calculated to obtain a two-dimensional correlation coefficient matrix with the same dimensions as the original video frames. Each element in the matrix ranges from 0 to 1, representing the temporal correlation between corresponding positions of two adjacent video frames. For a video segment to be encoded containing n video frames, there will be n-1 pairs of adjacent video frames, thus yielding n-1 correlation coefficient matrices. Then, the mean, variance, kurtosis, skewness, and information entropy of these n-1 correlation coefficient matrices are calculated, totaling five eigenvalues. For each eigenvalue, the mean and variance are calculated in the temporal domain, ultimately resulting in 10 (5×2) eigenvalues, or 10 NCC eigenvalues.

[0043] After calculating the GLCM and NCC feature values ​​using methods such as those described above, the two can be combined to obtain 70 feature values ​​as the spatiotemporal complexity information of the video segment to be encoded. This spatiotemporal complexity information can be represented in vector form.

[0044] Since grayscale co-occurrence information and normalization information can characterize the complexity of the video to be encoded from spatial and temporal perspectives, respectively, predicting multiple bitrate factors of the video segment to be encoded under different video coding standards based on these spatial complexity information can make the predicted bitrate factors more accurate, thereby helping to ensure the quality of the output video.

[0045] According to some embodiments of this disclosure, the spatiotemporal domain feature information may include coded feature information, and wherein step S120, calculating the spatiotemporal domain feature information of the video segment to be encoded based on one or more video frames, may include: preprocessing one or more video frames to generate a new video frame sequence, wherein the new video frame sequence includes a set of video frame pairs; and performing intra-frame coding and inter-frame coding on each video frame pair in the set of video frame pairs to obtain the coded feature information of the video segment to be encoded.

[0046] By performing intra-frame or inter-frame coding on each video frame in the video to be encoded, the spatiotemporal features of the video segment can be reflected from another dimension, namely spatial coding complexity and temporal coding complexity. This is because intra-frame coding uses the image data within the video frame to be encoded, while inter-frame coding uses the image data of the video frame to be encoded and its adjacent video frames, thus providing a more comprehensive characterization of the video segment to be encoded. Using this coding feature information in combination with the spatiotemporal complexity information composed of GLCM and NCC feature values ​​to predict multiple bitrate factors of the video segment to be encoded under different video coding standards can further improve the accuracy of the prediction results. This is particularly beneficial for hotspot videos that require higher quality and more refined coding.

[0047] According to some embodiments of this disclosure, in step S120, preprocessing one or more video frames to generate a new video frame sequence may include: copying video frames other than the first and last frames in one or more video frames to generate one or more copied video frames; and sorting the one or more video frames and the one or more copied video frames such that each of the one or more copied video frames is located after the corresponding video frame.

[0048] According to some embodiments, after generating one or more duplicate frames corresponding to video frames other than the first and last frames, each duplicate frame can be placed after the corresponding original video frame to generate a new video frame sequence. For example, suppose the video segment to be encoded consists of n video frames and the sequence of these n video frames is (f1, f2, f3, ..., f...). n-2 ,f n-1 ,f n ,), then except for the first frame f1 and the last frame f n In addition, for video frames f2 to f n-1 Each video frame in the sequence is copied to generate a new video frame sequence (f1, f2, f2, f3, f3, ..., f n-2 ,f n-2 ,f n-1 ,f n-1 ,f n , where (f1,f2), (f2,f3)……(f n-2 ,f n-1 ), (f n-1 ,f n () represents a video frame pair.

[0049] Compared to video frame sequences without duplicated frames, by copying video frames other than the first and last frames and generating one or more ordered video frame pairs, it is possible to ensure that data within each video frame and between adjacent video frames can be utilized, preserving data integrity. This facilitates more accurate intra-frame and inter-frame coding of each video frame of the video segment to be coded, thereby improving the accuracy of the predicted bitrate factor.

[0050] According to other embodiments of this disclosure, preprocessing one or more video frames may further include, for example, cropping each video frame to meet size requirements, removing unnecessary (interference) features from the video frames, etc. This can accelerate computation and prediction speed, quickly obtain coding feature information and predict the bitrate factor of the video to be encoded, and also improve the accuracy of the predicted bitrate factor.

[0051] After preprocessing the video segment to be encoded to obtain a set of video frame pairs, pre-encoding can be performed using, for example, a hardware H.264 encoder built into the graphics card. Pre-encoding can employ, for example, a fixed quantization parameter mode, and optionally, the quantization parameter can be any suitable value such as 28, 29, or 30. The encoded frame structure can adopt, for example, an IP structure, where I represents intra-frame coding and P represents inter-frame coding. In this coding mode, except for the first and last frames, each video frame of the video segment to be encoded is encoded twice (i.e., once using intra-frame coding I and once using inter-frame coding P). Intra-frame coding reflects the spatial coding complexity of each video frame, and inter-frame coding reflects the temporal coding complexity of each video frame. After IP coding of each set of video frame pairs, the encoded feature information of the video segment to be encoded can be obtained.

[0052] According to some embodiments of this disclosure, performing intra-frame coding and inter-frame coding on each video frame pair in the video frame pair set to obtain coding feature information of the video segment to be coded may include: performing intra-frame coding on the first video frame in each video frame pair in the video frame pair set to obtain intra-frame coding information; performing inter-frame coding on each video frame pair in the video frame pair set to obtain inter-frame coding information; and obtaining coding feature information based on the intra-frame coding information and the inter-frame coding information.

[0053] Each video frame pair of the video segment to be encoded contains two video frames. In this application, the first video frame refers to the video frame that appears first in the video frame pair. For example, for the video frame pair (f1, f2), f1 is the first video frame; for the video frame pair to be encoded (f2, f3), f2 is the first video frame, and so on.

[0054] Based on some examples, intra-coding the first video frame in each video frame pair to obtain intra-coding information may include: extracting feature values ​​of each intra-coding after intra-coding each first video frame, such as the number of encoded bits, the proportion of intra-prediction modes, and 34 other feature values; and calculating five statistical values ​​of the mean, variance, skewness, kurtosis, and information entropy of the corresponding feature values ​​of all intra-codings to obtain 170 (i.e., 34×5) intra-coding information.

[0055] For each video pair, the inter-frame coding information of each video frame pair can also be extracted. For example, for video frame pair (f1, f2), the inter-frame coding information of f1 and f2 is calculated; for video frame pair (f2, f3), the inter-frame coding information of f2 and f3 is calculated, and so on.

[0056] According to some examples, inter-frame coding for each pair of video frames to obtain inter-frame coding information may include: extracting the feature values ​​of each inter-frame code after inter-frame coding, such as the number of coded bits, the amplitude distribution of inter-frame motion vectors, and 26 other feature values; and calculating the mean, variance, skewness, kurtosis, and information entropy of the corresponding feature values ​​of all inter-frame codes to obtain 130 (i.e., 26×5) inter-frame coding information.

[0057] Continuing the example above, the 170 intra-frame coded information and 130 inter-frame coded information are concatenated to obtain the coded feature information of 300 video segments to be encoded. Similar to spatial complexity information, the coded feature information can also be represented in vector form.

[0058] It will be understood that, in the above embodiments, although the intra-frame coding steps are described as being performed after the inter-frame coding steps, this should not be construed as requiring intra-frame coding and inter-frame coding to be performed in this specific order. For example, the intra-frame coding steps may be performed in parallel with the inter-frame coding steps, or the intra-frame coding steps may be performed after the inter-frame coding steps.

[0059] It will also be understood that, in the above embodiments, although intra-frame coding information and inter-frame coding information are described as the mean, variance, skewness, kurtosis and information entropy of characteristic values ​​such as the number of coded bits, the proportion of intra-frame prediction modes, and the amplitude distribution of inter-frame motion vectors, these characteristic values ​​can also be directly determined as intra-frame coding information or inter-frame coding information.

[0060] According to some embodiments of this disclosure, step S130, determining multiple predicted bitrate factors of a video segment to be encoded under multiple preset video coding standards based on spatiotemporal domain feature information, may include: inputting the spatiotemporal domain feature information of the video segment to be encoded into a first bitrate factor prediction model, so as to determine multiple predicted bitrate factors of the video segment to be encoded under multiple preset video coding standards via the first bitrate factor prediction model.

[0061] Rate Factor (RF) is an encoding parameter used in constant rate factor encoding mode to control the bitrate and quality of encoded files. The value of the rate factor typically ranges from 0 to 50. A higher rate factor results in a lower bitrate and worse encoding quality, while a lower rate factor results in a higher bitrate and better encoding quality.

[0062] Examples of multiple preset video coding standards may include H.264, H.265, H.266, and AV1. It will be understood that preset video standards may also include any other existing or evolving video coding standards. It will also be understood that the number of preset video coding standards can be any integer value to meet practical needs. The scope of the subject matter claimed in this disclosure is not limited in either of these respects.

[0063] According to some embodiments of this disclosure, when the spatiotemporal domain feature information only includes spatiotemporal complexity information, the feature information of the combination of gray-level co-occurrence information and normalization information of the video segment to be encoded can be input into the first bitrate factor prediction model. The first bitrate factor prediction model is trained to determine the predicted bitrate factor of each video segment to be encoded under multiple preset video coding standards based on the spatiotemporal complexity information.

[0064] According to some other embodiments of this disclosure, when the spatiotemporal domain feature information only includes coding feature information, the feature information of the combination of intra-frame coding information and inter-frame coding information of the video segment to be encoded can be input into a first bitrate factor prediction model. The first bitrate factor prediction model is trained to determine the predicted bitrate factor of each video segment to be encoded under multiple preset video coding standards based on the coding feature information.

[0065] According to further embodiments of this disclosure, when the spatiotemporal domain feature information includes both spatiotemporal complexity information and coding feature information, the spatiotemporal complexity information and coding feature information can be combined to generate feature information represented by a set of feature vectors. Continuing with the example above where the spatiotemporal complexity information is a vector containing 70 feature values ​​and the coding feature information is a vector containing 300 feature values, combining these two vectors will yield a feature vector containing 370 feature values, thereby more completely representing the spatiotemporal characteristics of the video segment to be encoded. In this case, the generated set of feature vectors can be input into a first bitrate factor prediction model, wherein the first bitrate factor prediction model is trained to determine the predicted bitrate factor of each video segment to be encoded under multiple preset video coding standards based on both the spatiotemporal complexity information and the coding feature information.

[0066] By utilizing a trained first bitrate factor prediction model to predict the predicted bitrate factor for each video segment to be encoded under multiple preset video coding standards, the bitrate factor that meets quality requirements (e.g., a high video coding score for the encoded video segment) can be directly determined without having to perform multiple encoding and search operations each time, thus saving significant time costs. Furthermore, with sufficient sample data, the accuracy of the predicted bitrate factor can be ensured to a certain extent.

[0067] Figure 2 A flowchart of a method 200 for training a first code rate factor prediction model according to some embodiments of the present disclosure is shown. Figure 2 As shown, the first bitrate factor prediction model can be trained through the following steps of method 200: Step S210: For each of the multiple preset video coding standards, obtain the sample spatiotemporal feature information of the sample video segment and the sample bitrate factor of the sample video segment under the preset video coding standard; Step S220: Input the sample spatiotemporal feature information into the first bitrate factor prediction model to determine the sample predicted bitrate factor of the sample video segment under the preset video coding standard through the first bitrate factor prediction model; Step S230: Calculate the first model loss value based on the sample bitrate factor and the sample predicted bitrate factor; and Step S240: Adjust the parameters of the first bitrate factor prediction model based on the first model loss value under each preset video coding standard until the first model training stopping condition is reached.

[0068] By pre-training and adjusting the first rate factor prediction model using sample data, the accuracy of the predicted rate factor can be improved.

[0069] According to some embodiments, in step S210, sample video segments can be obtained using a method similar to that used in step S110 to obtain the video segment to be encoded, and the spatiotemporal feature information of the samples can be calculated using a method similar to that used in step S120 to calculate the spatiotemporal feature information of the video segment to be encoded. Therefore, this will not be elaborated further. To ensure the accuracy of the first bitrate factor prediction model after training, more sample video segments can be obtained while taking into account computational resources and time costs. The number of sample video segments can be, for example, at least 50,000 or at least 100,000.

[0070] According to some embodiments, in step S210, the sample bitrate factor can be the true bitrate factor, meaning that the quality of the encoded video obtained after encoding the sample video segment using this bitrate factor meets the target quality requirements. The quality of the encoded video can be evaluated, for example, by the video coding score (also known as the Video Multimethod Assessment Fusion, VMAF). The larger the VMAF value, the higher the quality of the encoded video; conversely, the smaller the VMAF value, the lower the quality of the encoded video. The VMAF value typically ranges from 0 to 100. The true bitrate factor can be obtained through the following steps: For the video coding standard, select an initial sample bitrate factor and a preset sample video coding score range; based on the initial sample bitrate factor, encode the sample video clip using the corresponding encoder in the real transcoding system (e.g., H.264 encoder, H.265 encoder, H.266 encoder, AV1 encoder, etc.) to obtain a sample pre-coded video clip under the video coding standard; compare the sample video coding score of the obtained sample pre-coded video clip with the preset sample video coding score range; if the sample video coding score is within the preset range... If the initial sample bitrate factor falls within the preset range of sample video coding scores (i.e., the encoded video meets the target quality requirements), it indicates that the initial sample bitrate factor under that video coding standard is feasible and can be used as the true bitrate factor. If the sample video coding score falls outside the preset range of sample video coding scores (i.e., the encoded video does not meet the target quality requirements), it indicates that the initial sample bitrate factor under that video coding standard is not feasible. Search methods such as binary search can be used to adjust the initial sample bitrate factor until the sample video coding score falls within the preset range of sample video coding scores. Typically, the bitrate factor differs for the same sample video segment under different video coding standards. Therefore, the above steps will be repeated for each video coding standard to search for and obtain the sample bitrate factors (true bitrate factors) for sample video segments under multiple video coding standards.

[0071] In the above embodiments, the preset sample video coding score interval can be, for example, a numerical range based on VMAF and customized by the user according to actual needs (e.g., [91, 93], [90, 95]), or it can be an interval set about each VMAF integer value to adapt the first bitrate factor prediction model to multiple target quality requirements (e.g., [91.5, 92.5], [92.5, 93.5]...[95.5, 96.5], etc.). The scope of the subject matter claimed in this disclosure is not limited in this respect.

[0072] According to some embodiments, in step S220, in the initial stage, initial parameters of the first bitrate factor prediction model can be selected for multiple video coding standards, and based on these initial parameters, multiple sample prediction bitrate factors of sample video segments under the multiple video coding standards can be determined using the obtained sample spatiotemporal domain feature information.

[0073] According to some embodiments, in step S230, for each preset video coding standard, the model loss value can be calculated, for example, via a loss function based on the corresponding sample prediction bitrate factor and sample prediction bitrate factor. Examples of loss functions include, but are not limited to, cross-entropy loss function, maximum loss function, average loss function, 0-1 loss function, etc. It will be understood that other suitable methods can also be used to calculate the model loss value in this disclosure, and the scope of the subject matter claimed in this disclosure is not limited in this respect.

[0074] According to some embodiments, in step S240, after calculating the model loss value under each preset video coding standard, these model loss values ​​can be backpropagated, and the parameters of the first bitrate factor prediction model can be adjusted. Then, steps S220-S240 are repeated until the model training stopping condition is reached. Specifically, the model training stopping condition may be that the model loss value is lower than a preset threshold, and / or the number of training epochs has reached a preset number of epochs. It will be understood that this disclosure does not limit the model training stopping condition, and it can be adjusted according to actual needs.

[0075] Figure 3 A structural block diagram of a first rate factor prediction model 300 according to some embodiments of the present disclosure is shown. Figure 3 The first bitrate factor prediction model 300 may include a batch normalization module 310, a first attention mechanism module 320, a residual module 330, a second attention mechanism module 340, and a fully connected module 350. The first bitrate factor prediction model 300 takes the spatiotemporal features of the video segment to be encoded as input and outputs the predicted bitrate factor of the video segment to be encoded under different video coding standards.

[0076] The batch normalization module 310 normalizes the original input features to have a Gaussian distribution with a mean of 0 and a variance of 1, thereby eliminating the influence of differences in the units of measurement between features. The normalized features are then input to the first attention mechanism module 320, which calculates a weight for each feature, assigning larger weights to features with higher discriminative power and smaller weights to features with lower information content, thus automatically filtering features and improving the accuracy of the prediction results. The filtered features are then input to the residual module 330, which may include multiple convolutional layers that transform the filtered features as input. Unchanged filtered features are skipped from these convolutional layers and passed to subsequent layers for overall processing to obtain the output of the residual module, thereby reducing information loss. The output of the residual module 330 can be further input to the second attention mechanism module 340 for further feature filtering, thereby further improving the accuracy of the prediction results. Finally, the input features can be aggregated via the fully connected module 350 to generate and output the prediction code rate factor.

[0077] Figure 4 A flowchart illustrating the determination of multiple target bitrate factors for a video segment to be encoded under multiple preset video coding standards, according to some embodiments of the present disclosure, is shown. Figure 4 As shown, step S140, determining multiple target bitrate factors for a video segment to be encoded under multiple preset video coding standards based on the first predicted bitrate factor corresponding to the first preset video coding standard and the preset video coding score range among multiple predicted bitrate factors, may include: step S442, performing a first encoding on the video segment to be encoded based on the first predicted bitrate factor to obtain a first pre-encoded video segment under the first preset video coding standard; step S444, determining whether the video coding score of the first pre-encoded video segment is within the preset video coding score range; and step S446, in response to determining that the video coding score of the first pre-encoded video segment is within the preset video coding score range: determining multiple predicted bitrate factors as multiple target bitrate factors.

[0078] In this disclosure, the first preset video coding standard refers to the video coding standard that takes the shortest time to perform video coding. For example, among the video coding standards H.264, H.265, H.266, and AV1, H.264 is the first preset video coding standard because it takes the shortest time to perform video coding.

[0079] In step S442, in addition to the first predicted bitrate factor, the encoding parameters used when encoding the video to be encoded may include, but are not limited to, preset, GOP (Group of Pictures), size, etc. Since the bitrate factor plays an important role in the quality of the encoded video, when encoding the video using the method of this disclosure, the values ​​of these encoding parameters can be set to fixed constants (e.g., default preset values).

[0080] In step S444, similar to the setting of the preset sample video encoding score range, the preset video encoding score range can also be a numerical range that the user can customize according to actual needs, so it will not be described in detail here.

[0081] Since the video coding standard with the shortest encoding time has the lowest computational complexity, its bitrate factor can, to some extent, reflect the accuracy of the bitrate factors under other video coding standards with longer encoding times. In other words, if the video coding score of a video segment encoded based on the first predicted bitrate factor corresponding to the first preset video coding standard (e.g., H.264 in the example above) is within the preset video coding score range (meeting video quality requirements), it indicates that the accuracy of other predicted bitrate factors corresponding to other preset video coding standards (e.g., H.265, H.266, and AV1 in the example above) is also high (e.g., if the video coding score of an encoded video under the H.264 standard has a 99% probability of being within the preset video coding score range, then the video coding scores of encoded videos under the H.265, H.266, and AV1 standards also have a greater than 95% probability of being within the preset video coding score range). The video coding scores of video clips encoded using these predicted bitrate factors are usually within a preset video coding score range (meeting video quality requirements). This allows for the correlation of different video coding standards, which helps reduce the computational load required for predicting bitrate factors. In some examples, data from multiple experiments show that when the predicted bitrate factor under the H.264 video coding standard meets the video quality requirements, the probability that the predicted bitrate factors under the H.265, H.266, and AV1 standards will also meet the requirements is 95%. Therefore, these predicted bitrate factors can be directly used as the target bitrate factors for encoding the video to be encoded.

[0082] Continue to refer to Figure 4According to some other embodiments of this disclosure, step S140, determining multiple target bitrate factors of the video segment to be encoded under multiple preset video coding standards based on the first predicted bitrate factor of the first preset video coding standard and the preset video coding score range among multiple predicted bitrate factors, may further include: step S448, in response to determining that the video coding score of the first pre-coded video segment is not within the preset video coding score range: updating multiple predicted bitrate factors to obtain multiple target bitrate factors based on the spatiotemporal domain feature information of the video segment to be encoded, multiple predicted bitrate factors and the video coding score of the first pre-coded video segment.

[0083] Continuing with the example of multiple preset video coding standards H.264, H.265, H.266, and AV1, when the video coding score of the pre-coded video segment obtained based on the predicted bitrate factor under the H.264 video coding standard is not within the preset video coding score range, the predicted bitrate factors under the H.265, H.266, and AV1 video coding standards and the video coding score of the pre-coded video segment under the H.264 video coding standard can be used as feedback information. Together with the spatiotemporal feature information of the video segment to be encoded, the bitrate factors under these video coding standards are predicted a second time to obtain multiple target bitrate factors.

[0084] Extensive experimental data shows that after the first prediction and encoding of the video segment using the predicted bitrate factor, the probability of the video coding score meeting the video quality requirements is 40%. Therefore, 60% of the video segments to be encoded will undergo the aforementioned second prediction and bitrate factor update operation. When the video coding score of the first pre-coded video segment is not within the preset video coding score range, performing a second prediction of the bitrate factor under different video coding standards can further improve the accuracy of the target bitrate factor, avoiding large errors in the bitrate factor prediction for the aforementioned 60% of the video segments to be encoded, which could lead to unstable and unreliable image quality.

[0085] Furthermore, the update of the predicted bitrate factor obtained from the first prediction is based on the video coding score of the first pre-coded video segment, thus eliminating the need to encode the video segments to be encoded separately using the predicted bitrate factors under other preset video coding standards. This will greatly save the time cost and computing resources required for encoding.

[0086] According to some embodiments of this disclosure, step S448, updating multiple predicted bitrate factors to obtain multiple target bitrate factors based on the spatiotemporal feature information of the video segment to be encoded, multiple predicted bitrate factors, and the video coding score of the first pre-coded video segment, may include: inputting the spatiotemporal feature information of the video segment to be encoded, multiple predicted bitrate factors, and the video coding score of the first pre-coded video segment into a second bitrate factor prediction model, so as to determine multiple target bitrate factors of the video segment to be encoded under multiple preset video coding standards via the second bitrate factor prediction model.

[0087] According to some embodiments of this disclosure, similar to the training method of the first bitrate factor prediction model, the second bitrate factor prediction model can be trained by the following operations: inputting sample spatiotemporal feature information, sample predicted bitrate factor under each preset video coding standard, and the first sample video coding score corresponding to the sample predicted bitrate factor under the first preset video coding standard into the second bitrate factor prediction model, so as to determine multiple sample target bitrate factors of sample video segments under multiple preset video coding standards through the second bitrate factor prediction model; for each preset video coding standard among the multiple preset video coding standards, calculating the second model loss value based on the sample bitrate factor and the sample target bitrate factor corresponding to the preset video coding standard; and adjusting the parameters of the second bitrate factor prediction model based on the second model loss value until the second model training stopping condition is reached.

[0088] It will be understood that, apart from adding the sample prediction bitrate factor and the first sample video coding score under each preset video coding standard as inputs to the second bitrate factor prediction model, the steps in the above method for training the second bitrate factor prediction model are similar to those in the method 200 for training the first bitrate factor prediction model, and therefore will not be repeated.

[0089] Accordingly, compared with the first bitrate factor prediction model, the second bitrate factor prediction model has the same structural framework as the first bitrate factor prediction model, except that it adds the sample predicted bitrate factor and the first sample video coding score under each preset video coding standard as input. Therefore, it will not be described again.

[0090] It will also be understood that the training of the second rate factor prediction model depends on the training results of the first rate factor prediction model. Therefore, the training of the second rate factor prediction model is performed after the training of the first rate factor prediction model is completed.

[0091] After determining multiple target bitrate factors, the corresponding encoders in the transcoding system (based on the corresponding preset video coding standards) are used to encode the video segments to be encoded according to these multiple bitrate factors. Data from multiple experiments shows that after encoding the video segments using the target bitrate factors obtained from the second prediction, the video coding score of the encoded video segments has a 99% probability of falling within the preset video coding score range. Therefore, the encoding results of video segments based on the target bitrate factors obtained from the second prediction can be directly trusted.

[0092] After encoding each video segment to be encoded using multiple target bitrate factors, method 100 may further include: for each of the multiple preset video encoding standards, obtaining all target video segments under that preset video encoding standard; and combining all target video segments based on the order of one or more sub-video segments in the video to be encoded to obtain the target video corresponding to the video to be encoded under that preset video encoding standard.

[0093] After performing the above encoding operations on each video segment in a video to be encoded, multiple target video segments can be obtained for each video segment to be encoded. Then, for each preset video encoding standard, the corresponding target video segments from the multiple target video segments for each video segment to be encoded can be written into the final video bitstream file, and the written target video segments are concatenated according to the order of each video segment to be encoded in the video to be encoded to obtain the target video corresponding to the video to be encoded under the preset video encoding standard.

[0094] As mentioned above, each video segment to be encoded is a single-shot video clip with similar and coherent content. Therefore, encoding each video segment to be encoded and obtaining the corresponding target video segment ensures the quality of each encoded video segment, thereby ensuring a stable picture quality experience in various scenarios. Furthermore, by splicing the obtained target video segments based on the order of these video segments in the original video, a complete video file can be reconstructed, avoiding video file discontinuities caused by splicing errors in the target video segments.

[0095] Figure 5 A structural block diagram of a video encoding apparatus 500 according to an embodiment of the present disclosure is shown. Figure 5As shown, the apparatus 500 may include: an acquisition module 510 configured to acquire a video segment to be encoded, the video segment to be encoded including one or more video frames; a calculation module 520 configured to calculate spatiotemporal feature information of the video segment to be encoded based on the one or more video frames; a prediction bitrate factor determination module 530 configured to determine multiple prediction bitrate factors of the video segment to be encoded under multiple preset video coding standards based on the spatiotemporal feature information; a target bitrate factor determination module 540 configured to determine multiple target bitrate factors of the video segment to be encoded under multiple preset video coding standards based on a first prediction bitrate factor corresponding to a first preset video coding standard among the multiple preset video coding standards and a preset video coding score range, wherein, among the multiple preset video coding standards, the video encoding time using the first preset video coding standard is the shortest; and an encoding module 550 configured to encode the video segment to be encoded according to the multiple target bitrate factors respectively to obtain multiple target video segments.

[0096] According to some embodiments of this disclosure, the spatiotemporal domain feature information may include spatiotemporal complexity information, and wherein the calculation module 530 may include: a module configured to calculate gray-level co-occurrence information and normalized information of the video segment to be encoded as spatiotemporal complexity information based on one or more video frames, wherein the gray-level co-occurrence information is used to describe the spatial complexity of the video segment to be encoded, and the normalized information is used to describe the temporal complexity of the video segment to be encoded; and a module configured to combine the gray-level co-occurrence information and the normalized information to generate the spatiotemporal complexity information of the video segment to be encoded.

[0097] According to some embodiments of this disclosure, the spatiotemporal domain feature information may include coded feature information, and wherein the calculation module 530 may include: a preprocessing module configured to preprocess one or more video frames to generate a new video frame sequence, the new video frame sequence including a set of video frame pairs; and an intra-frame and inter-frame coding module configured to perform intra-frame coding and inter-frame coding for each video frame pair in the set of video frame pairs to obtain coded feature information of the video segment to be coded.

[0098] According to some embodiments of this disclosure, the preprocessing module may include: a module configured to copy video frames other than the first and last frames in one or more video frames to generate one or more copied video frames; and a module configured to sort the one or more video frames and the one or more copied video frames such that each of the one or more copied video frames is located after the corresponding video frame.

[0099] According to some embodiments of this disclosure, the intra-frame and inter-frame coding module may include: a module configured to perform intra-frame coding on a first video frame in each video frame pair in the video frame pair set to obtain intra-frame coding information; a module configured to perform inter-frame coding on each video frame pair in the video frame pair set to obtain inter-frame coding information; and a module configured to obtain coding feature information based on the intra-frame coding information and the inter-frame coding information.

[0100] According to some embodiments of this disclosure, the coding feature information may be based on one or more of the following: the number of coded bits, the proportion of intra-frame prediction modes, and the inter-frame motion vector amplitude distribution information.

[0101] According to some embodiments of this disclosure, the prediction bitrate factor determination module 530 may include: a module configured to input spatiotemporal feature information of the video segment to be encoded into a first bitrate factor prediction model, so as to determine multiple prediction bitrate factors of the video segment to be encoded under multiple preset video coding standards via the first bitrate factor prediction model.

[0102] According to some embodiments of this disclosure, the target bitrate factor determination module 540 may include: a module configured to perform a first encoding of the video segment to be encoded based on a first predicted bitrate factor to obtain a first pre-encoded video segment under a first preset video encoding standard; a module configured to determine whether the video encoding score of the first pre-encoded video segment is within a preset video encoding score range; and a module configured to determine a plurality of predicted bitrate factors as a plurality of target bitrate factors in response to determining that the video encoding score of the first pre-encoded video segment is within the preset video encoding score range.

[0103] According to some embodiments of this disclosure, the target bitrate factor determination module 540 may further include: an update module configured to, in response to determining that the video coding score of the first pre-coded video segment is not within a preset video coding score range, update multiple predicted bitrate factors based on the spatiotemporal feature information of the video segment to be encoded, multiple predicted bitrate factors, and the video coding score of the first pre-coded video segment to obtain multiple target bitrate factors.

[0104] According to some embodiments of this disclosure, the updating module may include: a module configured to input the spatiotemporal domain feature information of the video segment to be encoded, multiple predicted bitrate factors and the video coding score of the first pre-coded video segment into a second bitrate factor prediction model, so as to determine multiple target bitrate factors of the video segment to be encoded under multiple preset video coding standards via the second bitrate factor prediction model.

[0105] According to some embodiments of this disclosure, the first bitrate factor prediction model can be trained by the following operations: for each of a plurality of preset video coding standards: acquiring sample spatiotemporal feature information of sample video segments and sample bitrate factor of sample video segments under the preset video coding standard; inputting the sample spatiotemporal feature information into the first bitrate factor prediction model to determine the sample predicted bitrate factor of sample video segments under the preset video coding standard via the first bitrate factor prediction model; calculating a first model loss value based on the sample bitrate factor and the sample predicted bitrate factor; and adjusting the parameters of the first bitrate factor prediction model based on the first model loss value under each preset video coding standard until the first model training stopping condition is met.

[0106] According to some embodiments of this disclosure, the second bitrate factor prediction model is trained by the following operations: inputting sample spatiotemporal feature information, sample predicted bitrate factor under each preset video coding standard, and a first sample video coding score corresponding to the sample predicted bitrate factor under the first preset video coding standard into the second bitrate factor prediction model, so as to determine multiple sample target bitrate factors of sample video segments under multiple preset video coding standards via the second bitrate factor prediction model; for each of the multiple preset video coding standards, calculating a second model loss value based on the sample bitrate factor and the sample target bitrate factor corresponding to that preset video coding standard; and adjusting the parameters of the second bitrate factor prediction model based on the second model loss value under each preset video coding standard until the second model training stopping condition is met.

[0107] According to some embodiments of this disclosure, the acquisition module 510 may include: a module configured to acquire a video to be encoded; a module configured to perform scene detection on the video to be encoded; a module configured to divide the video to be encoded into one or more sub-video segments based on the scene detection results; and a module configured to identify each video sub-segment in one or more video segments as a video segment to be encoded.

[0108] According to some embodiments of this disclosure, the apparatus 500 may further include: a module configured to acquire all target video segments under each of a plurality of preset video coding standards; and a module configured to combine all target video segments based on the order of one or more sub-video segments in the video to be encoded to obtain a target video corresponding to the video to be encoded under the preset video coding standard.

[0109] It should be understood that Figure 5 The various modules 510-550 of the device 500 shown can be used with reference to Figure 1The steps S110-S150 in the described method 100 correspond to each other. Therefore, the operations, features, and advantages described above for method 100 also apply to device 500 and its included modules. For the sake of brevity, some operations, features, and advantages will not be repeated here.

[0110] It should also be understood that this article can describe various technologies in the general context of software and hardware components or program modules. The above regarding... Figure 5 The various modules described can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuit. For example, in some embodiments, one or more of the acquisition module 510, calculation module 520, prediction rate factor determination module 530, target rate factor determination module 540, and encoding module 550 can be implemented together in a System on Chip (SoC). The SoC may include an integrated circuit chip (which includes a processor (e.g., a Central Processing Unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or one or more components of other circuitry) and may optionally execute received program code and / or include embedded firmware to perform functions.

[0111] According to another aspect of this disclosure, an electronic device is also provided, comprising: at least one processor; and at least one memory communicatively connected to the at least one processor; wherein the at least one memory stores a computer program that, when executed by the at least one processor, implements the video encoding method described above.

[0112] According to another aspect of this disclosure, a non-transitory computer-readable storage medium storing a computer program is also provided, wherein the computer program implements the above-described video encoding method when executed by a processor.

[0113] According to another aspect of this disclosure, a computer program product is also provided, comprising a computer program, wherein the computer program implements the above-described video encoding method when executed by a processor.

[0114] See Figure 6The present invention describes a structural block diagram of an electronic device 600 that can serve as a server of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device can be different types of computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0115] like Figure 6 As shown, the electronic device 600 may include at least one processor 610, working memory 620, input unit 640, display unit 650, speaker 660, storage unit 670, communication unit 680 and other output units 690 that are capable of communicating with each other via system bus 630.

[0116] Processor 610 may be a single processing unit or multiple processing units, and all processing units may include single or multiple computing units or multiple cores. Processor 610 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. Processor 610 may be configured to acquire and execute computer-readable instructions stored in working memory 620, storage unit 670, or other computer-readable media, such as program code of operating system 620a, program code of application program 620b, etc.

[0117] Working memory 620 and storage cell 670 are examples of computer-readable storage media for storing instructions that are executed by processor 610 to perform the various functions described above. Working memory 620 may include both volatile and non-volatile memory (e.g., RAM, ROM, etc.). Furthermore, storage cell 670 may include hard disk drives, solid-state drives, removable media including external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CDs, DVDs), storage arrays, network-attached storage, storage area networks, etc. Working memory 620 and storage cell 670 may be collectively referred to herein as memory or computer-readable storage media, and may be non-transitory media capable of storing computer-readable, processor-executable program instructions as computer program code that can be executed by processor 610 as a specific machine configured to perform the operations and functions described in the examples herein.

[0118] The input unit 660 can be any type of device capable of inputting information to the electronic device 600. The input unit 660 can receive input numerical or character information and generate key signal inputs related to user settings and / or function control of the electronic device. It can include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. The output unit can be any type of device capable of presenting information and can include, but is not limited to, a display unit 650, a speaker 660, and other output units 690. Other output units 690 can include, but are not limited to, video / audio output terminals, vibrators, and / or printers. The communication unit 680 allows the electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks. It can include, but is not limited to, a modem, network card, infrared communication device, wireless communication transceiver, and / or chipset, such as Bluetooth. TM Equipment, 802.11 equipment, Wi-Fi equipment, WiMAX equipment, cellular communication equipment and / or the like.

[0119] The application program 620b in working register 620 can be loaded to execute the various methods and steps described above, for example... Figure 1 Steps S110-S150 in the process Figure 2 Steps S210-S240 in the process Figure 4 Steps S442-S448 in the above description. For example, in some embodiments, the various methods described above can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 670. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 600 via storage unit 670 and / or communication unit 680. When the computer program is loaded and executed by processor 610, one or more steps of methods 100, 200 described above can be performed. Alternatively, in other embodiments, processor 610 can be configured to perform methods 100, 200 by any other suitable means (e.g., by means of firmware).

[0120] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0121] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0122] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0123] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0124] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0125] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0126] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0127] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A video encoding method, comprising: Acquire a video segment to be encoded, the video segment to be encoded comprising one or more video frames; Calculate the spatiotemporal feature information of the video segment to be encoded based on the one or more video frames; Based on the aforementioned spatiotemporal feature information, multiple prediction bitrate factors are determined for the video segment to be encoded under multiple preset video coding standards; Based on the first predicted bitrate factor and the preset video coding score range of the first preset video coding standard in the plurality of preset video coding standards, a plurality of target bitrate factors of the video segment to be encoded under the plurality of preset video coding standards are determined, wherein the video coding time using the first preset video coding standard is the shortest among the plurality of preset video coding standards. as well as The video segments to be encoded are encoded according to the multiple target bitrate factors to obtain multiple target video segments.

2. The method according to claim 1, wherein, The spatiotemporal domain feature information includes spatiotemporal complexity information, and wherein calculating the spatiotemporal domain feature information of the video segment to be encoded based on the one or more video frames includes: The gray-level co-occurrence information and normalized information of the video segment to be encoded are calculated based on the one or more video frames as the spatiotemporal complexity information, wherein the gray-level co-occurrence information is used to describe the spatial complexity of the video segment to be encoded, and the normalized information is used to describe the temporal complexity of the video segment to be encoded; and The grayscale co-occurrence information and the normalized information are combined to generate the spatiotemporal complexity information of the video segment to be encoded.

3. The method according to claim 1 or 2, wherein, The spatiotemporal feature information includes encoded feature information, and wherein calculating the spatiotemporal feature information of the video segment to be encoded based on the one or more video frames includes: The one or more video frames are preprocessed to generate a new video frame sequence, the new video frame sequence comprising a set of video frame pairs; and For each video frame pair in the set of video frame pairs, intra-frame coding and inter-frame coding are performed respectively to obtain the coding feature information of the video segment to be encoded.

4. The method according to claim 3, wherein, Preprocessing the one or more video frames to generate a new video frame sequence includes: Copying video frames, excluding the first and last frames, from the one or more video frames to generate one or more copied video frames; and The one or more video frames and the one or more copied video frames are sorted such that each of the one or more copied video frames is placed after the corresponding video frame.

5. The method according to claim 3, wherein, For each video frame pair in the set of video frame pairs, intra-frame coding and inter-frame coding are performed respectively to obtain the coding feature information of the video segment to be encoded, including: Intra-frame encoding is performed on the first video frame of each video frame pair in the video frame pair set to obtain intra-frame encoding information. Inter-frame coding is performed on each video frame pair in the set of video frame pairs to obtain inter-frame coding information; and The coding feature information is obtained based on the intra-frame coding information and the inter-frame coding information.

6. The method according to claim 3, wherein, The encoded feature information is based on one or more of the following: number of encoded bits, proportion of intra-frame prediction mode, and inter-frame motion vector amplitude distribution information.

7. The method according to claim 1 or 2, wherein, Based on the aforementioned spatiotemporal feature information, multiple prediction bitrate factors for the video segment to be encoded under multiple preset video coding standards are determined, including: The spatiotemporal feature information of the video segment to be encoded is input into the first bitrate factor prediction model, so as to determine the multiple predicted bitrate factors of the video segment to be encoded under the multiple preset video coding standards via the first bitrate factor prediction model.

8. The method according to claim 7, wherein, Based on the first predicted bitrate factor corresponding to the first preset video coding standard among the plurality of predicted bitrate factors and the preset video coding score range, the plurality of target bitrate factors for the video segment to be encoded under the plurality of preset video coding standards are determined as follows: The video segment to be encoded is encoded for the first time based on the first predicted bitrate factor to obtain the first pre-encoded video segment under the first preset video encoding standard. Determine whether the video encoding score of the first pre-encoded video segment is within the preset video encoding score range; and In response to determining that the video coding score of the first precoded video segment is within the preset video coding score range: The plurality of predicted bitrate factors are determined as the plurality of target bitrate factors.

9. The method according to claim 8, wherein, Based on the first predicted bitrate factor corresponding to the first preset video coding standard among the plurality of predicted bitrate factors and the preset video coding score range, determining the plurality of target bitrate factors for the video segment to be encoded under the plurality of preset video coding standards further includes: In response to determining that the video coding score of the first precoded video segment is not within the preset video coding score range: Based on the spatiotemporal feature information of the video segment to be encoded, the plurality of predicted bitrate factors, and the video coding score of the first precoded video segment, the plurality of predicted bitrate factors are updated to obtain the plurality of target bitrate factors.

10. The method according to claim 9, wherein, Based on the spatiotemporal feature information of the video segment to be encoded, the plurality of predicted bitrate factors, and the video coding score of the first pre-coded video segment, updating the plurality of predicted bitrate factors to obtain the plurality of target bitrate factors includes: The spatiotemporal feature information of the video segment to be encoded, the plurality of predicted bitrate factors, and the video coding score of the first pre-coded video segment are input into the second bitrate factor prediction model, so as to determine the plurality of target bitrate factors of the video segment to be encoded under the plurality of preset video coding standards through the second bitrate factor prediction model.

11. The method according to claim 10, wherein, The first rate factor prediction model is trained through the following operations: For each of the plurality of preset video encoding standards: Obtain the spatiotemporal feature information of the sample video segment and the sample bitrate factor of the sample video segment under the preset video coding standard; The sample spatiotemporal feature information is input into the first bitrate factor prediction model to determine the sample predicted bitrate factor of the sample video segment under the preset video coding standard via the first bitrate factor prediction model. as well as The first model loss value is calculated based on the sample bitrate factor and the sample predicted bitrate factor. as well as The parameters of the first bitrate factor prediction model are adjusted based on the first model loss value under each preset video coding standard until the first model training stopping condition is met.

12. The method according to claim 11, wherein, The second rate factor prediction model is trained through the following operations: The sample spatiotemporal feature information, the sample predicted bitrate factor under each preset video coding standard, and the first sample video coding score corresponding to the sample predicted bitrate factor under the first preset video coding standard are input into the second bitrate factor prediction model, so as to determine multiple sample target bitrate factors of the sample video segment under the multiple preset video coding standards through the second bitrate factor prediction model. For each of the plurality of preset video coding standards, a second model loss value is calculated based on the sample bitrate factor and the sample target bitrate factor corresponding to that preset video coding standard; as well as The parameters of the second bitrate factor prediction model are adjusted based on the second model loss value under each preset video coding standard until the training stopping condition of the second model is met.

13. The method according to claim 1, wherein, Obtaining the video segment to be encoded includes: Obtain the video to be encoded; Scene detection is performed on the video to be encoded; Based on scene detection results, the video to be encoded is divided into one or more sub-video segments; and Each video sub-segment in the one or more video segments is identified as the video segment to be encoded.

14. The method of claim 13, further comprising: For each of the plurality of preset video encoding standards, obtain all target video segments under that preset video encoding standard; as well as Based on the order of the one or more sub-video segments in the video to be encoded, all the target video segments are combined to obtain the target video corresponding to the video to be encoded under the preset video encoding standard.

15. A video encoding apparatus, comprising: The acquisition module is configured to acquire a video segment to be encoded, the video segment to be encoded including one or more video frames; The calculation module is configured to calculate the spatiotemporal feature information of the video segment to be encoded based on the one or more video frames; The module for determining the prediction bitrate factor is configured to determine multiple prediction bitrate factors of the video segment to be encoded under multiple preset video coding standards based on the spatiotemporal domain feature information. The target bitrate factor determination module is configured to determine multiple target bitrate factors of the video segment to be encoded under the multiple preset video coding standards based on the first predicted bitrate factor of the first preset video coding standard in the multiple preset video coding standards and the preset video coding score range among the multiple predicted bitrate factors, wherein the video coding time using the first preset video coding standard is the shortest among the multiple preset video coding standards. as well as The encoding module is configured to encode the video segments to be encoded according to the plurality of target bitrate factors to obtain a plurality of target video segments.

16. An electronic device comprising: At least one processor; as well as At least one memory communicatively connected to the at least one processor The at least one memory stores a computer program that, when executed by the at least one processor, implements the method as described in any one of claims 1-14.

17. A non-transitory computer-readable storage medium storing a computer program, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-14.

18. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-14.

Images