Video encoding method and device, electronic device, and storage medium

Abstract

The present disclosure provides a video encoding method and apparatus, an electronic device, and a storage medium, the method including: obtaining a video segment to be encoded, including one or more video frames; calculating spatio-temporal domain feature information of the video segment to be encoded based on the one or more video frames; determining a plurality of predicted bit rate coefficients for the video segment to be encoded in a plurality of preset video encoding standards based on the spatio-temporal domain feature information; determining a plurality of target bit rate coefficients for the video segment to be encoded in a plurality of preset video encoding standards based on a first predicted bit rate coefficient corresponding to a first preset video encoding standard among the plurality of preset video encoding standards and a preset video encoding score interval, where among the plurality of preset video encoding standards, a time required to encode the video using the first preset video encoding standard is shortest; and encoding the video segment to be encoded based on the plurality of target bit rate coefficients to obtain a plurality of target video segments.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This disclosure claims priority to Chinese Patent Application No. 202211690147.9, filed on December 27, 2022, the entire contents of which are incorporated herein by reference in their entirety.

[0002] The present disclosure relates to the technical field of computers, and in particular to video encoding methods and apparatus, electronic devices, computer-readable storage media, and computer program products. [Background technology]

[0003] With the development of Internet technology, video platforms are also developing rapidly, with more and more users sharing and watching videos. As the number of users on the platform continues to grow, the bandwidth costs for video transmission also increase. To reduce these costs, a transcoding system with better compression performance is required, which can ensure video quality while significantly reducing the size of the compressed video and reducing the network traffic required for transmission.

[0004] The methods described in this section are not necessarily methods previously contemplated or employed. Unless otherwise expressly stated, it should not be admitted that any method described in this section is prior art solely by reason of its inclusion in this section. Likewise, unless otherwise expressly stated, it should not be assumed that any problems addressed in this section have been recognized in any prior art. Summary of the Invention

[0005] The present disclosure provides a video encoding method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0006] According to one aspect of the present disclosure, there is provided a video encoding method, the method comprising: obtaining a video segment to be encoded, the video segment including one or more video frames; calculating spatio-temporal domain feature information of the video segment to be encoded based on the one or more video frames; determining a plurality of predicted bit rate coefficients for the video segment to be encoded in a plurality of preset video encoding standards based on the spatio-temporal domain feature information; determining a plurality of target bit rate coefficients for the video segment to be encoded in the plurality of preset video encoding standards based on a first predicted bit rate coefficient corresponding to a first preset video encoding standard among the plurality of preset video encoding standards and a preset video encoding score interval, where among the plurality of preset video encoding standards, a time required to encode the video using the first preset video encoding standard is shortest; and encoding the video segment to be encoded based on the plurality of target bit rate coefficients to obtain a plurality of target video segments.

[0007]

[0013] According to another aspect of the present disclosure, there is further provided a video encoding device, including: an acquiring module configured to acquire a video segment to be encoded including one or more video frames; a calculating module configured to calculate, based on the one or more video frames, spatio-temporal domain feature information of the video segment to be encoded; a predicted bit rate coefficient determination module configured to determine, based on the spatio-temporal domain feature information, a plurality of predicted bit rate coefficients for the video segment to be encoded in a plurality of preset video encoding standards; a target bit rate coefficient determination module configured to determine, based on the plurality of predicted bit rate coefficients, a first predicted bit rate coefficient corresponding to a first preset video encoding standard among the plurality of preset video encoding standards and a preset video encoding score interval, wherein the target bit rate coefficient determination module is configured to determine a plurality of target bit rate coefficients for the video segment to be encoded in a plurality of preset video encoding standards,

[0008] According to another aspect of the present disclosure, there is further provided an electronic device including at least one processor and at least one memory communicatively connected to the at least one processor, wherein the at least one memory stores a computer program that, when executed by the at least one processor, implements the above-described video encoding method.

[0009] According to another aspect of the present disclosure, there is further provided a non-transitory computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the above video encoding method.

[0010] According to another aspect of the present disclosure, there is further provided a computer program product including a computer program which, when executed by a processor, implements the above video encoding method.

[0011] According to one or more embodiments of the present disclosure, a first prediction is made for bitrate coefficients for multiple preset video encoding standards based on spatio-temporal domain feature information of a video segment to be encoded, and a second prediction is made for bitrate coefficients for multiple preset video encoding standards based on the predicted bitrate coefficients and a preset video encoding score, thereby improving the accuracy of the bitrate coefficients, effectively improving the accuracy of video encoding, and ensuring video quality. In addition, since the bitrate coefficients for multiple preset video encoding standards can be predicted simultaneously, the encoding system can simultaneously output multiple coded videos that meet video quality requirements despite being of different video encoding standards, thereby avoiding repeated prediction operations for multiple video encoding standards and saving time costs. Furthermore, the second prediction of the bit rate coefficient is performed based on the bit rate coefficient corresponding to the video coding standard with the shortest encoding time in the first prediction result. Since the video coding standard with the shortest encoding time has the lowest computational complexity, the bit rate coefficient of this video coding standard can to some extent reflect the accuracy of the bit rate coefficient of other video coding standards with relatively long encoding times. Therefore, by associating different video coding standards, it is advantageous to further reduce the amount of computation required for the predicted bit rate coefficient.

[0012] It should be understood that the contents described in this section are not intended to identify key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become more readily apparent from the following specification. [Brief explanation of the drawings]

[0013] The drawings illustratively illustrate examples and constitute a part of the specification and, together with the written description, are used to explain exemplary embodiments of the examples. The examples shown are for illustrative purposes only and are not intended to limit the scope of the claims. Throughout the drawings, like reference numerals refer to similar, but not necessarily identical, elements. [Figure 1] 1 illustrates a flowchart of a video encoding method according to some embodiments of the present disclosure. [Figure 2] 1 illustrates a flowchart of a method for training a first bitrate coefficient prediction model according to some embodiments of the present disclosure. [Figure 3] FIG. 2 illustrates a structural block diagram of a first bitrate coefficient prediction model according to some embodiments of the present disclosure. [Figure 4] 10 illustrates a flowchart for determining multiple target bitrate factors for a video segment to be encoded in multiple preset video encoding standards, according to some embodiments of the present disclosure. [Figure 5] 1 illustrates a structural block diagram of a video encoding device according to an embodiment of the present disclosure. [Figure 6] FIG. 1 illustrates a structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0014] The following describes exemplary embodiments of the present disclosure with reference to the drawings, and includes various details of the embodiments of the present disclosure for ease of understanding, and these details should be recognized as illustrative only. Therefore, it should be understood that those skilled in the art can make various changes and modifications to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for clarity and conciseness, the following description omits descriptions of known functions and structures.

[0015] In this disclosure, unless otherwise stated, when terms such as "first," "second," etc. are used to describe various elements, they are not intended to limit the positional, temporal, or importance relationships of these elements, and are used only to distinguish one element from another. In some examples, a first element and a second element may refer to the same instance of the element, but in some cases, they may refer to different instances based on the context.

[0016] In this disclosure, the terms used in the description of each of the above examples are merely for the purpose of describing specific examples and are not limiting. Unless otherwise clearly stated by the context, when the number of elements is not specifically limited, the elements may be one or more. Also, as used in this disclosure, the term "and / or" includes any one and all possible combinations of the listed items.

[0017] With the development of Internet technology, video platforms are also developing rapidly, with more and more users sharing and watching videos. As the number of users on the platform continues to grow, the bandwidth costs for video transmission also increase. To reduce these costs, a transcoding system with better compression performance is required, which can ensure video quality while significantly reducing the size of the compressed video and reducing the network traffic required for transmission.

[0018] Currently, common encoding modes in transcoding systems include fixed quantization parameter mode, constant bit rate mode, variable bit rate mode, etc., among which, the most commonly used encoding mode is constant bit rate mode. Although the video stream output in this mode has a stable bit rate and can improve the playback choppiness problem, this mode encodes all videos with the same bit rate coefficient, so for videos with rich scene content, it cannot guarantee the image quality experience of all videos and cannot avoid bit rate waste. For example, if a video with complex scenes is encoded with an average bit rate coefficient, the output video will have an insufficient bit rate and the image quality will be obviously reduced, while if a video with simple scenes is encoded with an average bit rate coefficient, it may result in unnecessary bit rate waste.

[0019] To ensure the quality experience and avoid bitrate waste, a fixed quality encoding mode can also be adopted. Specifically, a target quality is set, and the encoder encodes the video to obtain a video that meets the target quality. The purpose of this encoding mode is to make the quality of the encoded video as close as possible to the set target quality, and the closer the quality of the encoded video is to the set target quality, the better the encoding effect. If the actual quality is lower than the target quality, it may degrade the quality experience of the video, and if the actual quality exceeds the target quality, it may result in bitrate waste.

[0020] However, before encoding a video using such a coding mode, the relationship between the bit rate coefficient used for encoding and the actual quality of the encoding result is unknown, that is, the exact bit rate coefficient cannot be directly obtained by setting the target quality. The general solutions are as follows (1) and (2).

[0021] (1) The method of encoding the video multiple times and then searching and obtaining the optimal bitrate coefficient by post-verification method. However, such a method requires a large amount of computational resources. (2) A method for predicting a bitrate coefficient, for example, using a machine learning method to predict a bitrate coefficient based on the spatiotemporal complexity information of a video before encoding the video. However, this solution is generally a single-time prediction and encoding method, that is, predicting a bitrate coefficient based on the spatiotemporal complexity information of the video according to a predetermined goal, and encoding the video based on the predicted bitrate coefficient. This results in low accuracy of the bitrate coefficient prediction result, and may not be able to stably guarantee the image quality experience and avoid bitrate waste, especially for videos with complex scenes or high-profile videos.

[0022] The inventors further found that in order to adapt to various terminal players, a transcoding system needs to output a video bitstream that meets different video encoding standards. However, as described above, in the prior art, a single prediction and encoding method is generally adopted, so that the transcoding system needs to predict bitrate coefficients and encode video once for each video encoding standard, which increases the amount of calculation and time costs.

[0023] In view of this, an embodiment of the present disclosure provides a video encoding method, which performs a first prediction of bitrate coefficients for multiple preset video encoding standards based on spatio-temporal domain feature information of a video segment to be encoded, and performs a second prediction of bitrate coefficients for multiple preset video encoding standards based on the predicted bitrate coefficients and a preset video encoding score, thereby improving the accuracy of the bitrate coefficients, effectively improving the accuracy of video encoding, and ensuring video quality. In addition, since the bitrate coefficients for multiple preset video encoding standards can be predicted simultaneously, the encoding system can simultaneously output multiple coded videos that are of different video encoding standards but meet video quality requirements, thereby avoiding repeated prediction operations for multiple video encoding standards and saving time costs.

[0024] Furthermore, in this method, the second prediction of the bit rate coefficient is performed based on the bit rate coefficient corresponding to the video coding standard with the shortest coding time in the first prediction result. Because the video coding standard with the shortest coding time has the lowest computational complexity, the bit rate coefficient of this video coding standard can to some extent reflect the accuracy of the bit rate coefficient of other video coding standards with relatively long coding times, so that by relating different video coding standards, it is advantageous to further reduce the computational complexity required for the predicted bit rate coefficient.

[0025] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0026] 1 shows a flowchart of a video encoding method 100 according to some embodiments of the present disclosure. As shown in FIG. 1, the method 100 may include: obtaining a video segment to be encoded, the video segment including one or more video frames, calculating spatio-temporal domain feature information of the video segment to be encoded based on the one or more video frames, determining a plurality of predicted bit rate coefficients for the video segment to be encoded in a plurality of preset video encoding standards based on the spatio-temporal domain feature information, determining a plurality of target bit rate coefficients for the video segment to be encoded in a plurality of preset video encoding standards based on a first predicted bit rate coefficient corresponding to a first preset video encoding standard among the plurality of preset video encoding standards and a preset video encoding score interval among the plurality of predicted bit rate coefficients, where the first preset video encoding standard has the shortest time for encoding the video among the plurality of preset video encoding standards, and encoding the video segment to be encoded based on the plurality of target bit rate coefficients to obtain a plurality of target video segments, respectively.

[0027] The system uses spatiotemporal domain feature information of the video segment to be coded to determine bitrate coefficients for multiple preset video coding standards, and then determines target bitrate coefficients for the multiple preset video coding standards based on the predicted bitrate coefficients and the preset video coding scores, thereby realizing bitrate coefficient modification, thereby improving the accuracy of the bitrate coefficients, effectively improving the accuracy of video coding, and ensuring video quality. Furthermore, because bitrate coefficients for multiple preset video coding standards can be predicted simultaneously, the coding system can simultaneously output multiple coded videos that meet video quality requirements despite being based on different video coding standards, thereby avoiding repeated prediction operations for multiple video coding standards and saving time. Furthermore, the bitrate coefficient modification is based on the bitrate coefficient corresponding to the video coding standard with the shortest coding time in the first prediction result. Because the video coding standard with the shortest encoding time has the lowest computational complexity, the bitrate coefficients in this video coding standard can to some extent reflect the accuracy of the bitrate coefficients in other video coding standards with relatively longer encoding times, so that by correlating different video coding standards, joint prediction for the bitrate coefficients of different video coding standards can be realized, and the amount of calculation required for predicting the bitrate coefficients can be further reduced.

[0028] According to some embodiments of the present disclosure, the step S110 of obtaining a video segment to be encoded may include the steps of obtaining a video to be encoded, performing scene detection on the video to be encoded, dividing the video to be encoded into one or more sub-video segments based on the scene detection result, and identifying each sub-video segment in the one or more video segments as a video segment to be encoded.

[0029] According to some embodiments, the stored or cached video to be encoded can be read from a suitable storage device (local and / or remote). Alternatively, the video to be encoded can be received from another external device via a wired or wireless communication link. The video to be encoded can refer to any complete video file. For example, it can be a video file recorded by the user himself, a video file captured by the user from other video files, or a video file created by the user based on multiple video files; the scope of the subject matter claimed in this disclosure is not limited in this respect.

[0030] After obtaining the video to be coded, scene detection is performed on the video to be coded, and the video to be coded is divided based on the detection result, and each divided video segment can be set as a video segment to be coded.

[0031] According to some embodiments, scene detection can be performed on a video to be encoded based on an open-source encoder. For example, in response to receiving a video V uploaded by a user, the video V can be determined as a video to be encoded by calling the open-source encoder x264 to encode the video V and turn on a scene detection switch to obtain a corresponding scene change detection result. Based on the detection result, the video V can be divided into t independent single-shot video segments (V1, V2, ... Vt), and each single-shot video segment can be a video segment to be encoded.

[0032] According to some other embodiments, a video to be coded may be divided based on a frame division operation. Continuing with the example where V is the video to be coded, a frame division operation is performed on the video to be coded V to obtain multiple video frames. After that, for each of the video frames, a similarity between the video frame and an adjacent video frame is determined, and based on the calculated similarity, it is determined whether the video frame is a boundary video frame of the video to be processed for distinguishing adjacent shots. It should be understood that any suitable decoder or other technology (e.g., Open CV, FFmpeg, etc.) may be used to perform the frame division operation, and any suitable calculation method may be used to determine the similarity between adjacent video frames (e.g., calculating inter-frame differences, grayscale histograms, etc.), and the scope of the subject matter claimed in the present disclosure is not limited in this respect.

[0033] Compared with a complete video file, the content displayed in a single-shot video segment is generally similar and more consistent, so it is more reasonable to use a specific and identical bitrate coefficient for each single-shot video segment when encoding the video, thereby ensuring a stable image quality experience in each scene.

[0034] According to some embodiments of the present disclosure, the spatio-temporal domain feature information may include spatio-temporal complexity information, and wherein the step S120 of calculating the spatio-temporal domain feature information of the video segment to be encoded based on one or more video frames may include the steps of calculating, based on the one or more video frames, gray level co-occurrence information and normalization information of the video segment to be encoded as spatio-temporal complexity information, wherein the gray level co-occurrence information is used to describe the spatial complexity of the video segment to be encoded and the normalization information is used to describe the temporal domain complexity of the video segment to be encoded; and combining the gray level co-occurrence information and the normalization information to generate the spatio-temporal complexity information of the video segment to be encoded.

[0035] Gray level co-occurrence information (also called a gray level co-occurrence matrix, GLCM) is a feature for describing the spatial texture complexity of a video. In some embodiments, an m×m two-dimensional GLCM matrix can be calculated for each video frame of one or more video frames in a video segment to be coded, where the value of each element in the GLCM matrix ranges from 0 to 1. Five feature values, including information entropy, contrast, inverse difference moment, energy, and correlation, are then calculated for the matrix. Four statistical values, including the mean, variance, skewness, and kurtosis, are calculated for these feature values ​​corresponding to all video frames. For example, the mean, variance, skewness, and kurtosis of information entropy for each video frame to be processed can be calculated. Finally, a total of 20 (5×4) feature values ​​are obtained. Furthermore, to test the multi-scale characteristics of the video, each video frame can be downscaled by 4 times and 16 times, and the above steps can be repeated (i.e., GLCM feature extraction is performed twice). Therefore, 60 feature values, i.e., 60 GLCM feature values, are finally obtained for each video frame of the video segment to be coded.

[0036] Normalized information (also called a normalized correlation coefficient, or NCC) is a feature for describing time-domain complexity. In some embodiments, a two-dimensional correlation coefficient matrix having the same length and width as the original video frames can be obtained by calculating an NCC for each pair of adjacent video frames among one or more video frames in a video segment to be coded. Each element in the matrix has a value ranging from 0 to 1, representing the time-domain correlation between corresponding positions of two adjacent video frames. For a video segment to be coded including n video frames, there are n-1 adjacent video frame pairs, so n-1 correlation coefficient matrices can be obtained. Then, five feature values, including the mean value, variance, kurtosis, skewness, and information entropy, of the n-1 correlation coefficient matrices are calculated. Then, the mean value and variance of each feature value are calculated in the time domain. Finally, ten (5×2) feature values, i.e., ten NCC feature values, can be obtained.

[0037] For example, after calculating the GLCM feature values ​​and NCC feature values ​​using the above method, by combining the two, 70 feature values ​​can be obtained as spatiotemporal complexity information of the video segment to be coded, where the spatiotemporal complexity information can be represented in vector format.

[0038] Since the gray level co-occurrence information and the normalization information characterize the complexity of the video to be coded from spatial and temporal aspects respectively, by predicting multiple bit rate coefficients of the video segment to be coded in different video coding standards based on these spatial complexity information, the predicted bit rate coefficients can be more accurate, which is advantageous for ensuring the quality of the output video.

[0039] According to some embodiments of the present disclosure, the spatio-temporal domain feature information may include coding feature information, and wherein step S120 of calculating the spatio-temporal domain feature information of the video segment to be coded based on one or more video frames may include: performing pre-processing on the one or more video frames to generate a new video frame sequence including a set of video frame pairs; and performing intra-frame coding and inter-frame coding on each video frame pair in the set of video frame pairs, respectively, to obtain the coding feature information of the video segment to be coded.

[0040] By performing intraframe coding or interframe coding on each video frame in a video to be coded, the spatiotemporal domain feature information of the video segment to be coded, i.e., the spatial coding complexity and the temporal domain coding complexity, can be reflected from different dimensions. This is because intraframe coding employs internal image data of the video frame to be coded, while interframe coding employs image data of the video frame to be coded and its adjacent video frames, thereby enabling a more comprehensive characterization of the video segment to be coded. By combining the coding feature information with the spatiotemporal complexity information consisting of GLCM feature values ​​and NCC feature values, multiple bit rate coefficients of the video segment to be coded in different video coding standards can be predicted, thereby further improving the accuracy of the prediction result, which is particularly useful for expecting higher quality and more detailed coding of the target video.

[0041] According to some embodiments of the present disclosure, in step S120, the step of pre-processing one or more video frames and generating a new video frame sequence may include the steps of copying video frames other than the first frame and the last frame among the one or more video frames to generate one or more copy video frames, and rearranging the one or more video frames and the one or more copy video frames so that each copy video frame among the one or more copy video frames is located after a corresponding video frame.

[0042] JPEG2026500919000002.jpg62170

[0043] Compared to a video frame sequence in which no copy frames are generated, by copying video frames other than the first and last frames to generate one or more ordered video frame pairs, it is possible to ensure that all data within each video frame and between adjacent video frames is available, thereby maintaining data integrity, achieving more accurate intra-frame and inter-frame coding of each video frame of the video segment to be coded, and advantageously improving the accuracy of the predicted bit rate coefficients.

[0044] According to some other embodiments of the present disclosure, performing pre-processing on one or more video frames may further include, for example, trimming each video frame to meet a size requirement, removing unnecessary (interference) features in the video frame, etc., which is advantageous to improve the calculation and prediction speed, quickly obtain coding feature information, and quickly predict the bit rate coefficient of the video to be coded, and at the same time, improve the accuracy of the predicted bit rate coefficient.

[0045] After pre-processing the video segment to be encoded to obtain a set of video frame pairs, the video segment can be pre-encoded, for example, by a hardware H.264 encoder built into a graphics card. The pre-encoding can employ, for example, a fixed quantization parameter mode, and optionally, the quantization parameter can be any suitable value, such as 28, 29, or 30. The coding frame structure can employ, for example, an IP structure, where I represents intraframe coding and P represents interframe coding. In this coding mode, except for the first and last frames, each video frame of the video segment to be encoded is coded twice (i.e., once using intraframe coding I and once using interframe coding P), where the intraframe coding can reflect the spatial coding complexity of each video frame, and the interframe coding can reflect the temporal coding complexity of each video frame. After I·P coding is performed on each set of video frame pairs, coding feature information of the video segment to be encoded can be obtained.

[0046] According to some embodiments of the present disclosure, the step of performing intra-frame coding and inter-frame coding on each video frame pair in the set of video frame pairs and obtaining coding feature information of the video segment to be coded may include the steps of performing intra-frame coding on a first video frame in each video frame pair in the set of video frame pairs and obtaining intra-frame coding information, performing inter-frame coding on each video frame pair in the set of video frame pairs and obtaining inter-frame coding information, and obtaining coding feature information based on the intra-frame coding information and the inter-frame coding information.

[0047] JPEG2026500919000003.jpg30170

[0048] According to some examples, the step of performing intraframe coding on the first video frame in each video frame pair and obtaining intraframe coding information may include the steps of: extracting 34 feature values ​​of each intraframe coding, such as the number of coding bits and the proportion of intraframe prediction modes, after performing intraframe coding on each first video frame; and calculating five statistical values, namely the mean value, variance, skewness, kurtosis, and information entropy, of the corresponding feature values ​​of all intraframe codings, to obtain 170 (i.e., 34 x 5) pieces of intraframe coding information.

[0049] JPEG2026500919000004.jpg26170

[0050] According to some examples, the step of performing inter-frame coding on each video frame pair and obtaining inter-frame coding information may include the steps of: after performing inter-frame coding on each first video frame, extracting 26 feature values ​​of each inter-frame coding, such as the number of coding bits and the amplitude distribution of inter-frame motion vectors; and calculating five statistical values, namely the mean value, variance, skewness, kurtosis, and information entropy, of the corresponding feature values ​​of all inter-frame codings, to obtain 130 (i.e., 26 x 5) pieces of inter-frame coding information.

[0051] Continuing with the above example, 170 pieces of intraframe coding information and 130 pieces of interframe coding information are combined to obtain coding feature information for 300 video segments to be coded. Similar to the spatial complexity information, the coding feature information can also be represented in vector form.

[0052] It will be appreciated that although the above examples describe the intraframe coding step as being performed after the interframe coding step, this should not be construed as requiring that the intraframe coding and interframe coding be performed in this particular order. For example, the intraframe coding step may be performed in parallel with the interframe coding step or may be performed after the interframe coding step.

[0053] As will be further understood, in the above embodiments, the intraframe coding information and interframe coding information are described as the mean value, variance, skewness, kurtosis and information entropy of feature values ​​such as the number of coding bits, the proportion of intraframe prediction modes, and the amplitude distribution of interframe motion vectors, but these feature values ​​can also be determined directly as intraframe coding information or interframe coding information.

[0054] According to some embodiments of the present disclosure, the step S130 of determining a plurality of predicted bitrate coefficients of the video segment to be encoded in the plurality of preset video encoding standards based on the spatio-temporal domain feature information may include inputting the spatio-temporal domain feature information of the video segment to be encoded into a first bitrate coefficient prediction model, and determining a plurality of predicted bitrate coefficients of the video segment to be encoded in the plurality of preset video encoding standards using the first bitrate coefficient prediction model.

[0055] The bitrate factor (RF) is a coding parameter in the constant bitrate factor coding mode, used to control the bitrate and quality of the coded file. The value of the bitrate factor usually lies in the range of 0 to 50. The higher the bitrate factor, the lower the bitrate and the lower the coding quality, and vice versa.

[0056] Examples of the plurality of preset video encoding standards may include the H.264 video encoding standard, the H.265 video encoding standard, the H.266 video encoding standard, and the AV1 video encoding standard. As should be understood, the preset video standards may further include any other video encoding standards that exist or are created as technology evolves. As should be further understood, the number of the plurality of preset video encoding standards may be any integer value that meets actual needs. The scope of the subject matter claimed in this disclosure is not limited in either of these two respects.

[0057] According to some embodiments of the present disclosure, when the spatio-temporal domain feature information only includes spatio-temporal complexity information, feature information combining gray level co-occurrence information and normalization information of the video segment to be encoded can be input into a first bitrate coefficient prediction model, where the first bitrate coefficient prediction model is trained to determine predicted bitrate coefficients of each video segment to be encoded in multiple preset video coding standards based on the spatio-temporal complexity information.

[0058] According to some other embodiments of the present disclosure, when the spatio-temporal domain feature information includes only coding feature information, feature information combining intra-frame coding information and inter-frame coding information of the video segment to be coded can be input into a first bitrate coefficient prediction model, where the first bitrate coefficient prediction model is trained to determine predicted bitrate coefficients of each video segment to be coded in multiple preset video coding standards based on the coding feature information.

[0059] According to some other embodiments of the present disclosure, when the spatio-temporal domain feature information includes both spatio-temporal complexity information and coding feature information, the spatio-temporal complexity information and the coding feature information can be combined to generate feature information represented by a set of feature vectors. Continuing with an example where the spatio-temporal complexity information is a vector including 70 feature values ​​and the coding feature information is a vector including 300 feature values, combining these two vectors results in a feature vector including 370 feature values, which can more completely characterize the spatio-temporal characteristics of the video segment to be coded. In this case, the generated set of feature vectors can be input into a first bitrate coefficient prediction model, where the first bitrate coefficient prediction model is trained to determine predicted bitrate coefficients for each video segment to be coded under multiple preset video coding standards based on both the spatio-temporal complexity information and the coding feature information.

[0060] By using the trained first bitrate coefficient prediction model to predict the predicted bitrate coefficients of each video segment to be coded in multiple preset video coding standards, it is possible to directly determine the bitrate coefficients that meet the quality requirements (e.g., the video coding score of the coded video segment is relatively high) without performing multiple coding and searching during each coding, thereby saving a lot of time. At the same time, when sample data is sufficient, the accuracy of the predicted bitrate coefficients can be guaranteed to a certain extent.

[0061] 2 shows a flowchart of method 200 for training a first bitrate coefficient prediction model according to some embodiments of the present disclosure. As shown in FIG. 2, the first bitrate coefficient prediction model can be trained by the following steps of method 200: obtaining sample spatio-temporal domain feature information of a sample video segment and sample bitrate coefficients of the sample video segment in the preset video coding standard for each preset video coding standard in a plurality of preset video coding standards, inputting the sample spatio-temporal domain feature information into a first bitrate coefficient prediction model and determining sample predicted bitrate coefficients of the sample video segment in the preset video coding standard using the first bitrate coefficient prediction model, calculating a first model loss value based on the sample bitrate coefficients and the sample predicted bitrate coefficients, and adjusting parameters of the first bitrate coefficient prediction model based on the first model loss value in each preset video coding standard until a first model training stopping condition is achieved.

[0062] The accuracy of the predicted bitrate coefficients can be improved by pre-training and tuning the first bitrate coefficient prediction model using sample data.

[0063] According to some embodiments, in step S210, a sample video segment may be obtained in a manner similar to the manner of obtaining the video segment to be encoded in step S110, and the sample spatio-temporal domain feature information may be calculated in a manner similar to the manner of calculating the spatio-temporal domain feature information of the video segment to be encoded in step S120, and detailed descriptions thereof will be omitted here. To ensure the accuracy of the trained first bitrate coefficient prediction model, more sample video segments may be obtained under the premise of balancing computational resources and time costs, and the number of sample video segments may be, for example, at least 50,000, at least 100,000, etc.

[0064] According to some embodiments, in step S210, the sample bitrate coefficient may be an actual bitrate coefficient, i.e., the quality of the encoded video obtained by encoding the sample video segment using the bitrate coefficient meets the target quality requirement. The quality of the encoded video can be evaluated, for example, by a video encoding score (also known as Video Multimethod Assessment Fusion, VMAF), where a larger VMAF value indicates a higher quality of the encoded video, and conversely, a smaller VMAF value indicates a lower quality of the encoded video, where the VMAF value generally ranges from 0 to 100. The actual bitrate coefficient can be obtained by the following procedure: For the video coding standard, an initial sample bitrate coefficient and a preset sample video coding score interval are selected; a sample video segment is encoded using a corresponding encoder (e.g., H.264 encoder, H.265 encoder, H.266 encoder, AV1 encoder, etc.) in an actual transcoding system based on the initial sample bitrate coefficient to obtain a sample pre-coded video segment in the video coding standard; the sample video coding score of the obtained sample pre-coded video segment is compared with the preset sample video coding score interval; if the sample video coding score is within the preset sample video coding score interval (i.e., the encoded video satisfies the target quality requirement), it indicates that the initial sample bitrate coefficient in the video coding standard is feasible, and the initial sample bitrate coefficient can be adopted as the actual bitrate coefficient; if the sample video coding score is not within the preset sample video coding score interval (i.e., the encoded video does not meet the target quality requirement), it indicates that the initial sample bitrate coefficient in the video coding standard is not feasible, and the initial sample bitrate coefficient can be adjusted using a search method such as bisection until the sample video coding score is within the preset sample video coding score interval.Usually, for the same sample video segment, the bit rate coefficients in different video coding standards are generally different, so the above steps are repeated for each video coding standard to search and obtain the sample bit rate coefficients (actual bit rate coefficients) of the sample video segment in multiple video coding standards.

[0065] In the above embodiment, the preset sample video encoding score interval may be, for example, a range of values ​​(e.g., [91, 93], [90, 95]) based on VMAF and customized by a user according to actual needs, or may be an interval set for each VMAF integer value (e.g., [91.5, 92.5], [92.5, 93.5]...[95.5, 96.5], etc.) to adapt the first bitrate coefficient prediction model to various target quality requirements. The scope of the subject matter claimed in the present disclosure is not limited in this respect.

[0066] According to some embodiments, in step S220, at an initial stage, initial parameters of a first bitrate coefficient prediction model can be selected for multiple video coding standards, and based on the initial parameters, multiple sample predicted bitrate coefficients of a sample video segment in the multiple video coding standards can be determined using the obtained sample spatio-temporal domain feature information.

[0067] According to some embodiments, in step S230, for each preset video coding standard, a model loss value may be calculated based on the corresponding sample predicted bit rate coefficient and the sample predicted bit rate coefficient, e.g., by a loss function. Examples of loss functions include, but are not limited to, a cross-entropy loss function, a maximum loss function, a mean value loss function, a 0-1 loss function, etc. It will be appreciated that the present disclosure may also calculate the model loss value in other suitable manners, and the scope of the claimed subject matter in this disclosure is not limited in this respect.

[0068] According to some embodiments, in step S240, after calculating the model loss values ​​for each preset video coding standard, these model loss values ​​can be backpropagated and the parameters of the first bitrate coefficient prediction model can be adjusted, and steps S220 to S240 can be repeated until a model training stopping condition is met. Specifically, the model training stopping condition can be that the model loss value is less than a preset threshold and / or the number of rounds of model training reaches a preset number of rounds. It should be understood that the present disclosure does not limit the model training stopping condition and can be adjusted according to actual needs.

[0069] 3 shows a structural block diagram of a first bitrate coefficient prediction model 300 according to some embodiments of the present disclosure. As shown in FIG. 3, the first bitrate coefficient prediction model 300 may include a batch normalization module 310, a first attention module 320, a residual module 330, a second attention module 340, and a fully connected module 350, and takes spatio-temporal domain features of a video segment to be encoded as input and outputs predicted bitrate coefficients of the video segment to be encoded in different video coding standards.

[0070] The batch normalization module 310 normalizes the original input features into features with a mean of 0, a variance of 1, and a Gaussian distribution to eliminate the influence of dimensional differences between features. The normalized features are then input to a first attention module 320, which calculates the weight of each feature, where features with higher recognizability are given a higher weight and features with relatively less information content are given a lower weight, thereby automatically filtering out features and improving the accuracy of prediction results. The filtered features are then input to a residual module 330, which includes multiple convolutional layers. These convolutional layers modify the filtered input features, while the unchanged filtered features are passed through these convolutional layers to subsequent layers, and the overall processing is performed to obtain the output result of the residual module, thereby reducing information loss. The output of the residual module 330 can then be input to a second attention module 340, which further filters out features and further improves the accuracy of prediction results. Finally, the input features can be aggregated by a fully connected module 350 to generate and output predicted bitrate coefficients.

[0071] 4 shows a flowchart for determining a plurality of target bit rate coefficients for a video segment to be encoded with a plurality of preset video encoding standards according to some embodiments of the present disclosure. As shown in FIG. 4, step S140 of determining a plurality of target bit rate coefficients for a video segment to be encoded with a plurality of preset video encoding standards based on a first predicted bit rate coefficient corresponding to a first preset video encoding standard among the plurality of preset video encoding standards and a preset video encoding score interval may include step S442 of performing a first encoding of the video segment to be encoded based on the first predicted bit rate coefficient to obtain a first pre-encoded video segment with the first preset video encoding standard, step S444 of determining whether the video encoding score of the first pre-encoded video segment is within the preset video encoding score interval, and step S446 of determining the plurality of predicted bit rate coefficients as a plurality of target bit rate coefficients in response to the video encoding score of the first pre-encoded video segment being within the preset video encoding score interval.

[0072] In the present disclosure, the first preset video encoding standard refers to the video encoding standard that requires the shortest time for video encoding. For example, among the video encoding standards H.264, H.265, H.266, and AV1, the video encoding standard H.264 requires the shortest time for video encoding, so the video encoding standard H.264 is the first preset video encoding standard.

[0073] In step S442, except for the first predicted bit rate coefficient, the coding parameters used when encoding the target video may further include, but are not limited to, preset, GOP (Group of Picture), size, etc. Because the bit rate coefficient plays an important role in the quality of the encoded video, when encoding the video using the method of the present disclosure, the values ​​of these coding parameters can be set as fixed constants (e.g., default preset values).

[0074] In step S444, similar to the setting of the preset sample video encoding score interval, the preset video encoding score interval may be a numerical range customized by the user according to actual needs, and therefore detailed description thereof will be omitted here.

[0075] Because the video coding standard with the shortest encoding time has the lowest computational complexity, the bit rate coefficient for this video coding standard can reflect to some extent the accuracy of the bit rate coefficient for other video coding standards with relatively longer encoding times. In other words, when the video coding score of a video segment coded based on a first predicted bit rate coefficient corresponding to a first preset video coding standard (e.g., the video coding standard H.264 in the above example) is within a preset video coding score interval (meets the video quality requirement), it indicates that the accuracy of other predicted bit rate coefficients corresponding to other preset video coding standards (e.g., the video coding standards H.265, H.266, and AV1 in the above example) is also high (e.g., when the probability that the video coding score of a video coded using the video coding standard H.264 is within the preset video coding score interval is 99%, the probability that the video coding scores of videos coded using the video coding standards H.265, H.266, and AV1 are within the preset video coding score interval is also more than 95%). The video coding scores of the video segments coded with these predicted bit rate coefficients are also generally within the preset video coding score interval (meeting the video quality requirement), so that different video coding standards can be associated, which is advantageous for reducing the amount of calculation required for the predicted bit rate coefficients. In some examples, data from multiple tests shows that if the predicted bit rate coefficients in the video coding standard H.264 meet the video quality requirement, there is a 95% probability that the predicted bit rate coefficients in each of the video coding standards H.265, H.266, and AV1 will also meet the video quality requirement, so these predicted bit rate coefficients can be directly used as target bit rate coefficients for coding the video to be coded.

[0076] Continuing to refer to FIG. 4, according to some other embodiments of the present disclosure, step S140 of determining a plurality of target bitrate coefficients for a video segment to be encoded with a plurality of preset video encoding standards based on a first predicted bitrate coefficient corresponding to a first preset video encoding standard among the plurality of preset video encoding standards and a preset video encoding score interval may further include step S448 of updating the plurality of predicted bitrate coefficients to obtain a plurality of target bitrate coefficients based on spatio-temporal domain feature information of the video segment to be encoded, the plurality of predicted bitrate coefficients and the video encoding score of the first pre-encoded video segment in response to the video encoding score of the first pre-encoded video segment not being within the preset video encoding score interval.

[0077] Continuing with an example where the plurality of preset video encoding standards are H.264, H.265, H.266, and AV1, when the video encoding score of the pre-encoded video segment obtained based on the predicted bit rate coefficients in the video encoding standard H.264 is not within the preset video encoding score interval, a second prediction can be performed on the bit rate coefficients in these video encoding standards using the respective predicted bit rate coefficients in the video encoding standards H.265, H.266, and AV1, and the video encoding score of the pre-encoded video segment in the video encoding standard H.264 as feedback information, together with the spatio-temporal domain feature information of the video segment to be encoded, to obtain a plurality of target bit rate coefficients.

[0078] According to a large amount of experimental data, after the first prediction and encoding of the to-be-encoded video segment according to the predicted bit rate coefficient, the probability that the video coding score meets the video quality requirement is 40%. Therefore, for 60% of the to-be-encoded video segments, a second prediction and update operation of the bit rate coefficient is performed. When the video coding score of the first pre-encoded video segment is not within the preset video coding score range, a second prediction of the bit rate coefficient in a different video coding standard is performed, thereby further improving the accuracy of the target bit rate coefficient and avoiding the situation where the predicted bit rate coefficient for the 60% of the to-be-encoded video segments has a large error and cannot guarantee a stable picture quality experience.

[0079] In addition, the predicted bit rate coefficient obtained in the first prediction is updated based on the video coding score of the first pre-coded video segment, eliminating the need to code the video segments to be coded using predicted bit rate coefficients in other preset video coding standards, thereby significantly reducing the time cost and computational resources required for coding.

[0080] According to some embodiments of the present disclosure, step S448 of updating the plurality of predicted bitrate coefficients to obtain a plurality of target bitrate coefficients based on the spatiotemporal domain feature information of the video segment to be encoded, the plurality of predicted bitrate coefficients, and the video encoding score of the first pre-encoded video segment may include inputting the spatiotemporal domain feature information of the video segment to be encoded, the plurality of predicted bitrate coefficients, and the video encoding score of the first pre-encoded video segment into a second bitrate coefficient prediction model, and determining a plurality of target bitrate coefficients of the video segment to be encoded in a plurality of preset video encoding standards using the second bitrate coefficient prediction model.

[0081] According to some embodiments of the present disclosure, similar to the method of training the first bitrate coefficient prediction model, the second bitrate coefficient prediction model may be trained by: inputting sample spatio-temporal domain feature information, sample predicted bitrate coefficients for each preset video encoding standard, and first sample video coding scores corresponding to the sample predicted bitrate coefficients for the first preset video encoding standard into the second bitrate coefficient prediction model; determining a plurality of sample target bitrate coefficients for the sample video segment in the plurality of preset video encoding standards using the second bitrate coefficient prediction model; calculating, for each preset video encoding standard in the plurality of preset video encoding standards, a second model loss value based on the sample bitrate coefficients and the sample target bitrate coefficients corresponding to the preset video encoding standard; and adjusting parameters of the second bitrate coefficient prediction model based on the second model loss value until a second model training stopping condition is achieved.

[0082] As can be understood, in addition to adding the sample predicted bitrate coefficients for each preset video coding standard and the first sample video coding scores as inputs of the second bitrate coefficient prediction model, the steps in the above method for training the second bitrate coefficient prediction model are similar to the method 200 for training the first bitrate coefficient prediction model, and therefore will not be described in detail here.

[0083] Accordingly, compared with the first bitrate coefficient prediction model, in addition to adding sample predicted bitrate coefficients in each preset video encoding standard and the first sample video encoding score as inputs, the structural framework of the second bitrate coefficient prediction model is also similar to that of the first bitrate coefficient prediction model, so detailed description is omitted here.

[0084] It will be further understood that the training for the second bit-rate coefficient prediction model depends on the training result of the first bit-rate coefficient prediction model, and therefore, the training for the second bit-rate coefficient prediction model is performed after the training for the first bit-rate coefficient prediction model is completed.

[0085] After determining the plurality of target bit rate coefficients, the video segment to be encoded is encoded (according to the corresponding preset video encoding standard) using corresponding encoders in the transcoding system based on the plurality of bit rate coefficients. According to multiple test data, after encoding the video segment using the target bit rate coefficients obtained by the second prediction, the probability that the video encoding score of the encoded video segment is within the preset video encoding score range is 99%, so the video segment encoding result using the target bit rate coefficients obtained by the second prediction can be directly trusted.

[0086] After encoding each to-be-encoded video segment with the multiple target bit rate coefficients, method 100 may further include, for each preset video encoding standard in the multiple preset video encoding standards, obtaining all target video segments in the preset video encoding standard, and combining all target video segments based on the order of one or more sub-video segments in the to-be-encoded video to obtain a target video corresponding to the to-be-encoded video in the preset video encoding standard.

[0087] After performing the above encoding operation on each to-be-encoded video segment in the to-be-encoded video, a plurality of target video segments can be obtained for each to-be-encoded video segment. Then, for each preset video encoding standard, a corresponding target video segment among the plurality of target video segments for each to-be-encoded video segment is written into a final video bitstream file, and the written target video segments are combined according to the order of each to-be-encoded video segment in the to-be-encoded video to obtain a target video corresponding to the to-be-encoded video for the preset video encoding standard.

[0088] As described above, since each of the to-be-encoded video segments is a single-shot video segment with similar display content and consistency, by encoding each of the to-be-encoded video segments to obtain a corresponding target video segment, the quality of each of the encoded video segments can be guaranteed, and a stable image quality experience can be ensured for each scene. Furthermore, by combining and combining the obtained target video segments based on the order of these to-be-encoded video segments in the to-be-encoded video, a complete video file can be restored, and discontinuities in the video file due to errors in combining the target video segments can be avoided.

[0089] 5 shows a structural block diagram of a video encoding device 500 according to an embodiment of the present disclosure. As shown in FIG. 5, the device 500 may include: an acquisition module 510 configured to acquire a to-be-encoded video segment including one or more video frames; a calculation module 520 configured to calculate, based on the one or more video frames, spatio-temporal domain feature information of the to-be-encoded video segment; a predicted bitrate coefficient determination module 530 configured to determine, based on the spatio-temporal domain feature information, a plurality of predicted bitrate coefficients for the to-be-encoded video segment in a plurality of preset video encoding standards; a target bitrate coefficient determination module 540 configured to determine, based on the plurality of predicted bitrate coefficients, a first predicted bitrate coefficient corresponding to a first preset video encoding standard among the plurality of preset video encoding standards and a preset video encoding score interval, where the target bitrate coefficient determination module 550 is configured to determine, among the plurality of preset video encoding standards, a plurality of target bitrate coefficients for the to-be-encoded video segment in a plurality of preset video encoding standards,

[0090] According to some embodiments of the present disclosure, the spatio-temporal domain feature information may include spatio-temporal complexity information, and wherein the calculation module 530 may include: a module configured to calculate, based on one or more video frames, gray level co-occurrence information and normalization information of the video segment to be encoded as spatio-temporal complexity information, where the gray level co-occurrence information is used to describe the spatial complexity of the video segment to be encoded and the normalization information is used to describe the temporal domain complexity of the video segment to be encoded; and a module configured to combine the gray level co-occurrence information and the normalization information to generate the spatio-temporal complexity information of the video segment to be encoded.

[0091] According to some embodiments of the present disclosure, the spatio-temporal domain feature information may include coding feature information, and wherein the calculation module 530 may include a pre-processing module configured to perform pre-processing on one or more video frames to generate a new video frame sequence including a set of video frame pairs, and intra-frame and inter-frame coding modules configured to perform intra-frame coding and inter-frame coding, respectively, on each video frame pair in the set of video frame pairs to obtain coding feature information of the video segment to be coded.

[0092] According to some embodiments of the present disclosure, the pre-processing module may include a module configured to copy video frames other than the first and last frames of the one or more video frames to generate one or more copy video frames, and a module configured to rearrange the one or more video frames and the one or more copy video frames such that each copy video frame of the one or more copy video frames is located after a corresponding video frame.

[0093] According to some embodiments of the present disclosure, the intraframe and interframe coding module may include a module configured to perform intraframe coding on a first video frame in each video frame pair of the set of video frame pairs to obtain intraframe coding information, a module configured to perform interframe coding on each video frame pair of the set of video frame pairs to obtain interframe coding information, and a module configured to obtain coding feature information based on the intraframe coding information and the interframe coding information.

[0094] According to some embodiments of the present disclosure, the coding feature information may be based on one or more of the number of coding bits, the intra-frame prediction mode ratio, and amplitude distribution information of inter-frame motion vectors.

[0095] According to some embodiments of the present disclosure, the predicted bitrate coefficient determination module 530 may include a module configured to input spatiotemporal domain feature information of the video segment to be encoded into a first bitrate coefficient prediction model, and determine a plurality of predicted bitrate coefficients of the video segment to be encoded in a plurality of preset video coding standards using the first bitrate coefficient prediction model.

[0096] According to some embodiments of the present disclosure, the target bitrate coefficient determination module 540 may include a module configured to perform a first encoding of the video segment to be encoded based on a first predicted bitrate coefficient to obtain a first pre-encoded video segment in a first preset video encoding standard; a module configured to determine whether the video encoding score of the first pre-encoded video segment is within a preset video encoding score interval; and a module configured to determine a plurality of predicted bitrate coefficients as a plurality of target bitrate coefficients in response to the video encoding score of the first pre-encoded video segment being within the preset video encoding score interval.

[0097] According to some embodiments of the present disclosure, the target bitrate coefficient determination module 540 may further include an update module configured to update the plurality of predicted bitrate coefficients to obtain a plurality of target bitrate coefficients based on the spatiotemporal domain feature information of the video segment to be encoded, the plurality of predicted bitrate coefficients, and the video coding score of the first pre-encoded video segment in response to the video coding score of the first pre-encoded video segment not being within a preset video coding score interval.

[0098] According to some embodiments of the present disclosure, the update module may include a module configured to input spatiotemporal domain feature information of the video segment to be encoded, a plurality of predicted bitrate coefficients, and a video encoding score of the first pre-encoded video segment into a second bitrate coefficient prediction model, and determine a plurality of target bitrate coefficients of the video segment to be encoded in a plurality of preset video encoding standards using the second bitrate coefficient prediction model.

[0099] According to some embodiments of the present disclosure, the first bitrate coefficient prediction model may be trained by, for each preset video coding standard in a plurality of preset video coding standards, obtaining sample spatio-temporal domain feature information of a sample video segment and sample bitrate coefficients of the sample video segment at the preset video coding standard; inputting the sample spatio-temporal domain feature information into the first bitrate coefficient prediction model; determining sample predicted bitrate coefficients of the sample video segment at the preset video coding standard using the first bitrate coefficient prediction model; calculating a first model loss value based on the sample bitrate coefficients and the sample predicted bitrate coefficients; and adjusting parameters of the first bitrate coefficient prediction model based on the first model loss value at each preset video coding standard until a first model training stopping condition is achieved.

[0100] According to some embodiments of the present disclosure, the second bitrate coefficient prediction model may be trained by: inputting the sample spatio-temporal domain feature information, the sample predicted bitrate coefficients for each preset video encoding standard, and the first sample video coding scores corresponding to the sample predicted bitrate coefficients for the first preset video encoding standard into the second bitrate coefficient prediction model; determining a plurality of sample target bitrate coefficients for the sample video segment for the plurality of preset video encoding standards using the second bitrate coefficient prediction model; calculating, for each preset video encoding standard in the plurality of preset video encoding standards, a second model loss value based on the sample bitrate coefficients and the sample target bitrate coefficients corresponding to the preset video encoding standard; and adjusting parameters of the second bitrate coefficient prediction model based on the second model loss value for each preset video encoding standard until a second model training stopping condition is achieved.

[0101] According to some embodiments of the present disclosure, the acquisition module 510 may include a module configured to acquire a video to be encoded, a module configured to perform scene detection on the video to be encoded, a module configured to divide the video to be encoded into one or more sub-video segments based on the scene detection result, and a module configured to identify each sub-video segment in the one or more video segments as a video segment to be encoded.

[0102] According to some embodiments of the present disclosure, the apparatus 500 may further include, for each preset video encoding standard in the plurality of preset video encoding standards, a module configured to obtain all target video segments in the preset video encoding standard, and a module configured to combine all target video segments based on the order of one or more sub-video segments in the video to be encoded to obtain a target video in the preset video encoding standard corresponding to the video to be encoded.

[0103] It should be understood that each of the modules 510-550 of the apparatus 500 shown in Figure 5 may correspond to each of the steps S110-S150 in the method 100 described with reference to Figure 1. Accordingly, the operations, features, and advantages described above with respect to the method 100 also apply to the apparatus 500 and the modules included therein. For the sake of brevity, certain operations, features, and advantages will not be described in detail herein.

[0104] Furthermore, it should be understood that various techniques may be described herein in the general context of software and hardware elements or program modules. As described above, each module illustrated in FIG. 5 may be implemented in hardware or a combination of software and / or firmware. For example, these modules may be implemented as computer program code / instructions executed by one or more processors and stored on a computer-readable storage medium. Alternatively, these modules may be implemented as hardware logic / circuitry. For example, in some embodiments, one or more of the acquisition module 510, the calculation module 520, the predicted bitrate coefficient determination module 530, the target bitrate coefficient determination module 540, and the encoding module 550 may be integrated into a system on a chip (SoC). An SoC may include an integrated circuit chip (including one or more components of a processor (e.g., a Central Processing Unit (CPU), microcontroller, microprocessor, Digital Signal Processor (DSP), etc.), memory, one or more communication interfaces, and / or other circuitry) and may include embedded firmware to selectively execute received program code and / or perform functions.

[0105] According to another aspect of the present disclosure, there is further provided an electronic device including at least one processor and at least one memory communicatively connected to the at least one processor, wherein the at least one memory stores a computer program that, when executed by the at least one processor, implements the above-described video encoding method.

[0106] According to another aspect of the present disclosure, there is further provided a non-transitory computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the above video encoding method.

[0107] According to another aspect of the present disclosure, there is further provided a computer program product including a computer program which, when executed by a processor, implements the above video encoding method.

[0108] Referring to FIG. 6 , a structural block diagram of an electronic device 600 usable as a server of the present disclosure is described, which is an example of a hardware device applicable to each aspect of the present disclosure. The electronic device may be different types of computing devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, large-scale computers, and other suitable computers. The electronic device may also represent various types of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the practice of the present disclosure as described and / or claimed herein.

[0109] As shown in FIG. 6, the electronic device 600 may include at least one processor 610, a working memory 620, an input unit 640, a display unit 650, a speaker 660, a storage unit 670, a communication unit 680 and other output units 690 that are in communication with each other via a system bus 630.

[0110] The processor 610 may be a single processing unit or multiple processing units, and all processing units may include single or multiple computational units or multiple cores. The processor 610 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operational instructions. The processor 610 may be configured to retrieve and execute computer-readable instructions stored in the working memory 620, the storage unit 670, or other computer-readable medium, such as program code for the operating system 620a, program code for the application program 620b, etc.

[0111] The working memory 620 and the storage unit 670 are examples of computer-readable storage media for storing instructions that are executed by the processor 610 to perform the various functions described above. The working memory 620 may include both volatile and non-volatile memory (e.g., RAM, ROM, etc.). The storage unit 670 may also include hard disk drives, solid-state drives, removable media, external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CDs, DVDs), storage arrays, network-attached storage, storage area networks, etc. The working memory 620 and the storage unit 670 may be collectively referred to herein as memory or computer-readable storage media, and may also be non-transitory media that can store computer-readable, processor-executable program instructions as computer program code that can be executed by the processor 610 as a specific machine configured to perform the operations and functions described in the examples herein.

[0112] The input unit 660 may be any type of device capable of inputting information into the electronic device 600, receiving input numeric or character information, and generating key signal inputs related to user settings and / or function control of the electronic device, and may include, but is not limited to, a mouse, keyboard, touch screen, trackpad, trackball, joystick, microphone, and / or remote control. The output unit may be any type of device capable of presenting information, and may include, but is not limited to, a display unit 650, a speaker 660, and other output units 690, which may include, but are not limited to, a video / audio output terminal, a vibrator, and / or a printer. The communication unit 680 enables the electronic device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various communication networks, and may include a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, e.g., Bluetooth. TM This may include, but is not limited to, devices, 802.11 devices, Wi-Fi devices, WiMAX devices, cellular communication devices and / or the like.

[0113] The application program 620b in the working memory 620 may be loaded to perform the above-described methods and steps thereof, e.g., steps S110-S150 in FIG. 1, steps S210-S240 in FIG. 2, and steps S442-S448 in FIG. 4. For example, in some embodiments, the above-described methods may be implemented as a computer software program tangibly contained in a machine-readable medium, such as the storage unit 670. In some embodiments, part or all of the computer program may be loaded and / or installed in the electronic device 600 via the storage unit 670 and / or the communication unit 680. When the computer program is loaded and executed by the processor 610, it may perform one or more steps of the above-described methods 100, 200. Alternatively, in other embodiments, the processor 610 may be configured to perform the methods 100, 200 in any other suitable manner (e.g., using firmware).

[0114] Various embodiments of the systems and techniques described herein may be realized in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include being embodied in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general purpose programmable processor, and that can receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0115] Program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, so that when the program code is executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are performed. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a separate software package, or entirely on a remote machine or server.

[0116] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, device, or apparatus. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of machine-readable storage media include one or more wire-based electrical connections, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0117] To provide for user interaction, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other types of devices can also be used to provide for user interaction; for example, the feedback provided to the user can be any form of sensing feedback (e.g., visual feedback, auditory feedback, or tactile feedback) and can receive input from the user in any form (including sound, speech, or tactile input).

[0118] The systems and techniques described herein can be implemented in a computing system that includes background components (e.g., a data server), or middleware components (e.g., an application server), or front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or any combination of such background, middleware, or front-end components. The components of the system can be connected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0119] A computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the corresponding computers and having a client-server relationship to each other.

[0120] It should be understood that steps can be rearranged, added, or deleted using the flow of each of the above-described forms. For example, each step described in the present disclosure may be performed in parallel, sequentially, or in a different order, and this specification does not limit this, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.

[0121] Although the embodiments or examples of the present disclosure have been described with reference to the drawings, it should be understood that the above-described methods, systems, and devices are merely exemplary embodiments or examples, and that the scope of the present invention is not limited by these embodiments or examples, but only by the scope of the patented claims and their equivalents. Elements in the embodiments or examples may be omitted or replaced with equivalent elements. Steps may also be performed in an order different from that described in this disclosure. Furthermore, elements in the embodiments or examples may be combined in various forms. It is important to note that with the development of technology, many of the elements described herein may be replaced by equivalent elements that appear later in this disclosure.

Claims

1. obtaining a video segment to be encoded, the video segment comprising one or more video frames; calculating spatiotemporal domain feature information of the video segment to be encoded based on the one or more video frames; determining a plurality of predicted bit rate coefficients for the video segment to be encoded in a plurality of preset video encoding standards based on the spatio-temporal domain feature information; determining a plurality of target bitrate coefficients for the video segment to be encoded in the plurality of preset video encoding standards based on a first predicted bitrate coefficient corresponding to a first preset video encoding standard among the plurality of preset video encoding standards and a preset video encoding score interval, wherein among the plurality of preset video encoding standards, a time required to encode video using the first preset video encoding standard is shortest; encoding the to-be-encoded video segment based on the plurality of target bit rate coefficients to obtain a plurality of target video segments; A video encoding method comprising:

2. The spatio-temporal domain feature information includes spatio-temporal complexity information, and calculating the spatio-temporal domain feature information of the video segment to be encoded based on the one or more video frames includes: calculating gray level co-occurrence information and normalization information of the video segment to be encoded as spatio-temporal complexity information based on the one or more video frames, wherein the gray level co-occurrence information is used to describe the spatial complexity of the video segment to be encoded and the normalization information is used to describe the temporal complexity of the video segment to be encoded; combining the gray level co-occurrence information and normalization information to generate the spatio-temporal complexity information for the video segment to be encoded.

3. The spatio-temporal domain feature information includes encoding feature information, and the step of calculating the spatio-temporal domain feature information of the video segment to be encoded based on the one or more video frames includes: performing pre-processing on the one or more video frames to generate a new video frame sequence comprising a set of video frame pairs; and performing intra-frame coding and inter-frame coding on each video frame pair of the set of video frame pairs to obtain the coding feature information of the video segment to be coded.

4. The step of pre-processing the one or more video frames to generate a new video frame sequence comprises: copying video frames other than the first frame and the last frame of the one or more video frames to generate one or more copy video frames; and reordering the one or more video frames and the one or more copy video frames such that each copy video frame of the one or more copy video frames is located after a corresponding video frame.

5. The step of performing intra-frame coding and inter-frame coding on each video frame pair of the set of video frame pairs and obtaining the coding feature information of the video segment to be coded includes: performing intraframe coding on a first video frame in each video frame pair of the set of video frame pairs to obtain intraframe coding information; performing inter-frame coding on each video frame pair of the set of video frame pairs to obtain inter-frame coding information; and obtaining the coding feature information based on the intraframe coding information and the interframe coding information.

6. The method according to claim 3 , wherein the coding feature information is based on one or more of the number of coding bits, the intra-frame prediction mode ratio, and amplitude distribution information of inter-frame motion vectors.

7. determining a plurality of predicted bit rate coefficients for the video segment to be encoded in a plurality of preset video encoding standards based on the spatio-temporal domain feature information, 7. The method of claim 1, further comprising: inputting the spatio-temporal domain feature information of the video segment to be encoded into a first bitrate coefficient prediction model; and determining the plurality of predicted bitrate coefficients of the video segment to be encoded in the plurality of preset video coding standards using the first bitrate coefficient prediction model.

8. determining a plurality of target bitrate coefficients for the video segment to be encoded in the plurality of preset video encoding standards based on a first predicted bitrate coefficient corresponding to a first preset video encoding standard among the plurality of preset video encoding standards and a preset video encoding score interval, comprising: encoding the to-be-encoded video segment a first time based on the first predicted bit rate coefficient to obtain a first pre-encoded video segment in the first preset video encoding standard; determining whether the video coding score of the first pre-encoded video segment is within the preset video coding score interval; in response to the video encoding score of the first pre-encoded video segment being within the preset video encoding score interval; and determining said plurality of predicted bit rate coefficients as said plurality of target bit rate coefficients.

9. determining a plurality of target bitrate coefficients for the video segment to be encoded in the plurality of preset video encoding standards based on a first predicted bitrate coefficient corresponding to a first preset video encoding standard among the plurality of preset video encoding standards and a preset video encoding score interval, comprising: in response to the video encoding score of the first pre-encoded video segment not being within the preset video encoding score interval; 9. The method of claim 8, further comprising updating the plurality of predicted bit rate coefficients to obtain the plurality of target bit rate coefficients based on spatio-temporal domain feature information of the video segment to be encoded, the plurality of predicted bit rate coefficients, and a video encoding score of the first pre-encoded video segment.

10. updating the plurality of predicted bitrate coefficients to obtain the plurality of target bitrate coefficients based on the spatio-temporal domain feature information of the video segment to be encoded, the plurality of predicted bitrate coefficients, and the video encoding score of the first pre-encoded video segment, 10. The method of claim 9, further comprising: inputting the spatiotemporal domain feature information of the video segment to be encoded, the plurality of predicted bit rate coefficients, and the video encoding score of the first pre-encoded video segment into the second bit rate coefficient prediction model; and determining the plurality of target bit rate coefficients of the video segment to be encoded in the plurality of preset video encoding standards using the second bit rate coefficient prediction model.

11. The first bitrate coefficient prediction model is for each preset video encoding standard in the plurality of preset video encoding standards: Obtaining sample spatio-temporal domain feature information of a sample video segment and a sample bit rate coefficient of the sample video segment in the preset video encoding standard; inputting the sample spatio-temporal domain feature information into the first bitrate coefficient prediction model, and determining a sample predicted bitrate coefficient of the sample video segment in the preset video coding standard through the first bitrate coefficient prediction model; calculating a first model loss value based on the sample bit rate coefficients and the sample predicted bit rate coefficients; and adjusting parameters of the first bitrate coefficient prediction model based on the first model loss value for each of the preset video coding standards until a first model training stopping condition is achieved.

12. The second bitrate coefficient prediction model is inputting the sample spatio-temporal domain feature information, the sample predicted bitrate coefficients for each of the preset video encoding standards, and first sample video encoding scores corresponding to the sample predicted bitrate coefficients for the first preset video encoding standard into the second bitrate coefficient prediction model, and determining a plurality of sample target bitrate coefficients for the sample video segment in the plurality of preset video encoding standards according to the second bitrate coefficient prediction model; calculating, for each preset video encoding standard in the plurality of preset video encoding standards, a second model loss value based on the sample bit rate coefficient and a sample target bit rate coefficient corresponding to the preset video encoding standard; and adjusting parameters of the second bitrate coefficient prediction model based on the second model loss value for each of the preset video coding standards until a second model training stopping condition is achieved.

13. The step of obtaining a video segment to be encoded includes: obtaining a video to be encoded; performing scene detection on the to-be-encoded video; Dividing the to-be-encoded video into one or more sub-video segments based on scene detection results; and identifying each sub-video segment in the one or more video segments as a video segment to be encoded.

14. for each preset video encoding standard in the plurality of preset video encoding standards, obtaining all target video segments in the preset video encoding standard; 14. The method of claim 13, further comprising: combining all the target video segments based on an order of the one or more sub-video segments in the to-be-encoded video to obtain a target video corresponding to the to-be-encoded video in the preset video encoding standard.

15. a capture module arranged to capture a video segment to be encoded comprising one or more video frames; a calculation module configured to calculate spatiotemporal domain feature information of the video segment to be encoded based on the one or more video frames; a predicted bitrate coefficient determination module configured to determine a plurality of predicted bitrate coefficients of the video segment to be encoded in a plurality of preset video coding standards based on the spatio-temporal domain feature information; a target bitrate coefficient determination module configured to determine a plurality of target bitrate coefficients for the video segment to be encoded in the plurality of preset video encoding standards based on a first predicted bitrate coefficient corresponding to a first preset video encoding standard among the plurality of preset video encoding standards and a preset video encoding score interval, wherein the target bitrate coefficient determination module determines a shortest time for encoding video using the first preset video encoding standard among the plurality of preset video encoding standards; an encoding module configured to encode the to-be-encoded video segment based on the plurality of target bit rate coefficients to obtain a plurality of target video segments; 1. A video encoding device comprising:

16. at least one processor; at least one memory communicatively connected to the at least one processor, 15. An electronic device, wherein said at least one memory stores a computer program which, when executed by said at least one processor, implements the method of any one of claims 1 to 14.

17. A non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any one of claims 1 to 14.

18. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1 to 14.

Images