Video transcoding method and device, electronic equipment, storage medium, program product and method for generating bit stream

Abstract

The invention provides a video transcoding method and device, electronic equipment, a storage medium, a program product and a method for generating a bit stream. The video transcoding method comprises the following steps: extracting video features of currently received video clip data; processing the video features by using a pre-trained prediction model to obtain image quality code rate mapping information of the video clip data, the image quality code rate mapping information being used for representing a mapping relationship between an image quality index and a code rate; determining an optimal coding parameter of the video clip data according to the image quality code rate mapping information; and transcoding the video clip data according to the preferred coding parameters to obtain video stream data corresponding to the video clip data.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of video processing. More particularly, the present disclosure relates to a video transcoding method and device, an electronic device, a storage medium, a program product, and a method of generating a bitstream. BACKGROUND

[0002] With the development of network technology, live streaming has become one of the most influential application scenarios in the multimedia field. In order to serve millions, tens of millions, or even hundreds of millions of heterogeneous network users, most content providers provide users with a variety of live streaming content of different bit rates and resolutions for users to choose from, and provide adaptive multi-bit rate algorithms to help users dynamically select different bit rates and resolution levels under different network conditions to maximize the resolution and smoothness experience of users.

[0003] However, in the live streaming scenario, on the one hand, due to the real-time requirement of live streaming content, different bit rates and resolution levels need to be transcoded in real time, and on the other hand, due to the limitations of the live streaming link system, the live streaming content may change frequently, and the system cannot perceive the complexity change of the live streaming content in advance. Therefore, the combination of different bit rates and resolution levels in live streaming is usually a fixed combination designed through offline analysis. However, this method cannot be widely adapted to different live streaming scenarios, especially in the case of large changes in live streaming content, and cannot be well adapted. SUMMARY

[0004] Embodiments of the present disclosure provide a video transcoding method, device, electronic device, storage medium, program product, and method of generating a bitstream, for solving at least one of the above problems.

[0005] According to an aspect of the present disclosure, a video transcoding method is provided, the video transcoding method comprising: extracting a video feature of currently received video segment data; processing the video feature using a pre-trained prediction model to obtain quality-rate mapping information of the video segment data, wherein the quality-rate mapping information is used to represent a mapping relationship between a quality index and a bit rate; determining preferred encoding parameters of the video segment data according to the quality-rate mapping information; and transcoding the video segment data according to the preferred encoding parameters to obtain video stream data corresponding to the video segment data.

[0006] Optionally, the prediction model is trained through the following steps: constructing a sample data set, wherein the sample data set includes sample video clip data of multiple different content types and multiple different bit rates; extracting the video features of each sample video clip data, and determining the actual picture quality index of each sample video clip data; using the video features as input parameters of the prediction model to be trained, and using the mapping relationship between the actual picture quality index and the bit rate as the training target of the prediction model to be trained, training the prediction model to be trained, and obtaining the prediction model.

[0007] Optionally, determining the preferred encoding parameters of the video clip data based on the image quality bit rate mapping information includes: for each clarity level among a plurality of clarity levels, determining at least one group of candidate encoding parameters for the clarity level according to the image quality bit rate mapping information, wherein each group of candidate encoding parameters includes an image quality index, a bit rate, and corresponding other encoding parameters; for each clarity level among a plurality of clarity levels, determining a group of candidate encoding parameters from the at least one group of candidate encoding parameters as preferred encoding parameters for the clarity level, so that the comprehensive encoding parameters of the plurality of clarity levels meet a balance condition, wherein the comprehensive encoding parameters are statistical values ​​of the preferred encoding parameters of the plurality of clarity levels, and the balance condition indicates that when the image quality index in the comprehensive encoding parameters meets a preset image quality requirement, the bit rate in the comprehensive encoding parameters reaches a minimum.

[0008] Optionally, the comprehensive encoding parameter is a weighted average of the preferred encoding parameters of the multiple definition levels, and the weight of each definition level is obtained by statistically analyzing the network quality distribution at the viewer end.

[0009] Optionally, the video features include at least one of the following: content features, coding features, wherein the content features include at least one of the following: content type features, complexity features, aesthetic features, the content type features include at least one of the following: motion scenes, static scenes, the complexity features include at least one of the following: motion vectors, color change features, the aesthetic features include at least one of the following: composition features, color matching features, and the coding features include at least one of the following: bit rate, noise.

[0010] Optionally, the prediction model includes at least one of the following: XGBoost model, random forest, support vector machine, neural network, reinforcement learning network.

[0011] Optionally, the image quality index includes at least one of the following: an objective image quality index and a subjective image quality index, and the objective image quality index includes a distortion index.

[0012] Optionally, the preferred encoding parameters include at least one of the following parameters: constant rate factor, resolution, bit rate, and sharpening effect parameter.

[0013] According to another aspect of the present disclosure, a video transcoding device is provided, comprising: an extraction unit configured to extract video features of currently received video clip data; a processing unit configured to process the video features using a pre-trained prediction model to obtain image quality bitrate mapping information of the video clip data, wherein the image quality bitrate mapping information is used to represent a mapping relationship between image quality indicators and bitrate; a determination unit configured to determine preferred encoding parameters of the video clip data based on the image quality bitrate mapping information; and a transcoding unit configured to perform transcoding processing on the video clip data according to the preferred encoding parameters to obtain video stream data corresponding to the video clip data.

[0014] Optionally, the prediction model is trained through the following steps: constructing a sample data set, wherein the sample data set includes sample video clip data of multiple different content types and multiple different bit rates; extracting the video features of each sample video clip data, and determining the actual picture quality index of each sample video clip data; using the video features as input parameters of the prediction model to be trained, and using the mapping relationship between the actual picture quality index and the bit rate as the training target of the prediction model to be trained, training the prediction model to be trained, and obtaining the prediction model.

[0015] Optionally, the determination unit is further configured to: for each of the multiple clarity levels, determine at least one group of candidate coding parameters for the clarity level based on the picture quality bit rate mapping information, wherein each group of candidate coding parameters includes picture quality indicators, bit rate and corresponding other coding parameters; for each of the multiple clarity levels, determine a group of candidate coding parameters from the at least one group of candidate coding parameters as preferred coding parameters for the clarity level, so that the comprehensive coding parameters of the multiple clarity levels meet a balance condition, wherein the comprehensive coding parameters are statistical values ​​of the preferred coding parameters of the multiple clarity levels, and the balance condition indicates that when the picture quality indicators in the comprehensive coding parameters meet the preset picture quality requirements, the bit rate in the comprehensive coding parameters reaches the minimum.

[0016] Optionally, the comprehensive encoding parameter is a weighted average of the preferred encoding parameters of the multiple definition levels, and the weight of each definition level is obtained by statistically analyzing the network quality distribution at the viewer end.

[0017] Optionally, the video features include at least one of the following: content features, coding features, wherein the content features include at least one of the following: content type features, complexity features, aesthetic features, the content type features include at least one of the following: motion scenes, static scenes, the complexity features include at least one of the following: motion vectors, color change features, the aesthetic features include at least one of the following: composition features, color matching features, and the coding features include at least one of the following: bit rate, noise.

[0018] Optionally, the prediction model includes at least one of the following: XGBoost model, random forest, support vector machine, neural network, reinforcement learning network.

[0019] Optionally, the image quality index includes at least one of the following: an objective image quality index and a subjective image quality index, and the objective image quality index includes a distortion index.

[0020] Optionally, the preferred encoding parameters include at least one of the following parameters: constant rate factor, resolution, bit rate, and sharpening effect parameter.

[0021] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer-executable instructions, wherein when the computer-executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the video transcoding method described above.

[0022] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer-executable instructions, wherein when the computer-executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the video transcoding method described above.

[0023] According to another aspect of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the video transcoding method as described above.

[0024] According to another aspect of the present disclosure, a computer program product is provided, comprising computer instructions, which implement the video transcoding method described above when the computer instructions are executed by at least one processor.

[0025] According to another aspect of the present disclosure, a method for generating a bitstream is provided, comprising: generating the bitstream according to the video transcoding method described above.

[0026] According to the video transcoding method, device, electronic device, storage medium, program product and method for generating a bitstream according to the exemplary embodiments of the present disclosure, by analyzing the video features of each video clip data multiple times during the transcoding process, and using a prediction model to obtain corresponding image quality bit rate mapping information, and then determining the preferred encoding parameters for different clarity levels, and performing transcoding processing based on this, it is possible to dynamically optimize the image quality and bit rate for specific video content, thereby improving the user's overall viewing experience.

[0027] It is to be understood that both the foregoing general description and the following detailed description are examples only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate examples according to the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0029] Figure 1 is a flowchart illustrating an exemplary video transcoding method according to some embodiments of the present disclosure.

[0030] Figure 2 2 is a schematic diagram showing the transcoding process of a live broadcast system according to a specific embodiment of the present disclosure.

[0031] Figure 3 3 is a flow chart showing a video transcoding method according to a specific embodiment of the present disclosure.

[0032] Figure 4 is a block diagram illustrating an exemplary video transcoding apparatus according to some embodiments of the present disclosure.

[0033] Figure 5 is a diagram illustrating a computing environment coupled with a user interface according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0034] Reference will now be made in detail to the specific embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to facilitate understanding of the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein may be implemented on many types of electronic devices with digital video capabilities.

[0035] It should be noted that the terms "first," "second," and the like in the description, claims, and drawings of the present disclosure are used to distinguish between objects, and are not used to describe any specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments of the present disclosure described herein can be implemented in an order other than that shown in the drawings or described in the present disclosure.

[0036] With the development of network technology, live streaming has become one of the most influential application scenarios in the multimedia field. To serve millions, tens of millions, or even hundreds of millions of users across heterogeneous networks, most content providers offer a variety of live streaming content with different bitrates and resolutions for users to choose from. They also provide adaptive multi-bitrate algorithms to help users dynamically select different bitrates and resolutions under varying network conditions, maximizing the user experience in terms of clarity and smoothness.

[0037] However, in live streaming scenarios, due to the real-time requirements of live content, different bitrates and resolution levels require real-time transcoding and output. Furthermore, due to limitations in the live streaming link system, live content can frequently change significantly, making it impossible for the system to proactively detect changes in content complexity. Therefore, the combinations of different bitrates and resolution levels used in live streaming are typically fixed combinations designed through offline analysis. While most content providers use a constant rate factor (CRF) approach for bitrate control, combining different CRF values ​​with resolution levels and adjusting the bitrate based on the current CRF value while maintaining image quality, this approach is still not widely adaptable to diverse live streaming scenarios and struggles with significant content variability. Specifically, when live streaming scenarios are complex (such as outdoor scenarios), overly conservative encoding level design can lead to insufficient bitrate allocation, resulting in poor live streaming clarity. When live streaming scenarios are less complex (such as indoor live streaming), excessively high bitrates can be used, resulting in wasted bitrate and indirectly impacting the smoothness of the video for viewers with weaker network connections. In addition, due to the use of fixed-speed bitrate control rules, the gradient settings of different clarity levels of live broadcasts may be unreasonable in different live content scenarios. If the gear gradient is too large in some scenarios, the adaptive multi-bitrate algorithm on the audience side will not have a suitable intermediate gear to choose from under some network conditions, resulting in gear jumps, and gaps and large jumps in the user clarity experience. If the gear gradient is too small in some scenarios, the adaptive multi-bitrate algorithm will not be able to hit the intermediate gear under some network conditions, and the user experience clarity level difference will be small, and there will be gear waste. At this time, although the bitrate has been optimized, the user experience has not been significantly improved, and the picture quality may even degrade. It can be seen that the rationality of the gear setting directly affects the upper limit of the user experience and the upper limit of cost optimization under the adaptive multi-bitrate algorithm on the audience side.

[0038] One of the most core issues in the audio and video field is how to more accurately design different bitrate and clarity levels for live content, given limited resources (transcoding and storage resources), to optimize the overall experience for all users. To improve user clarity in live broadcast scenarios and reduce bandwidth costs, this disclosure proposes an adaptive dynamic level decision method based on live content, compared to the existing fixed level schemes in live broadcast scenarios, which set a fixed level logic before the live broadcast begins. This method dynamically predicts image quality bitrate mapping information based on the live content during the live broadcast and adjusts the preferred bitrate parameters, specifically the bitrate parameters for different clarity levels. This method dynamically adjusts the level structure distribution to make it more reasonable, thereby optimizing bitrate usage and improving user experience while ensuring image quality. This disclosure not only addresses the issue of fluctuating user clarity in live broadcasts caused by unreasonable level gradient settings under fixed bitrate control rules, but also combines it with an adaptive multi-bitrate algorithm to improve overall live broadcast clarity while reducing bandwidth costs, thereby achieving targeted optimization.

[0039] In addition, by considering the real-time changes in the audience's network quality distribution when adjusting the preferred bitrate parameters, the gear distribution can be further optimized, taking into account the overall clarity and bandwidth cost of the user's live broadcast.

[0040] Figure 1 This is a flowchart illustrating an exemplary video transcoding method according to some embodiments of the present disclosure. This video transcoding method can be implemented in an electronic device with sufficient computing power. It should be noted that the video transcoding method is used to transcode encoded video data into multiple versions with different bit rates and resolutions for selective playback by viewers, thereby meeting the viewing needs of different viewers.

[0041] Reference Figure 1 In step S101, video features of the currently received video clip data are extracted.

[0042] This step specifically receives the video segment data of the encoded video data. For example, in a live broadcast scenario, the video stream continuously shot and encoded by the host can be received. The currently received video segment data is the video stream segment currently received from the host, so that it can be executed multiple times during the push process of the host. Figure 1 The video transcoding method shown is used to dynamically adjust the transcoding frame rate.

[0043] For example, video features include at least one of the following: content features and encoding features. These features are extracted from two dimensions: the video's content and the encoding process performed by the video capture device. Specifically, content features include at least one of the following: content type features, complexity features, and aesthetic features. Content type features include at least one of the following: moving scenes or static scenes. Complexity features include at least one of the following: motion vectors (e.g., global motion vectors of objects / content in the video), color change features (e.g., inter-frame difference metrics such as inter-frame color distance, dynamic range metrics such as color gamut coverage, and temporal features such as color trend coefficients). Aesthetic features include at least one of the following: composition features (e.g., classic compositional features such as the rule of thirds, element coordinates, subject area proportion, and spatial dispersion), and color matching features (e.g., color relationship types such as contrasting color matching, color emotional labels such as highly saturated colors, and color harmony metrics). Content type features provide a simple summary of the content characteristics of video clip data. Complexity features further describe the changes in the video image. Aesthetic features describe the video content from the user's sensory perspective. These multi-dimensional content features help enrich the content of the video. Coding features include at least one of the following: bitrate and noise. These features reflect the encoding process performed by the video capture terminal and provide additional dimensions for describing the video. This multi-dimensional feature analysis method provides a more comprehensive description of the video's characteristics, providing richer data support for subsequent frame rate optimization decisions.

[0044] In step S102, a pre-trained prediction model is used to process video features to obtain quality bitrate mapping information of the video segment data.

[0045] The image quality bitrate mapping information is used to represent the mapping relationship between image quality indicators and bitrates. For example, it can be an image quality bitrate curve, where the horizontal axis of the curve is the bitrate and the vertical axis is the image quality indicator. In this way, the image quality indicators and bitrates of different clarity levels can be restricted in combination with the mapping relationship, thereby realizing the determination of various encoding parameters including bitrates of different clarity levels. An example of an image quality bitrate curve is an RD curve (Rate-Distortion Curve), where the vertical axis is distortion, which refers to the degree of difference between the video and the original video after compression encoding (usually manifested as image quality loss, such as blurring, color blocks, loss of details, etc.). The smaller the distortion, the closer the video quality is to the original picture. It should be understood that distortion is an objective image quality indicator. In addition, the image quality indicator can also adopt a subjective image quality indicator to reflect the subjective image quality perceived by the user when watching the video.

[0046] For different videos, when using the same bitrate, the video quality often varies. By using a pre-trained prediction model to analyze video features, we can obtain quality-bitrate mapping information suitable for the current video clip data.

[0047] Specifically, taking the picture quality-rate mapping information as an example of the picture quality-rate curve, when actually determining the encoding parameters of different definition levels, a cluster of curves will be used instead of only one curve, for example, resolution, encoding speed (preset), encoder, pre / post-processing mode, and each value of each of these variables can correspond to a curve, so that multiple curves can be obtained. A combination of a rate and a definition level needs to be a point on these multiple curves (this point includes both the picture quality indicator of the vertical coordinate and the rate of the horizontal coordinate, and other encoding parameters corresponding to the point). In other words, determining the combination of the rate and the definition level is to determine multiple points (one point for each level) from the multiple curves, and different points can come from the same curve or from different curves, and these curves collectively serve as the source of the points.

[0048] On this basis, the related technology uses static curves for different live videos, that is, the same cluster of curves is used for different live videos, and multiple points (usually 5, corresponding to the 5 definition levels of the video, namely, smooth, standard definition, high definition, 4k, and Blu-ray) are selected from this cluster of curves in advance to obtain 5 fixed combinations of rates and definitions, or at most, the combinations of rates and definitions for live videos of the same content type are simply distinguished according to the content type.

[0049] The present disclosure is to dynamically predict the picture quality-rate mapping information of the current video segment data. For the embodiment using the picture quality-rate curve, that is, to dynamically predict the optimal cluster of curves for the video data.

[0050] In step S103, the preferred encoding parameters of the video segment data are determined according to the picture quality-rate mapping information.

[0051] Optionally, step S103 includes: for each of the multiple definition levels, determining at least one set of candidate coding parameters for the definition level based on the image quality-to-bitrate mapping information, wherein each set of candidate coding parameters includes an image quality indicator, a bitrate, and corresponding other coding parameters; and for each of the multiple definition levels, determining a set of candidate coding parameters from the at least one set of candidate coding parameters as preferred coding parameters for the definition level, such that the combined coding parameters of the multiple definition levels meet a balance condition, wherein the combined coding parameters are statistical values ​​(e.g., including but not limited to an average, a weighted average, a median, etc.) of the preferred coding parameters of the multiple definition levels, and the balance condition indicates that the bitrate of the combined coding parameters is minimized when the image quality indicators in the combined coding parameters meet a preset image quality requirement. This means that it is necessary to perform trial combinations of the candidate coding parameters for each definition level to find the optimal combination that meets the balance condition, thereby obtaining the preferred coding parameters for each definition level. By comprehensively analyzing the encoding parameters of multiple deficiencies and combining them with balance conditions, the encoding parameters of different deficiencies can be optimized as a whole, thereby improving the overall clarity of user live broadcasts while reducing live broadcast bandwidth costs, thereby achieving targeted optimization.

[0052] As an example, the comprehensive encoding parameter is a weighted average of the preferred encoding parameters of multiple definition levels, and the weight of each definition level is obtained by counting the network quality distribution at the viewer end. It should be understood that the better the network quality at the viewer end, the more likely it is to select a high-definition level. Therefore, the probability of each definition level being selected by the user is likely to be similar to the distribution of different network qualities at the viewer end. By taking statistics on the network quality distribution at the viewer end and determining the weight for calculating the weighted average based on the statistical results, it is possible to fully take into account the real-time changes in the network quality distribution of the viewer, thereby further optimizing the level distribution in a targeted manner.

[0053] For example, the network speed of the audience end can be clustered to obtain multiple continuously distributed network speed intervals and the proportion of the number of audience ends in each network speed interval. The proportion of the number of audience ends is the ratio of the number of audience ends in the network speed interval to the number of all audience ends counted. Each network speed interval is then mapped to a clarity level to obtain the weight of the clarity level. For example, when the number of network speed intervals is equal to the number of clarity levels, the two can form a one-to-one correspondence, and the proportion of the number of audience ends in the high network speed interval is used as the weight of the high-definition level; when the number of network speed intervals is greater than the number of clarity levels, adjacent network speed intervals can be merged so that the number of merged network speed intervals is equal to the number of clarity levels; when the number of network speed intervals is less than the number of clarity levels, adjacent clarity levels can be made to use the same encoding parameters, which is equivalent to merging adjacent clarity levels. In the last case, it should be understood that to ensure a stable display of resolution levels on the user side, the number of resolution levels displayed on the user side may not change. However, when the user switches between resolution levels that have already been merged, the resolution version of the pulled video stream data remains unchanged. It should also be understood that the method for calculating the network quality distribution at the viewer side described here is merely an example; other reasonable methods may also be used, and this disclosure does not limit this.

[0054] In addition to the above methods, heuristic optimization methods such as genetic algorithms (Genetic Algorithms) or particle swarm optimization (PSO) may also be used to determine the optimal encoding parameters, and the present disclosure does not impose any restrictions on this.

[0055] As an example, the preferred encoding parameters include at least one of the following parameters: constant rate factor, resolution, bit rate, and sharpening effect parameters. By dynamically adjusting the constant rate factor, it is possible to maintain high image quality even when network bandwidth is limited, thereby improving the user's viewing experience. Sharpening effect parameters include, but are not limited to, parameters such as SharpenAltD and SharpenAltUd in the HiSilicon platform. SharpenAltD is used to control the sharpness of large edges. Increasing this parameter will make the white edges more obvious, but the picture will look sharper. SharpenAltUd is used to control details. Increasing this parameter will result in a significant increase in noise, and it is recommended to lower it under low illumination.

[0056] In step S104, the video segment data is transcoded according to the preferred encoding parameters to obtain video stream data corresponding to the video segment data.

[0057] It should be understood that during the transcoding process, multiple definition versions of the video stream data are transcoded based on the preferred encoding parameters of the multiple definition levels. It should also be understood that the transcoded video stream data will be sent to the user terminal for decoding and playback by the user terminal, but not all definition versions will be sent. Instead, the video stream data corresponding to the definition level selected by the user terminal will be sent to the user terminal.

[0058] Next, a video transcoding method according to an exemplary embodiment of the present disclosure is further introduced.

[0059] In some embodiments, optionally, during the process of receiving video data, the Figure 1 The process shown in the figure allows for real-time streaming data analysis and instant adjustment of optimal encoding parameters. For example, a stream processing framework such as Apache Kafka or Spark Streaming can be used to process live streaming data in real time and dynamically adjust encoding parameters based on the characteristics of the current content.

[0060] In other embodiments, step S101 is optionally executed in response to a preset condition being met, which means that in the process of receiving video data, each time the preset condition is met, step S101 needs to be executed. Figure 1 The process shown is to update the preferred encoding parameters and perform subsequent transcoding processing according to the updated preferred encoding parameters. This can not only more reliably achieve the dynamic update of the preferred encoding parameters, but also control the update frequency of the preferred encoding parameters by reasonably setting preset conditions, reserving sufficient processing time for related calculations, and meeting the real-time requirements of video transcoding.

[0061] As an example, each time step S101 is executed, the extracted data may be a recent video segment of a certain length, or all video segment data received from the last execution of step S101 to the current execution of step S101, or a transcoding template segment extracted from all the video segment data.

[0062] As an example, the preset condition includes reaching a preset interval length, that is, the time elapsed between the last execution of step S101 and the current execution of step S101 reaches the preset interval length. The preset interval length can remain unchanged during the process of receiving video data (it can be the same or different for different video data), for example, but not limited to 5s or 10s, to achieve regular and stable updates of the preferred coding parameters; it can also be negatively correlated to the degree of change of the preferred coding parameters (for example, it can be a statistical value of the absolute value of the percentage change of each coding parameter). In this case, if the change of the preferred coding parameters is more significant, it indicates that there is a higher demand for updating the preferred coding parameters, and the update interval length can be shortened. Conversely, if the change of the preferred coding parameters is smaller, it indicates that there is a lower demand for updating the preferred coding parameters, and the update interval length can be extended to save computing power.

[0063] Optionally, the prediction model is trained by the following steps: constructing a sample data set, wherein the sample data set includes sample video clip data of multiple different content types and multiple different bitrates; extracting video features of each sample video clip data and determining the actual image quality index of each sample video clip data; using the video features as input parameters of the prediction model to be trained, using the mapping relationship between the actual image quality index and the bitrate as the training target of the prediction model to be trained, and training the prediction model to be trained to obtain the prediction model. By collecting sample video clip data of different bitrates, determining their actual image quality index as the training target of the prediction model to be trained, and using the video features of the sample video clips as model input parameters, the prediction model can be guided to learn how to predict the mapping relationship between the bitrate and the image quality index based on the video features. Specifically, each sample video clip data has its unique content type and bit rate. The sample data set includes sample video clip data of multiple different content types and multiple different bit rates, which means that the sample data set includes sample video clip data of multiple different content types, and the sample video clip data of the same content type covers multiple different bit rates, and can even include multiple sample video clip data with exactly the same content but different bit rates, so that the model can learn the impact of bit rate on image quality in a targeted manner.

[0064] As an example, the actual quality indicator can be an objective quality indicator, for example including but not limited to distortion indicators. The actual quality indicator can also be a subjective quality indicator, and can be an indicator used in the related art to measure subjective quality, or can be obtained by designing and implementing a subjective survey experiment, for example, different users can be asked to play different sample video segment data of different scenes, content categories, and picture complexity under different bit rates, and these users can be asked to score the played video segments, and different scores of the same sample video segment data can be cleaned, counted, and processed, and finally the statistical value obtained is taken as the actual quality indicator of the sample video segment, which can reduce the influence of individual differences. Such a subjective quality indicator pays more attention to the actual viewing experience of users, and thus can provide an optimization scheme closer to user needs.

[0065] As an example, cross-validation, hyperparameter tuning, and other methods can also be used in training to improve the performance of the prediction model.

[0066] As an example, the prediction model includes at least one of the following: an XGBoost model, a random forest, a support vector machine (SVM), a neural network, and a reinforcement learning network. These models can all achieve prediction of quality-bit rate mapping information, and can be selected as needed to improve the flexibility of the scheme. The XGBoost model has strong fitting ability and efficient training speed, which helps to more accurately predict the quality indicators of different videos under different bit rates, thereby guiding the optimization decision of the encoding parameters. It should be understood that the XGBoost model can be used to perform classification tasks, regression tasks, and ranking tasks, and for the present disclosure, the regression task is specifically performed. The random forest can be used for classification and regression tasks, and can improve the accuracy and robustness of prediction by integrating multiple decision trees. The support vector machine is suitable for data classification and regression tasks in high-dimensional space, and can handle nonlinear relationships. Deep learning algorithms, such as convolutional neural networks (CNNs), are particularly suitable for feature extraction of image and video data, and can capture local and global features; recurrent neural networks (RNNs) are suitable for processing sequence data and can capture temporal dependencies. Reinforcement learning networks, such as Q-learning, learn the optimal policy through trial and error, and are suitable for decision optimization in dynamic environments; deep Q-networks (DQNs) combine deep learning and reinforcement learning, and are suitable for decision optimization in complex environments.

[0067] Next, taking a live streaming scenario as an example, the following Figure 2 and Figure 3A video transcoding method according to a specific embodiment of the present disclosure is introduced.

[0068] In this specific embodiment, Figure 2 As shown in the figure, the live streaming system includes the host, the origin server, the Content Delivery Network (CDN), and the viewer. When the host pushes the stream—that is, the host sends the captured and encoded video stream to the origin server—the origin server can perform transcoding scheduling and initiate a bypass task to calculate real-time video features. It then uses a prediction model to process these features to obtain the image quality and bitrate mapping information for the current video clip. Based on this information, it makes encoding parameter optimization decisions. Finally, it transcodes the video stream sent by the host according to the optimal encoding parameters. The transcoded video stream is then output to the CDN and the viewer, ensuring users receive the best viewing experience.

[0069] like Figure 3 As shown, this specific embodiment provides an adaptive dynamic gear decision method based on live content. By analyzing the characteristics of live content in real time and dynamically adjusting transcoding parameters, it optimizes bitrate usage while ensuring image quality and improves user experience. The system architecture includes a decision model and an adjustment module. The method includes the following aspects.

[0070] 1. Building an offline dataset First, we constructed an offline dataset containing a variety of live content types. These types of content include, but are not limited to, gaming competitions, outdoor activities, and livestreaming e-commerce. By analyzing these types of content, we extracted the content features and encoding features of the videos. The specific features are described above and will not be repeated here.

[0071] 2. Train the XGBoost model.

[0072] Using the aforementioned offline dataset, an XGBoost model was trained as a prediction model to predict the optimal RD curve for different video features, which serves as the quality-to-bitrate mapping information. This machine learning-based approach not only improves prediction accuracy but also enhances the system's generalization capabilities, enabling it to handle a wider range of live broadcast scenarios. Cross-validation and hyperparameter tuning can further improve the model's prediction accuracy and generalization capabilities.

[0073] 3. Real-time transcoding scheduling.

[0074] During the actual operation of the system, when the anchor pushes the stream, transcoding scheduling is triggered. The system deploys bypass tasks to perform real-time feature calculation and extraction to obtain the feature data required by the model. These feature data include the initial encoding parameters, content features, encoding features, etc. of the video. It should be noted that the encoding parameters are used to guide encoding, and specific encoding features will be output after encoding. The initial encoding parameters are the encoding parameters when the system starts running, and the result directly affected is the final encoding features of the video, which will be used as model input. This specific embodiment does not input encoding parameters into the prediction model, but the actual solution is not limited to this, and encoding parameters can also be input.

[0075] 4. Predict RD curve.

[0076] The prediction model uses feature data input to predict the optimal RD curve. This process takes into account the dynamic changes in video content, ensuring that the prediction results can reflect the actual needs of the current live content.

[0077] 5. Coding parameter optimization decision.

[0078] The decision module uses optimization methods to calculate the encoding parameters that achieve the best trade-off between image quality and bitrate. These parameters include, but are not limited to, CRF value, resolution, bitrate, and sharpening parameters.

[0079] 6. Adjust transcoding parameters.

[0080] The decision module transparently transmits the optimal decision result to the adjustment module. The adjustment module performs corresponding transcoding based on these preferred encoding parameters, ensuring that the output video stream has an optimized bit rate while maintaining image quality. It also enables flexible dynamic parameter adjustment while ensuring that the live broadcast content is not interrupted or jumpy for the audience consumption side.

[0081] 7. Output transcoded stream.

[0082] Finally, the transcoded video stream is output to the CDN and the viewer end, ensuring that users can get the best viewing experience.

[0083] This specific embodiment has the following advantages: it can adjust transcoding parameters in real time according to the dynamic changes of live broadcast content to ensure the optimal balance between picture quality and bit rate; by optimizing encoding parameters, it can improve the user's viewing experience and reduce freezes and picture quality degradation; while ensuring picture quality, it can optimize bit rate usage, reduce bandwidth costs, and improve resource utilization; through automated and intelligent dynamic adjustment, it reduces the need for manual intervention, making the transcoding process more efficient, which not only saves labor costs, but also shortens transcoding time and improves overall work efficiency; by introducing the XGBoost model and optimization method, it can achieve accurate analysis and dynamic adjustment of live broadcast content. This technological innovation not only solves current technical problems, but also lays the foundation for future technological innovation. By continuously optimizing and improving the model, the performance and effect of the system can be further improved.

[0084] Figure 4 is a block diagram illustrating an exemplary video transcoding apparatus according to some embodiments of the present disclosure.

[0085] Reference Figure 4 The video transcoding device 400 includes an extraction unit 401 , a processing unit 402 , a determination unit 403 , and a transcoding unit 404 .

[0086] The extraction unit 401 may extract video features of the currently received video segment data.

[0087] Optionally, the video features include at least one of the following: content features, coding features, wherein the content features include at least one of the following: content type features, complexity features, aesthetic features, content type features include at least one of the following: motion scene, static scene, complexity features include at least one of the following: motion vector, color change feature, aesthetic features include at least one of the following: composition feature, color matching feature, coding features include at least one of the following: bit rate, noise.

[0088] The processing unit 402 may use a pre-trained prediction model to process video features to obtain image quality and bit rate mapping information of the video segment data, wherein the image quality and bit rate mapping information is used to represent a mapping relationship between image quality indicators and bit rates.

[0089] Optionally, the prediction model includes at least one of the following: XGBoost model, random forest, support vector machine, neural network, reinforcement learning network.

[0090] Optionally, the image quality index includes at least one of the following: an objective image quality index and a subjective image quality index, and the objective image quality index includes a distortion index.

[0091] Optionally, the prediction model is trained through the following steps: constructing a sample data set, wherein the sample data set includes sample video clip data of multiple different content types and multiple different bit rates; extracting video features of each sample video clip data, and determining the actual picture quality index of each sample video clip data; using the video features as input parameters of the prediction model to be trained, and using the mapping relationship between the actual picture quality index and the bit rate as the training target of the prediction model to be trained, training the prediction model to be trained, and obtaining the prediction model.

[0092] The determining unit 403 may determine the preferred encoding parameters of the video segment data according to the image quality bit rate mapping information.

[0093] Optionally, the preferred encoding parameters include at least one of the following parameters: constant rate factor, resolution, bit rate, and sharpening effect parameter.

[0094] Optionally, the determination unit 403 may also: for each clarity level among the multiple clarity levels, determine at least one group of candidate coding parameters for the clarity level according to the picture quality bit rate mapping information, wherein each group of candidate coding parameters includes picture quality indicators, bit rate and corresponding other coding parameters; for each clarity level among the multiple clarity levels, determine a group of candidate coding parameters from at least one group of candidate coding parameters as preferred coding parameters for the clarity level, so that the comprehensive coding parameters of the multiple clarity levels meet the balance condition, wherein the comprehensive coding parameters are the statistical values ​​of the preferred coding parameters of the multiple clarity levels, and the balance condition indicates that when the picture quality indicators in the comprehensive coding parameters meet the preset picture quality requirements, the bit rate in the comprehensive coding parameters reaches the minimum.

[0095] Optionally, the comprehensive encoding parameter is a weighted average of preferred encoding parameters of multiple definition levels, and the weight of each definition level is obtained by statistically analyzing the network quality distribution at the viewer end.

[0096] The transcoding unit 404 may perform transcoding processing on the video segment data according to the preferred encoding parameters to obtain video stream data corresponding to the video segment data.

[0097] Regarding the apparatus in the above embodiment, the specific manner in which each unit performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0098] Figure 5 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.

[0099] Reference Figure 5The electronic device 500 includes at least one memory 501 and at least one processor 502, wherein the at least one memory 501 stores a set of computer-executable instructions. When the computer-executable instruction set is executed by the at least one processor 502, a video transcoding method according to an exemplary embodiment of the present disclosure is executed.

[0100] As an example, electronic device 500 may be a PC, tablet device, personal digital assistant, smartphone, or other device capable of executing the aforementioned set of instructions. Here, electronic device 500 is not necessarily a single electronic device, but may also be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction set) individually or in combination. Electronic device 500 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that interfaces with local or remote devices (e.g., via wireless transmission).

[0101] In electronic device 500, processor 502 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0102] The processor 502 can execute instructions or codes stored in the memory 501, wherein the memory 501 can also store data. Instructions and data can also be sent and received over a network via a network interface device, wherein the network interface device can use any known transmission protocol.

[0103] The memory 501 may be integrated with the processor 502, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. Furthermore, the memory 501 may comprise a separate device, such as an external disk drive, a storage array, or any other storage device usable by a database system. The memory 501 and the processor 502 may be operatively coupled or may communicate with each other, for example, via an I / O port, a network connection, or the like, such that the processor 502 can access files stored in the memory.

[0104] In addition, the electronic device 500 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 500 may be connected to each other via a bus and / or a network.

[0105] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium may also be provided, which, when the instructions in the computer-readable storage medium are executed by at least one processor, prompts the at least one processor to perform the video transcoding method according to the exemplary embodiment of the present disclosure. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as a multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device configured to store a computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be executed in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0106] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided. The computer program product includes computer instructions. When the computer instructions are executed by at least one processor, the at least one processor is prompted to perform the video transcoding method according to the exemplary embodiment of the present disclosure.

[0107] According to an exemplary embodiment of the present disclosure, a method for generating a bitstream may further be provided, including: generating a bitstream according to the video transcoding method of the exemplary embodiment of the present disclosure.

[0108] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0109] Unless otherwise specifically stated, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to actual circumstances. In addition, at least one of the steps of the method according to the present disclosure can be adjusted, combined, or deleted according to actual needs.

[0110] The examples are chosen and described in order to explain the principles of the present disclosure and to enable others skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications as are suited to the particular use contemplated. Therefore, it will be understood that the scope of the present disclosure is not limited to the specific examples of the embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the present disclosure.

Claims

1. A video transcoding method, characterized in that: The video transcoding method comprises: Extracting video features of the currently received video clip data; Processing the video features using a pre-trained prediction model to obtain image quality and bitrate mapping information of the video segment data, wherein the image quality and bitrate mapping information is used to represent a mapping relationship between image quality indicators and bitrate; determining, according to the image quality bit rate mapping information, preferred encoding parameters for the video segment data; The video segment data is transcoded according to the preferred encoding parameters to obtain video stream data corresponding to the video segment data.

2. The video transcoding method according to claim 1, wherein: The prediction model is trained through the following steps: Constructing a sample data set, wherein the sample data set includes sample video segment data of multiple different content types and multiple different bit rates; Extracting the video features of each sample video segment data, and determining an actual image quality index of each sample video segment data; The video features are used as input parameters of the prediction model to be trained, the mapping relationship between the actual image quality index and the bit rate is used as the training target of the prediction model to be trained, and the prediction model to be trained is trained to obtain the prediction model.

3. The video transcoding method according to claim 1, wherein: The determining, according to the image quality bit rate mapping information, the preferred encoding parameters of the video segment data includes: For each definition level among the plurality of definition levels, determining at least one set of candidate coding parameters for the definition level according to the image quality bit rate mapping information, wherein each set of candidate coding parameters includes an image quality indicator, a bit rate, and corresponding other coding parameters; For each of the multiple definition levels, a group of candidate coding parameters is determined from the at least one group of candidate coding parameters as the preferred coding parameters of the definition level, so that the comprehensive coding parameters of the multiple definition levels meet a balance condition, wherein the comprehensive coding parameters are statistical values ​​of the preferred coding parameters of the multiple definition levels, and the balance condition indicates that when the image quality indicators in the comprehensive coding parameters meet the preset image quality requirements, the bit rate in the comprehensive coding parameters reaches the minimum.

4. The video transcoding method according to claim 3, wherein: The comprehensive encoding parameter is a weighted average of the preferred encoding parameters of the multiple definition levels, and the weight of each definition level is obtained by statistically analyzing the network quality distribution at the viewer end.

5. The video transcoding method according to any one of claims 1 to 4, wherein: The video features include at least one of the following: content features, coding features, wherein the content features include at least one of the following: content type features, complexity features, and aesthetic features, the content type features include at least one of the following: motion scenes and static scenes, the complexity features include at least one of the following: motion vectors and color change features, the aesthetic features include at least one of the following: composition features and color matching features, and the coding features include at least one of the following: bit rate and noise; and / or The prediction model includes at least one of the following: XGBoost model, random forest, support vector machine, neural network, reinforcement learning network; and / or The image quality index includes at least one of the following: an objective image quality index and a subjective image quality index, wherein the objective image quality index includes a distortion index; and / or The preferred encoding parameters include at least one of the following parameters: constant rate factor, resolution, bit rate, and sharpening effect parameter.

6. A video transcoding device, characterized in that: The video transcoding device comprises: an extraction unit configured to extract video features of currently received video clip data; a processing unit configured to process the video features using a pre-trained prediction model to obtain image quality and bitrate mapping information of the video segment data, wherein the image quality and bitrate mapping information is used to represent a mapping relationship between image quality indicators and bitrate; a determining unit configured to determine preferred encoding parameters of the video segment data according to the image quality bit rate mapping information; The transcoding unit is configured to perform transcoding processing on the video segment data according to the preferred encoding parameters to obtain video stream data corresponding to the video segment data.

7. An electronic device, characterized in that: include: at least one processor; at least one memory storing computer-executable instructions, When the computer executable instructions are executed by at least one processor, the computer executable instructions prompt the at least one processor to execute the video transcoding method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the video transcoding method according to any one of claims 1 to 5.

9. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by at least one processor, the video transcoding method according to any one of claims 1 to 5 is implemented.

10. A method for generating a bit stream, characterized in that: include: A bit stream is generated according to the video transcoding method according to any one of claims 1 to 5.

Images