Video processing method, device, computer equipment and computer readable storage medium

Abstract

This disclosure relates to a video processing method, apparatus, computer device, and storage medium, belonging to the field of video technology. This disclosure involves a live streaming server determining a first bandwidth for transmitting the live video stream at the current moment and predicting the peak bandwidth of the live streaming server. If the first bandwidth is lower than the predicted peak bandwidth, and the ratio between these two bandwidths is greater than or equal to a first target bitrate adjustment range, the bitrate of the currently received live video stream is reduced. This reduces the bandwidth used when pushing the reduced bitrate live video stream to the terminal, thus preventing the bandwidth used when pushing the video stream from reaching the predicted peak bandwidth, thereby reducing the peak bandwidth of the live streaming service.

Description

Technical Field

[0001] This disclosure relates to the field of video technology, and in particular to a video processing method, apparatus, computer equipment, and computer-readable storage medium. Background Technology

[0002] Live streaming platforms can provide video live streaming services to users. During the off-peak period of video live streaming services, relatively little bandwidth is needed to push video streams to users. During the peak period of video live streaming services, a large amount of bandwidth is needed to push video streams to users. As the maximum bandwidth consumed by the live streaming platform during peak periods (i.e., peak bandwidth) increases, the stuttering rate of live video will increase. Therefore, there is an urgent need for a method to reduce peak bandwidth. Summary of the Invention

[0003] This disclosure provides a video processing method, apparatus, computer device, and computer-readable storage medium to reduce the peak bandwidth of live streaming servers and decrease the stuttering rate of live video. The technical solution of this disclosure is as follows:

[0004] According to one aspect of the present disclosure, a video processing method is provided, the method being applied to a live streaming server, the method comprising:

[0005] Receive the first live video stream from the first terminal;

[0006] Determine the total bandwidth of the live video stream sent by the live streaming server at the current moment to obtain the first bandwidth;

[0007] The peak bandwidth of the live streaming server is predicted to obtain the first peak bandwidth;

[0008] If the first bandwidth is less than the first peak bandwidth, and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, the bitrate of the first live video stream is reduced by the first target bitrate adjustment range to obtain the second live video stream.

[0009] The second live video stream is sent to the second terminal.

[0010] Optionally, reducing the bitrate of the first live video stream by the first target bitrate adjustment range if the first bandwidth is less than the first peak bandwidth and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range includes:

[0011] If the first bandwidth is less than the first peak bandwidth, and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, the video features of the first live video stream are extracted to obtain the first video features, which are used to describe the live video stream.

[0012] Based on the first video features, the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target are predicted to obtain the first transcoding parameter values. The first transcoding target includes reducing the bitrate of the live video stream by the first target bitrate adjustment range and ensuring that the video quality evaluation value of the transcoded live video stream meets the target conditions.

[0013] The first live video stream is transcoded using the first transcoding parameter value.

[0014] Optionally, the step of predicting the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target based on the first video features, and obtaining the first transcoding parameter values, includes:

[0015] Obtain the first transcoding parameter prediction model corresponding to the first target bitrate adjustment range, wherein the first transcoding parameter prediction model is a shallow neural network.

[0016] The first video feature is input into the first transcoding parameter prediction model. Based on the first video feature, the first transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target, and outputs the first transcoding parameter values.

[0017] Optionally, before receiving the first live video stream from the first terminal, the method further includes:

[0018] Receive the first historical live video streams from multiple third-party terminals;

[0019] Based on multiple first bitrate adjustment ranges, the transcoding parameter values ​​required for the first historical live video stream to reach the first transcoding target are predicted to obtain the first historical transcoding parameter values.

[0020] The video features of the first historical live video stream are extracted to obtain the first historical video features;

[0021] The first transcoding parameter prediction model is trained based on the first historical transcoding parameter value and the first historical video features.

[0022] Optionally, the step of predicting the transcoding parameter values ​​required for the first historical live video stream to reach the first transcoding target based on multiple first bitrate adjustment magnitudes, and obtaining the first historical transcoding parameter values, includes:

[0023] Based on the multiple first bitrate adjustment ranges, the first historical live video stream is transcoded multiple times to obtain multiple second historical live video streams, each of which has a lower bitrate than the first historical live video stream.

[0024] Obtain transcoding information of the plurality of second historical live video streams, wherein the transcoding information of each second historical live video stream includes the first bitrate adjustment range and transcoding parameter values ​​used when obtaining the second historical live video stream;

[0025] Based on the transcoding information of the multiple second historical live video streams, the transcoding parameter values ​​required for the first historical live video stream to reach the first transcoding target are obtained.

[0026] Optionally, after predicting the peak bandwidth of the live streaming server to obtain the first peak bandwidth, the method further includes:

[0027] If the first bandwidth is less than the first peak bandwidth, and the difference between the first bandwidth and the first peak bandwidth is greater than the target difference, the bitrate of the first live video stream is increased by the second target bitrate adjustment range to obtain the third live video stream.

[0028] The third live video stream is sent to the second terminal.

[0029] Optionally, the step of increasing the bitrate of the first live video stream by the first target bitrate adjustment amount if the first bandwidth is less than the first peak bandwidth and the difference between the first bandwidth and the first peak bandwidth is greater than the target difference includes:

[0030] If the first bandwidth is less than the first peak bandwidth, and the difference between the first bandwidth and the first peak bandwidth is greater than the target difference, the video features of the first live video stream are extracted to obtain the first video features, which are used to describe the live video stream.

[0031] Based on the first video features, the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target are predicted to obtain the second transcoding parameter values. The second transcoding target includes increasing the bitrate of the live video stream by the second target bitrate adjustment range and ensuring that the video quality evaluation value of the transcoded live video stream meets the target conditions.

[0032] The first live video stream is transcoded using the second transcoding parameter value.

[0033] Optionally, the step of predicting the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target based on the first video features, and obtaining the second transcoding parameter values, includes:

[0034] Obtain the second transcoding parameter prediction model corresponding to the second target bitrate adjustment range, wherein the second transcoding parameter prediction model is a shallow neural network;

[0035] The first video feature is input into the second transcoding parameter prediction model. Based on the first video feature, the second transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target, and outputs the second transcoding parameter values.

[0036] Optionally, before receiving the first live video stream from the first terminal, the method further includes:

[0037] Receive third-generation historical live video streams from multiple fourth terminals;

[0038] Based on multiple second bitrate adjustment ranges, the transcoding parameter values ​​required for the third historical live video stream to reach the second transcoding target are predicted to obtain the second historical transcoding parameter values.

[0039] The video features of the third historical live video stream are extracted to obtain the second historical video features;

[0040] The second transcoding parameter prediction model is trained based on the second historical transcoding parameter values ​​and the second historical video features.

[0041] Optionally, the step of predicting the transcoding parameter values ​​required for the third historical live video stream to reach the second transcoding target based on multiple second bitrate adjustment amplitudes, and obtaining the second historical transcoding parameter values, includes:

[0042] Based on the multiple second bitrate adjustment ranges, the third historical live video stream is transcoded multiple times to obtain multiple fourth historical live video streams, each of which has a higher bitrate than the third historical live video stream.

[0043] Obtain transcoding information of the plurality of fourth historical live video streams, wherein the transcoding information of each fourth historical live video stream includes the second bitrate adjustment range and transcoding parameter values ​​used when obtaining the fourth historical live video stream;

[0044] Based on the transcoding information of the multiple fourth historical live video streams, the transcoding parameter values ​​required for the third historical live video stream to reach the second transcoding target are obtained.

[0045] Optionally, before increasing the bitrate of the first live video stream by the second target bitrate adjustment amount, the method further includes:

[0046] Based on the first bandwidth, the first peak bandwidth, and the safety factor, the second target bitrate adjustment range is obtained. The safety factor is used to mitigate the impact of the difference between the first bandwidth and the second bandwidth on bandwidth cost. The second bandwidth is the bandwidth required when the third live video stream is sent.

[0047] Optionally, predicting the peak bandwidth of the live streaming server to obtain the first peak bandwidth includes:

[0048] The second peak bandwidth and the historical bitrate adjustment range within the target historical time period are obtained. The second peak bandwidth is the maximum bandwidth occupied when sending live video streams within the target historical time period, and the historical bitrate adjustment range is the bitrate adjustment range used when adjusting the bitrate of live video streams within the target historical time period.

[0049] Based on the second peak bandwidth and the historical bitrate adjustment range, a third peak bandwidth is obtained. The third peak bandwidth is the maximum bandwidth required by the live streaming server to send a live video stream within the target historical time period without adjusting the live video stream bitrate.

[0050] According to another aspect of the present disclosure, a video processing apparatus is provided, the apparatus being applied to a live streaming server, the apparatus comprising:

[0051] The receiving unit is configured to receive the first live video stream from the first terminal;

[0052] The determining unit is configured to determine the total bandwidth of the live video stream sent by the live streaming server at the current moment, and obtain the first bandwidth;

[0053] The prediction unit is configured to perform a prediction of the peak bandwidth of the live streaming server to obtain a first peak bandwidth;

[0054] The bitrate adjustment unit is configured to reduce the bitrate of the first live video stream by the first target bitrate adjustment range if the first bandwidth is less than the first peak bandwidth and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, thereby obtaining a second live video stream.

[0055] The sending unit is configured to send the second live video stream to the second terminal.

[0056] Optionally, the bitrate adjustment unit includes:

[0057] The first extraction subunit is configured to extract video features from the first live video stream if the first bandwidth is less than the first peak bandwidth and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, thereby obtaining a first video feature, which is used to describe the live video stream.

[0058] The first prediction subunit is configured to perform prediction of the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target based on the first video features, and obtain the first transcoding parameter values. The first transcoding target includes reducing the bitrate of the live video stream by the first target bitrate adjustment range and ensuring that the video quality evaluation value of the transcoded live video stream meets the target conditions.

[0059] The first transcoding subunit is configured to transcode the first live video stream using the first transcoding parameter value.

[0060] Optionally, the first prediction subunit is configured to perform:

[0061] Obtain the first transcoding parameter prediction model corresponding to the first target bitrate adjustment range, wherein the first transcoding parameter prediction model is a shallow neural network.

[0062] The first video feature is input into the first transcoding parameter prediction model. Based on the first video feature, the first transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target, and outputs the first transcoding parameter values.

[0063] Optionally, the device further includes a first feature extraction unit and a first training unit;

[0064] The receiving unit is also configured to receive a first historical live video stream from multiple third terminals;

[0065] The prediction unit is further configured to perform a prediction based on multiple first bitrate adjustment magnitudes of the transcoding parameter values ​​required for the first historical live video stream to reach the first transcoding target, thereby obtaining the first historical transcoding parameter values.

[0066] The first feature extraction unit is configured to extract the video features from the first historical live video stream to obtain the first historical video features;

[0067] The first training unit is configured to train the first transcoding parameter prediction model based on the first historical transcoding parameter values ​​and the first historical video features.

[0068] Optionally, the prediction unit is configured to perform:

[0069] Based on the multiple first bitrate adjustment ranges, the first historical live video stream is transcoded multiple times to obtain multiple second historical live video streams, each of which has a lower bitrate than the first historical live video stream.

[0070] Obtain transcoding information of the plurality of second historical live video streams, wherein the transcoding information of each second historical live video stream includes the first bitrate adjustment range and transcoding parameter values ​​used when obtaining the second historical live video stream;

[0071] Based on the transcoding information of the multiple second historical live video streams, the transcoding parameter values ​​required for the first historical live video stream to reach the first transcoding target are obtained.

[0072] Optionally, the bitrate adjustment unit is further configured to perform the following: if the first bandwidth is less than the first peak bandwidth and the difference between the first bandwidth and the first peak bandwidth is greater than a target difference, increase the bitrate of the first live video stream by a second target bitrate adjustment amount to obtain a third live video stream.

[0073] The sending unit is also configured to send the third live video stream to the second terminal.

[0074] Optionally, the bit rate adjustment unit includes:

[0075] The second feature extraction subunit is configured to extract video features from the first live video stream if the first bandwidth is less than the first peak bandwidth and the difference between the first bandwidth and the first peak bandwidth is greater than a target difference, thereby obtaining a first video feature, which is used to describe the live video stream.

[0076] The second prediction subunit is configured to perform prediction of the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target based on the first video features, and obtain the second transcoding parameter values. The second transcoding target includes increasing the bitrate of the live video stream by the second target bitrate adjustment range and ensuring that the video quality evaluation value of the transcoded live video stream meets the target conditions.

[0077] The second transcoding subunit is configured to transcode the first live video stream using the second transcoding parameter value.

[0078] Optionally, the second prediction subunit is configured to perform:

[0079] Obtain the second transcoding parameter prediction model corresponding to the second target bitrate adjustment range, wherein the second transcoding parameter prediction model is a shallow neural network;

[0080] The first video feature is input into the second transcoding parameter prediction model. Based on the first video feature, the second transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target, and outputs the second transcoding parameter values.

[0081] Optionally, the apparatus method further includes a second feature extraction unit and a second training unit;

[0082] The receiving unit is also configured to receive third historical live video streams from multiple fourth terminals;

[0083] The prediction unit is further configured to perform prediction of the transcoding parameter values ​​required for the third historical live video stream to reach the second transcoding target based on multiple second bitrate adjustment magnitudes, thereby obtaining the second historical transcoding parameter values;

[0084] The second feature extraction unit is configured to extract the video features from the third historical live video stream to obtain the second historical video features;

[0085] The second training unit is configured to train the second transcoding parameter prediction model based on the second historical transcoding parameter values ​​and the second historical video features.

[0086] Optionally, the prediction unit is further configured to perform:

[0087] Based on the multiple second bitrate adjustment ranges, the third historical live video stream is transcoded multiple times to obtain multiple fourth historical live video streams, each of which has a higher bitrate than the third historical live video stream.

[0088] Obtain transcoding information of the plurality of fourth historical live video streams, wherein the transcoding information of each fourth historical live video stream includes the second bitrate adjustment range and transcoding parameter values ​​used when obtaining the fourth historical live video stream;

[0089] Based on the transcoding information of the multiple fourth historical live video streams, the transcoding parameter values ​​required for the third historical live video stream to reach the second transcoding target are obtained.

[0090] Optionally, the device further includes:

[0091] The acquisition unit is configured to acquire the second target bitrate adjustment range based on the first bandwidth, the first peak bandwidth, and a safety factor. The safety factor is used to mitigate the impact of the difference between the first bandwidth and the second bandwidth on bandwidth cost. The second bandwidth is the bandwidth required when the third live video stream is sent.

[0092] Optionally, the prediction unit is used for:

[0093] The second peak bandwidth and the historical bitrate adjustment range within the target historical time period are obtained. The second peak bandwidth is the maximum bandwidth occupied when sending live video streams within the target historical time period, and the historical bitrate adjustment range is the bitrate adjustment range used when adjusting the bitrate of live video streams within the target historical time period.

[0094] Based on the second peak bandwidth and the historical bitrate adjustment range, a third peak bandwidth is obtained. The third peak bandwidth is the maximum bandwidth required by the live streaming server to send a live video stream within the target historical time period without adjusting the live video stream bitrate.

[0095] According to another aspect of the embodiments of this disclosure, a computer device is provided, comprising:

[0096] One or more processors;

[0097] One or more memories for storing the one or more processor-executable instructions;

[0098] The one or more processors are configured to perform a video processing method in any of the possible implementations of the above-described aspects.

[0099] According to another aspect of the present disclosure, a computer-readable storage medium is provided such that, when at least one instruction in the computer-readable storage medium is executed by one or more processors of a computer device, the computer device is enabled to perform a video processing method in any possible implementation of the above aspect.

[0100] According to another aspect of the present disclosure, a computer program product is provided, including one or more instructions that can be executed by one or more processors of a computer device, enabling the computer device to perform a video processing method in any possible implementation of the above aspect.

[0101] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0102] By determining the first bandwidth for sending the live video stream at the current moment and predicting the peak bandwidth of the live video server, if the first bandwidth is lower than the predicted peak bandwidth of the live video server, and the ratio between the two bandwidths is greater than or equal to the first target bitrate adjustment range, the bitrate of the live video stream received at the current moment is reduced. This reduces the bandwidth used when pushing the reduced bitrate live video stream to the terminal, thus avoiding the bandwidth used when pushing the video stream from reaching the predicted peak bandwidth, thereby reducing the peak bandwidth of the live streaming service.

[0103] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0104] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0105] Figure 1 This is a schematic diagram illustrating the implementation environment of a video processing method according to an exemplary embodiment;

[0106] Figure 2 This is a schematic diagram of the historical bandwidth curve of a live streaming server according to an exemplary embodiment;

[0107] Figure 3 This is a flowchart illustrating a video processing method according to an exemplary embodiment;

[0108] Figure 4 This is an interactive flowchart illustrating a video processing method according to an exemplary embodiment;

[0109] Figure 5 This is an interactive flowchart illustrating a video processing method according to an exemplary embodiment;

[0110] Figure 6 This is a structural block diagram of a video device according to an exemplary embodiment;

[0111] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0112] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0113] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0114] The user information disclosed herein may be information authorized by the user or fully authorized by all parties.

[0115] In some embodiments, the meaning of A and / or B includes three cases: A and B, and A and B.

[0116] Figure 1 This is a schematic diagram illustrating an implementation environment of a video processing method according to an exemplary embodiment. See also: Figure 1 The implementation environment includes a live streaming server 101 and a terminal 102. The live streaming server 101 is connected to the terminal 102 via a wired or wireless network, as detailed below:

[0117] The live streaming server 101 is used to provide backend services for the target application, which provides live video streaming services. Live video streaming may or may not be the target application's main business. The target application may be any of the following: a live streaming application, a short video application, a social application, or a game application.

[0118] The live streaming server 101 includes at least one of the following: a single server, multiple servers, a cloud computing platform, or a virtualization center. Optionally, the live streaming server 101 is an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Optionally, during the interaction between the live streaming server 101 and the terminal 102, the live streaming server 101 undertakes the main computing work, and the terminal 102 undertakes the secondary computing work; or, the live streaming server 101 undertakes the secondary computing work, and the terminal 102 undertakes the main computing work; or, the live streaming server 101 and the terminal 102 use a distributed computing architecture for collaborative computing.

[0119] There is at least one terminal 102 in this implementation environment. The number of terminals 102 is not limited in this embodiment. The terminal 102 in this implementation environment includes at least one of smartphones, tablets, smart speakers, smartwatches, laptops, smart handheld consoles, portable gaming devices, or desktop computers. The type of terminal 102 is not limited to these.

[0120] In this implementation environment, each terminal 102 runs the target application. Users can conduct live video streaming through the target application or watch live video streaming through the target application. For ease of description, the user conducting live video streaming through the application is referred to as the streamer, and the terminal 102 used by the streamer is referred to as the first terminal. The user watching the live video streaming is referred to as the viewer, and the terminal 102 used by the viewer is referred to as the second terminal. There may be one or more first terminals and second terminals in this implementation environment. Here, this embodiment of the disclosure does not limit the number of first terminals and second terminals in this implementation environment.

[0121] The interaction process between the live streaming server and the first and second terminals is described below:

[0122] In response to a broadcaster conducting a live video stream via the target application, the first terminal generates and plays the live video stream and sends it to the live streaming server (commonly known as "pushing the stream"). In response to a viewer's request to watch the live video stream, the second terminal requests the live streaming server to retrieve the live video stream from the first terminal (commonly known as "pulling the stream"). When the first terminal pushes the live video stream to the live streaming server, in response to the second terminal's request, the live streaming server pushes the live video stream to the second terminal so that the second terminal can play the live video stream.

[0123] Furthermore, because most users' viewing habits are concentrated in a specific period, the video live streaming business experiences peak and off-peak periods. For example... Figure 2 A schematic diagram of the historical bandwidth curve of a live streaming server is shown according to an exemplary embodiment, such as... Figure 2 As shown, the historical bandwidth curve 200 represents the trend of live streaming bandwidth changes on a specific day in history. Live streaming bandwidth refers to the total bandwidth used by the server when sending live video streams. The historical bandwidth curve 200 shows troughs and peaks. The troughs represent the minimum bandwidth P consumed by the live video service, while the peaks represent the maximum bandwidth Q consumed by the live video service. The maximum bandwidth B is also the peak bandwidth.

[0124] In some embodiments, the live streaming server is also referred to as a live streaming platform. Considering that the bandwidth cost of the live streaming platform is related to the peak bandwidth consumed during peak periods, this disclosure provides a video processing method for reducing peak bandwidth. Next, based on the above-described implementation environment, the video processing method is described as follows:

[0125] Figure 3 This is a flowchart illustrating a video processing method according to an exemplary embodiment, see [link to flowchart]. Figure 3 This video processing method is applied to a live streaming server and includes the following steps.

[0126] In step 301, the live streaming server receives the first live video stream from the first terminal.

[0127] In step 302, the live streaming server determines the total bandwidth of the live video stream sent by the live streaming server at the current moment, and obtains the first bandwidth.

[0128] In step 303, the live streaming server predicts the peak bandwidth of the live streaming server to obtain the first peak bandwidth.

[0129] In step 304, if the first bandwidth is less than the first peak bandwidth, and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, the live streaming server reduces the bitrate of the first live video stream by the first target bitrate adjustment range to obtain the second live video stream.

[0130] In step 305, the live streaming server sends the second live video stream to the second terminal.

[0131] The method provided in this disclosure reduces the bitrate of the received live video stream when the live streaming server determines the first bandwidth for sending the live video stream at the current moment and predicts the peak bandwidth of the live streaming server. If the first bandwidth is lower than the predicted peak bandwidth of the live streaming server, and the ratio between the two bandwidths is greater than or equal to the first target bitrate adjustment range, the bandwidth occupied when pushing the live video stream with the reduced bitrate to the terminal will also be reduced, so as to avoid the bandwidth occupied when pushing the video stream reaching the predicted peak bandwidth, thereby reducing the peak bandwidth of the live streaming service.

[0132] Optionally, reducing the bitrate of the first live video stream by the first target bitrate adjustment range if the first bandwidth is less than the first peak bandwidth and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range includes:

[0133] If the first bandwidth is less than the first peak bandwidth, and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, the video features of the first live video stream are extracted to obtain the first video features, which are used to describe the live video stream.

[0134] Based on the first video features, the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target are predicted to obtain the first transcoding parameter values. The first transcoding target includes reducing the bitrate of the live video stream by the first target bitrate adjustment range and ensuring that the video quality evaluation value of the transcoded live video stream meets the target conditions.

[0135] The first live video stream is transcoded using the first transcoding parameter value.

[0136] Optionally, the step of predicting the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target based on the first video features, and obtaining the first transcoding parameter values, includes:

[0137] Obtain the first transcoding parameter prediction model corresponding to the first target bitrate adjustment range, wherein the first transcoding parameter prediction model is a shallow neural network.

[0138] The first video feature is input into the first transcoding parameter prediction model. Based on the first video feature, the first transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target, and outputs the first transcoding parameter values.

[0139] Optionally, before receiving the first live video stream from the first terminal, the method further includes:

[0140] Receive the first historical live video streams from multiple third-party terminals;

[0141] Based on multiple first bitrate adjustment ranges, the transcoding parameter values ​​required for the first historical live video stream to reach the first transcoding target are predicted to obtain the first historical transcoding parameter values.

[0142] The video features of the first historical live video stream are extracted to obtain the first historical video features;

[0143] The first transcoding parameter prediction model is trained based on the first historical transcoding parameter value and the first historical video features.

[0144] Optionally, the step of predicting the transcoding parameter values ​​required for the first historical live video stream to reach the first transcoding target based on multiple first bitrate adjustment magnitudes, and obtaining the first historical transcoding parameter values, includes:

[0145] Based on the multiple first bitrate adjustment ranges, the first historical live video stream is transcoded multiple times to obtain multiple second historical live video streams, each of which has a lower bitrate than the first historical live video stream.

[0146] Obtain transcoding information of the plurality of second historical live video streams, wherein the transcoding information of each second historical live video stream includes the first bitrate adjustment range and transcoding parameter values ​​used when obtaining the second historical live video stream;

[0147] Based on the transcoding information of the multiple second historical live video streams, the transcoding parameter values ​​required for the first historical live video stream to reach the first transcoding target are obtained.

[0148] Optionally, after predicting the peak bandwidth of the live streaming server to obtain the first peak bandwidth, the method further includes:

[0149] If the first bandwidth is less than the first peak bandwidth, and the difference between the first bandwidth and the first peak bandwidth is greater than the target difference, the bitrate of the first live video stream is increased by the second target bitrate adjustment range to obtain the third live video stream.

[0150] The third live video stream is sent to the second terminal.

[0151] Optionally, the step of increasing the bitrate of the first live video stream by the first target bitrate adjustment amount if the first bandwidth is less than the first peak bandwidth and the difference between the first bandwidth and the first peak bandwidth is greater than the target difference includes:

[0152] If the first bandwidth is less than the first peak bandwidth, and the difference between the first bandwidth and the first peak bandwidth is greater than the target difference, the video features of the first live video stream are extracted to obtain the first video features, which are used to describe the live video stream.

[0153] Based on the first video features, the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target are predicted to obtain the second transcoding parameter values. The second transcoding target includes increasing the bitrate of the live video stream by the second target bitrate adjustment range and ensuring that the video quality evaluation value of the transcoded live video stream meets the target conditions.

[0154] The first live video stream is transcoded using the second transcoding parameter value.

[0155] Optionally, the step of predicting the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target based on the first video features, and obtaining the second transcoding parameter values, includes:

[0156] Obtain the second transcoding parameter prediction model corresponding to the second target bitrate adjustment range, wherein the second transcoding parameter prediction model is a shallow neural network;

[0157] The first video feature is input into the second transcoding parameter prediction model. Based on the first video feature, the second transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target, and outputs the second transcoding parameter values.

[0158] Optionally, before receiving the first live video stream from the first terminal, the method further includes:

[0159] Receive third-generation historical live video streams from multiple fourth terminals;

[0160] Based on multiple second bitrate adjustment ranges, the transcoding parameter values ​​required for the third historical live video stream to reach the second transcoding target are predicted to obtain the second historical transcoding parameter values.

[0161] The video features of the third historical live video stream are extracted to obtain the second historical video features;

[0162] The second transcoding parameter prediction model is trained based on the second historical transcoding parameter values ​​and the second historical video features.

[0163] Optionally, the step of predicting the transcoding parameter values ​​required for the third historical live video stream to reach the second transcoding target based on multiple second bitrate adjustment amplitudes, and obtaining the second historical transcoding parameter values, includes:

[0164] Based on the multiple second bitrate adjustment ranges, the third historical live video stream is transcoded multiple times to obtain multiple fourth historical live video streams, each of which has a higher bitrate than the third historical live video stream.

[0165] Obtain transcoding information of the plurality of fourth historical live video streams, wherein the transcoding information of each fourth historical live video stream includes the second bitrate adjustment range and transcoding parameter values ​​used when obtaining the fourth historical live video stream;

[0166] Based on the transcoding information of the multiple fourth historical live video streams, the transcoding parameter values ​​required for the third historical live video stream to reach the second transcoding target are obtained.

[0167] Optionally, before increasing the bitrate of the first live video stream by the second target bitrate adjustment amount, the method further includes:

[0168] Based on the first bandwidth, the first peak bandwidth, and the safety factor, the second target bitrate adjustment range is obtained. The safety factor is used to mitigate the impact of the difference between the first bandwidth and the second bandwidth on bandwidth cost. The second bandwidth is the bandwidth required when the third live video stream is sent.

[0169] Optionally, predicting the peak bandwidth of the live streaming server to obtain the first peak bandwidth includes:

[0170] The second peak bandwidth and the historical bitrate adjustment range within the target historical time period are obtained. The second peak bandwidth is the maximum bandwidth occupied when sending live video streams within the target historical time period, and the historical bitrate adjustment range is the bitrate adjustment range used when adjusting the bitrate of live video streams within the target historical time period.

[0171] Based on the second peak bandwidth and the historical bitrate adjustment range, a third peak bandwidth is obtained. The third peak bandwidth is the maximum bandwidth required by the live streaming server to send a live video stream within the target historical time period without adjusting the live video stream bitrate.

[0172] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0173] Next, we will provide a detailed explanation of the process by which the live streaming server reduces the bitrate of the live video stream to be sent:

[0174] Figure 4 This is an interactive flowchart illustrating a video processing method according to an exemplary embodiment. See also: Figure 4 The video processing method includes the following steps.

[0175] In step 401, the first terminal sends the first live video stream to the live streaming server.

[0176] The first terminal can be any terminal that performs live video streaming through the target application. The first live video stream is a video stream generated by the first terminal based on the live video streaming being performed.

[0177] For example, the host performs an operation to start live video streaming on the target application on the first terminal. In response to this operation, the first terminal turns on the camera, captures video through the camera, obtains the first live video stream, and sends the first live video stream to the live streaming server.

[0178] In step 402, the live streaming server receives the first live video stream from the first terminal.

[0179] Since the live streaming server provides background services for each terminal running the target program, at the current moment, the live streaming server receives at least one first live video stream from at least one first terminal.

[0180] In step 403, the live streaming server determines the total bandwidth of the live video stream sent by the live streaming server at the current moment, and obtains the first bandwidth.

[0181] The first bandwidth can also be understood as the live bandwidth when at least one first live video stream is received. Live bandwidth refers to the total bandwidth used by the live server to send live video streams, that is, the total bandwidth used to push live video streams to the second terminal. Live bandwidth is also known as downlink bandwidth.

[0182] In one possible implementation, the live streaming server continuously monitors the local live streaming bandwidth (i.e., real-time monitoring) and uses the live streaming bandwidth detected when receiving at least one first live video stream as the first bandwidth.

[0183] In step 404, the live streaming server predicts the peak bandwidth of the live streaming server to obtain the first peak bandwidth.

[0184] The first peak bandwidth is the peak bandwidth that the live streaming server may reach within the target duration after the predicted current moment without adjusting the live stream bitrate. The target duration can be set according to the specific implementation scenario; however, this embodiment does not limit the target duration.

[0185] In one possible implementation, the live streaming server uses a first duration as a bandwidth statistics period, and counts the live streaming bandwidth within that first duration to obtain live streaming bandwidth data. For example, within a bandwidth statistics period, the live streaming server uses a second duration as a bandwidth sampling period, and collects the detected live streaming bandwidth once at each bandwidth sampling period interval. The multiple live streaming bandwidths collected within this bandwidth statistics period constitute the live streaming bandwidth data for that period. The first duration is longer than the second duration; the first duration can be, for example, one day, one week, or one year, and the second duration can be, for example, 3 minutes, 5 minutes, or 10 minutes. The first and second durations can be set according to the actual application scenario; this embodiment does not limit the target duration.

[0186] The target duration is defined as a time interval, and the live bandwidth data for each time interval is stored. For ease of description, the time interval to which the current moment belongs is called the current time interval, the time intervals preceding the current time interval are called historical time intervals, and the live bandwidth data of the historical time intervals are called historical live bandwidth data. In this case, the third duration is the duration after the current moment in the current time interval, or it can be the first bandwidth sampling period after the current moment.

[0187] When executing step 404, the live streaming server obtains historical live streaming bandwidth data for the target historical time period. Based on this data, the peak bandwidth of the live streaming server is predicted. The target historical time period is a historical period with a high degree of similarity to the current time period's bandwidth change trend. If the current time period is a holiday, the target historical time period is the nearest holiday to the current time period. If the current time period is not a holiday, the target historical time period is the previous time period, or the target-number historical time period preceding the current time period. For example, using days as the time period, the current time period is today, and the previous time period is the day before. Taking a target number of 7 as an example, if today is Sunday, the target historical time period is last Sunday.

[0188] The following steps, 4041-4042, describe the process of predicting the peak bandwidth of a live streaming server based on historical live streaming bandwidth data for a target historical time period:

[0189] In step 4041, the live streaming server obtains the second peak bandwidth and the historical bitrate adjustment range within the target historical time period. The second peak bandwidth is the maximum bandwidth occupied when sending the live video stream within the target historical time period, and the historical bitrate adjustment range is the bitrate adjustment range used when adjusting the bitrate of the live video stream within the target historical time period.

[0190] The historical bitrate adjustment range includes a first historical bitrate adjustment range and a second historical bitrate adjustment range. The first historical bitrate adjustment range is the bitrate adjustment range used when reducing the bitrate of the live video stream within the target historical time period. The second historical bitrate adjustment range is the bitrate adjustment range used when increasing the bitrate of the live video stream within the target historical time period.

[0191] For ease of description, the bitrate adjustment range used when reducing the bitrate is called the bitrate reduction adjustment range, and the bitrate adjustment range used when increasing the bitrate is called the bitrate increase adjustment range. Each time period corresponds to a bitrate reduction adjustment range and a bitrate increase adjustment range. For either the bitrate reduction adjustment range or the bitrate increase adjustment range, this range can be determined based on the peak bandwidth of the historical time period or based on business requirements. Business requirements include, for example, ensuring that the average resolution of the pushed live video stream is not lower than the target resolution and that the bandwidth cost is not higher than a cost threshold. Under such business requirements, the determination of this bitrate adjustment range can be, for example, ensuring that the average resolution of the pushed live video stream is not lower than the target resolution and that the bandwidth cost is not higher than the cost threshold, that the live streaming server selects a first bitrate adjustment range within the target value range, where the target value range is 0-10%, although other target value ranges are also possible. The first and second bitrate adjustment ranges corresponding to different time periods may be the same or different. In this embodiment, the target value range and the first and second bitrate adjustment ranges corresponding to each time period are not limited.

[0192] The live streaming server stores the bitrate adjustment range and bitrate adjustment range corresponding to multiple historical time periods. In this step 4041, the live streaming server obtains the bitrate adjustment range corresponding to the target historical time period as the historical bitrate adjustment range.

[0193] Regarding the method for obtaining the second peak bandwidth, in some embodiments, the live streaming server obtains the live streaming bandwidth data for the target historical time period from live streaming bandwidth data of at least one historical time period. Then, the maximum bandwidth in the obtained live streaming bandwidth data is used as the second peak bandwidth. In this case, the second peak bandwidth is the peak bandwidth of the target historical time period. Figure 2 For example, within a target historical time period, the live streaming server collects bandwidth data at 5-minute intervals, using the collected bandwidth for that period as the live streaming bandwidth data. After obtaining the bandwidth data for the target historical time period, the server plots the historical bandwidth curve 200 (as shown in the graph) and uses the maximum bandwidth Q on the curve as the second peak bandwidth. Alternatively, the server may not plot a historical bandwidth curve but instead compare the live streaming bandwidth data and use the maximum bandwidth found in the comparison as the second peak bandwidth.

[0194] Alternatively, each time period includes at least one peak period of the live video service, and each peak period has a peak bandwidth. The live streaming server uses the maximum peak bandwidth among at least one peak bandwidth of the target historical time period as the second peak bandwidth. Alternatively, if the historical moment corresponding to the current moment in the target historical time period is in a peak period, the live streaming server uses the peak bandwidth of the peak period to which the historical moment belongs as the second peak bandwidth. If the historical moment is not in a peak period, the server uses the peak bandwidth of the first peak period after that historical moment as the second peak bandwidth.

[0195] In other embodiments, the live streaming server does not store live streaming bandwidth data for historical time periods, but stores at least one peak bandwidth for historical time periods. Each peak bandwidth corresponds to a peak period. When performing this step 4041, a peak bandwidth is selected from the at least one peak bandwidth of the target historical time period as the second peak bandwidth. The selection method can be referred to above.

[0196] In step 4042, the live streaming server obtains a third peak bandwidth based on the second peak bandwidth and the historical bitrate adjustment range. The third peak bandwidth is the maximum bandwidth required by the live streaming server to send the live video stream within the target historical time period without adjusting the live video stream bitrate.

[0197] Since the live streaming server reduces the bitrate of the live video stream to be sent based on the real-time tested live streaming bandwidth in order to adjust the peak bandwidth, the second peak bandwidth is the peak bandwidth within the target historical time period when the live video stream bitrate is adjusted (e.g., reduced), or it can be understood as the peak bandwidth after peak shaving within the target historical time period. Correspondingly, the third peak bandwidth can also be understood as the peak bandwidth within the target historical time period without peak shaving.

[0198] In some embodiments, the live streaming server uses the bitrate reduction adjustment magnitude corresponding to the target historical time period as the historical bitrate adjustment magnitude. Using this historical bitrate adjustment magnitude as an increase ratio, a second peak bandwidth with a reduced historical bitrate adjustment magnitude is added to the second peak bandwidth to obtain a third peak bandwidth, thus restoring the un-shaving peak bandwidth of the target historical time period. Assuming the second peak bandwidth is BB and the historical bitrate adjustment magnitude is XX%, then the third peak bandwidth B = BB * (XX% + 1). The third peak bandwidth B is used as the first peak bandwidth.

[0199] In other embodiments, since the live streaming bandwidth during the peak period of the target historical time period is reduced proportionally (the proportion being the corresponding first bitrate adjustment magnitude), if the historical moment corresponding to the current moment in the target historical time period is at its peak, the live streaming server can also predict the first peak bandwidth based on the live streaming bandwidth of the historical moment, the first bandwidth, and the third bandwidth. For example, assuming the first bandwidth at the current moment is A, the live streaming bandwidth at the historical moment is AA, and the second peak bandwidth after peak reduction in the target historical time period is BB, then the first peak bandwidth C = (A / AA) * BB * (XX% + 1).

[0200] Based on the second peak bandwidth and historical bitrate adjustment range within the target historical time period, the live streaming server reconstructs the maximum bandwidth required to send the live video stream without adjusting the bitrate of the live video stream within the target historical time period. The reconstructed maximum bandwidth is used as the first peak bandwidth to predict the future peak bandwidth of the live streaming server. This allows the server to determine whether to adjust the bitrate of the first live video stream in advance based on the predicted peak bandwidth.

[0201] In step 405, if the first bandwidth is less than the first peak bandwidth, and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, the live streaming server reduces the bitrate of the first live video stream by the first target bitrate adjustment range to obtain the second live video stream.

[0202] The first target bitrate adjustment range is the bitrate reduction adjustment range corresponding to the current time period.

[0203] If the first bandwidth is less than the first peak bandwidth, and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, and the first live video stream received at the current moment is pushed to the second terminal, the bandwidth occupied within the target duration after the current moment may reach the predicted first peak bandwidth. Therefore, the live streaming server activates a peak-shaving strategy to reduce the peak bandwidth of the live stream in advance. Activating the peak-shaving strategy means reducing the bitrate of the first live video stream by the first target bitrate adjustment range. The live streaming server can reduce the bitrate of the first live video stream by the first target bitrate adjustment range through transcoding.

[0204] For example, for each received first live video stream, the live streaming server obtains a first target bitrate based on the bitrate of the first live video stream and the first target bitrate adjustment margin. The first target bitrate is the bitrate of the first live video stream after downsampling. Assuming the bitrate of the first live video stream is N1 and the first target bitrate adjustment margin is R1%, the first target bitrate T1 = N1 * (1 - R1%). Then, the live streaming server inputs the first live video stream and the first target bitrate into a video transcoder, which transcodes the first live video stream into a second live video stream with the first target bitrate and outputs the second live video stream.

[0205] When reducing the bitrate of the first live video stream through transcoding, at least one transcoding parameter may be involved. The transcoding parameter values ​​required to transcode the second live video stream can be predicted first, and then transcoding can be performed based on these transcoding parameter values. The following section, in conjunction with steps 4051-4053, will provide a detailed explanation of this method.

[0206] In step 4051, if the first bandwidth is greater than the first peak bandwidth and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, the live streaming server extracts the video features of the first live video stream to obtain the first video features, which are used to describe the live video stream.

[0207] The video features of any live video stream include at least one of the basic video features and video content features. The basic video features are used to describe the live video stream at a lower dimension. For example, the basic video features include the time complexity and space complexity of the live video stream. The time complexity characterizes the amount of change of the live video stream over time, and the space complexity characterizes the texture complexity of the video frames. In other embodiments, the basic video features also include other low-dimensional features besides time complexity and space complexity. This disclosure does not limit the scope of these other low-dimensional features. The video content features are used to describe the video content of the live video stream. For example, the video content features include the content category to which the live video stream belongs (e.g., food video type, pet video type, fashion video type, etc.). In other embodiments, the video features also include other features describing the video content besides the content category, such as features deeper than the content category. This disclosure does not limit the scope of these other features describing the video content.

[0208] In some embodiments, for each first live video stream, the live streaming server inputs the first live video stream into a video feature extraction model, which extracts the video features of the first live video stream and outputs the first video features of the first live video stream.

[0209] Among them, the video feature extraction model is a network model trained to extract video features from live video streams. This network model can be a neural network model, a machine learning model, or a deep learning model.

[0210] In other embodiments, the live streaming server may extract video features of the first live video stream without using a video feature extraction model, but through other methods. Here, this disclosure does not limit the other methods, as long as they can extract the video features of the first live video stream.

[0211] In step 4052, the live streaming server predicts the transcoding parameter values ​​required for the first live streaming video stream to reach the first transcoding target based on the first video features, and obtains the first transcoding parameter values. The first transcoding target includes reducing the bitrate of the live streaming video stream by the first target bitrate adjustment range and ensuring that the video quality evaluation value of the transcoded live streaming video stream meets the target conditions.

[0212] In this process, at least one transcoding parameter that affects bitrate and video clarity may be involved in transcoding the video stream. Based on the first video features, the live streaming server predicts at least one transcoding parameter value required for the first live video stream to reach the first transcoding target, thereby obtaining at least one first transcoding target parameter value. This at least one transcoding parameter includes CRF (Constant Rate Factor), VBV-maxrate (Video Buffering Verifier-Maxrate), and at least one other transcoding parameter. In this embodiment, no limitation is made to the other transcoding parameters.

[0213] The video quality assessment value of a live video stream is the parameter value of VQA (Video Quality Assessment). VQA parameters are objective indicators used to evaluate subjective video quality experience. VQA parameters include VMAF (Video Multimethod Assessment Fusion) parameters, SSIM (Structural Similarity Index), or other VQA parameters. Among them, VMAF is used to reflect subjective image quality evaluation, while SSIM is used to reflect the degree of similarity between video frames.

[0214] The video quality evaluation value of the transcoded live video stream meets the target condition, meaning that the highest video quality evaluation value that the transcoded at least one live video stream can achieve when the average bitrate of at least one live video stream is reduced by the first target bitrate adjustment amount. In this step 4052, the at least one live video stream is also the at least one first live video stream to be sent.

[0215] In some embodiments, transcoding parameter values ​​are predicted using a prediction model of transcoding parameters. Step 4052 is described below in conjunction with steps A1 and A2.

[0216] In step A1, the live streaming server obtains the first transcoding parameter prediction model corresponding to the first target bitrate adjustment range.

[0217] The first transcoding parameter prediction model has the function of predicting the transcoding parameter values ​​required for the live video stream to reach the first transcoding target based on the input video features. In one possible implementation, the first transcoding parameter prediction model is a neural network model, such as a shallow neural network, wherein the number of network layers in the shallow neural network is less than a first threshold, such as 3, 4, or 5. Taking a first threshold of 3 as an example, the shallow neural network includes an input layer, a fully connected layer, and an output layer. This embodiment of the present disclosure does not limit the first threshold. In another possible implementation, the first transcoding parameter prediction model may also be a neural network model other than a shallow neural network; this embodiment of the present disclosure does not limit other neural network models.

[0218] In some embodiments, the live streaming server stores at least one first transcoding parameter prediction model, each first transcoding parameter prediction model corresponding to a bitrate reduction adjustment range, and the live streaming server obtains the first transcoding parameter prediction model corresponding to the first target bitrate adjustment range from the stored at least one first transcoding parameter prediction model.

[0219] The at least one first transcoding parameter prediction model can be trained by the live streaming server or by a device other than the live streaming server. The following steps B1-B4, taking the first transcoding parameter prediction model corresponding to the first target bitrate adjustment range as an example, will be used to describe in detail the process of training the first transcoding parameter prediction model.

[0220] In step B1, the live streaming server receives the first historical live video streams from multiple third terminals.

[0221] Among them, the third terminal is the terminal that conducted video live streaming in the historical time period before the current time period, and the first historical live video stream is the live video stream sent by the third terminal to the live streaming server in that historical time period. The live streaming server can receive multiple first historical live video streams in that historical time period.

[0222] In step B2, the live streaming server predicts the transcoding parameter values ​​required for the first historical live streaming video stream to reach the first transcoding target based on multiple first bitrate adjustment ranges, and obtains the first historical transcoding parameter values.

[0223] The multiple first bitrate adjustment ranges are between 0 and 1. The first historical transcoding parameter is the parameter value of at least one transcoding parameter predicted based on the multiple first bitrate adjustment ranges. If there is at least one transcoding parameter, there is also one first historical transcoding parameter value. If there are multiple transcoding parameters, there are multiple first historical transcoding parameter values. These multiple first historical transcoding parameter values ​​are combinations of the parameter values ​​of the multiple transcoding parameters, which can be understood as a set of first historical transcoding parameter values. This set of first historical transcoding parameter values ​​represents the combination of transcoding parameter values ​​that achieves the highest video quality evaluation among the multiple second historical live video streams when the average bitrate of the multiple first historical live video streams is reduced by the first target bitrate adjustment range. The second historical live video stream is the first historical live video stream after the bitrate reduction.

[0224] In one possible implementation, the live streaming server transcodes multiple first historical live video streams using multiple first bitrate adjustment increments, and then predicts the first historical transcoding parameter values ​​based on the transcoding results. This approach is described below in conjunction with steps B21-B23.

[0225] In step B21, the live streaming server transcodes the first historical live streaming video stream multiple times based on multiple first bitrate adjustment ranges to obtain multiple second historical live streaming video streams, each second historical live streaming video stream having a lower bitrate than the first historical live streaming video stream.

[0226] In this process, the live streaming server executes step B21 for each first historical live video stream, thus each first historical live video stream corresponds to multiple second historical live video streams. Taking any first historical live video stream as an example, step B21 is described as follows:

[0227] For each of the multiple first bitrate adjustment ranges, the live streaming server obtains a second target bitrate based on the bitrate of the first historical live video stream and the first bitrate adjustment range. The second target bitrate is the bitrate of the first historical live video stream after the bitrate reduction. The method for obtaining the second target bitrate is the same as that for obtaining the first target bitrate, and will not be repeated here. Then, the live streaming server inputs the first historical live video stream and the second target bitrate into a video transcoder. The video transcoder transcodes the first historical live video stream into a second historical live video stream with the second target bitrate and outputs the second historical live video stream, completing one transcoding cycle. The live streaming server also obtains the parameter values ​​(i.e., a set of transcoding parameter values) of at least one transcoding parameter used by the video transcoder when converting the first historical live video stream into the second historical live video stream.

[0228] Accordingly, the live streaming server performs multiple transcoding operations on the first historical live streaming video stream based on multiple first bitrate adjustment ranges, resulting in multiple second historical live streaming video streams and a set of transcoding parameter values ​​used to transcode each second historical live streaming video stream.

[0229] In step B22, the live streaming server obtains the transcoding information of the multiple second historical live streaming video streams. The transcoding information of each second historical live streaming video stream includes the first bitrate adjustment range used when obtaining the second historical live streaming video stream and the transcoding parameter value.

[0230] In this process, the live streaming server executes step B22 on multiple second historical live streaming streams corresponding to each first historical live streaming stream, thereby obtaining the transcoding information of each second historical live streaming stream. Taking the acquisition of transcoding information of any second historical live streaming stream as an example, step B22 is described as follows:

[0231] The live streaming server will transcode the second historical live video stream into the first bitrate adjustment range and a set of transcoding parameter values ​​used by the second historical live video stream, as the transcoding information of the second historical live video stream.

[0232] In step B23, the live streaming server obtains the transcoding parameter values ​​required for the first historical live streaming video stream to reach the first transcoding target based on the transcoding information of the multiple second historical live streaming video streams.

[0233] In some embodiments, the live streaming server uses a dynamic programming algorithm model to obtain the transcoding parameter values ​​required for the first historical live streaming video stream to reach the first transcoding target, based on the transcoding information of the plurality of second historical live streaming video streams. The dynamic programming algorithm model includes a knapsack algorithm model.

[0234] For example, the live streaming server inputs the transcoding information of multiple second historical live streaming video streams and the target conditions into the dynamic programming algorithm model. Based on the transcoding information of multiple second historical live streaming video streams, the dynamic programming algorithm model determines the transcoding parameter values ​​required for the first historical live streaming video stream to reach the first transcoding target, and outputs a set of transcoding parameter values.

[0235] Taking multiple transcoding parameters as an example, the dynamic programming algorithm model, based on the transcoding information of multiple second historical live video streams, aims to achieve the highest video quality evaluation among the multiple second historical live video streams when the average bitrate of the multiple first historical live video streams is reduced by the first target bitrate adjustment. It then dynamically programs the transcoding parameter values ​​from the multiple transcoding information to obtain a combination of these values—a set of first historical transcoding parameter values. When transcoding live video streams using this set of first historical transcoding parameter values, the transcoded live video streams can reach or approach the first transcoding target. Specifically, this set of first historical transcoding parameter values ​​represents the combination of transcoding parameter values ​​that achieves the highest video quality evaluation among the multiple second historical live video streams when the average bitrate of the multiple first historical live video streams is reduced by the first target bitrate adjustment.

[0236] For the process shown in steps B21-B23 above, the live streaming server first transcodes the first historical live video stream into multiple second historical indicator video streams. Based on the transcoding information of the multiple second historical live video streams, it obtains the transcoding parameters required for the first historical live video stream to reach the first transcoding target. This allows the video features of the multiple first historical live video streams to be used as training samples, and the transcoding parameters required for the multiple first historical live video streams to reach the first transcoding target to be used as the expected output, to train a first transcoding parameter prediction model. This enables the trained first transcoding parameter prediction model to learn the logic for predicting and calculating the transcoding parameter values ​​required for the live video stream to reach the first transcoding target based on the video features of the live video stream.

[0237] In step B3, the live streaming server extracts the video features of the first historical live streaming video stream to obtain the first historical video features.

[0238] The first historical video feature is the video feature of the first historical live video stream. The extraction method can refer to step 4051 above. Here, this embodiment of the disclosure will not repeat step B3.

[0239] In step B4, the live streaming server trains a first transcoding parameter prediction model based on the first historical transcoding parameter values ​​and the first historical video features.

[0240] The obtained first transcoding parameter prediction model corresponds to the first target bitrate adjustment value.

[0241] For example, the first historical video features of multiple first historical live video streams are used as input data to a shallow neural network. A set of first historical transcoding parameter values ​​is obtained and used as the model's ground truth (correct data), i.e., the expected output. The shallow neural network is then iteratively trained. During each training iteration, the shallow neural network outputs a set of transcoding parameter values ​​based on the first historical video features of multiple first historical live video streams using the current model parameters. If the error between the output transcoding parameter values ​​and the first historical transcoding parameter values ​​is greater than a second threshold, the current model parameters are updated, and the process continues until multiple transcoding parameter values ​​with an error less than the second threshold are generated consecutively. Training then ends. Through training, the shallow neural network learns the logic for predicting and calculating the transcoding parameter values ​​required for the live video stream to achieve the first transcoding target based on the video features of the live video stream. The trained shallow neural network is thus the first transcoding parameter prediction model corresponding to the first target bitrate adjustment range. The second threshold can be set according to the specific implementation scenario; however, this embodiment does not limit the second threshold.

[0242] In step A2, the live streaming server inputs the first video feature into the corresponding first transcoding parameter prediction model. Based on the first video feature, the first transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target and outputs the first transcoding parameter values.

[0243] Since the first transcoding parameter prediction model corresponding to the first target bitrate adjustment magnitude has learned the calculation logic for predicting the transcoding parameter values ​​required for the live video stream to reach the first transcoding target based on the video features of the live video stream, the live server inputs the first video features into the corresponding first transcoding parameter prediction model. Based on the first video features, the first transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target, and outputs a set of first transcoding parameter values ​​that enable the first live video stream to reach the first transcoding target. This ensures that when transcoding the first live video stream using these first transcoding parameter values, the transcoded live video stream conforms to the first transcoding target of the first live video stream. Furthermore, when the first transcoding parameter model is a shallow neural network, the number of network layers is small, facilitating real-time online prediction.

[0244] In step 4053, the live streaming server uses the first transcoding parameter value to transcode the first live video stream.

[0245] For example, for each first live video stream, the live server inputs a set of first transcoding parameter values ​​and the first live video stream into a video transcoder. The video transcoder then transcodes the first live video stream according to the input set of first transcoding parameter values ​​and outputs a second live video stream.

[0246] On the one hand, the increased peak bandwidth consumption of live streaming platforms during peak periods leads to increased buffering rates in live videos. On the other hand, under the Content Delivery Network (CDN) bandwidth billing method, CDN bandwidth billing references the peak bandwidth of the live streaming platform during peak periods, not the situation where the peak bandwidth is not reached. Therefore, peak bandwidth determines the bandwidth cost of the live streaming platform. To reduce the buffering rate and bandwidth cost of live videos, related technologies involve the live streaming server reducing the resolution of the live video stream played on terminals (such as the first terminal and the second terminal). When users have a high-resolution requirement for the live video stream, they actively select high-resolution playback on their terminals. This solution reduces the user's experience of the clarity of the live video stream, increases the operational cost for users who pursue a higher resolution experience, and leads to an average decrease in the resolution of the target program, increasing user churn. In this embodiment of the disclosure, since the first transcoding parameters can reduce the average bitrate of multiple first live video streams by the first target bitrate adjustment range, so that the video quality of the second live video stream obtained after transcoding reaches the highest level, the live bandwidth can be reduced when sending the transcoded first live video stream, thus avoiding the live bandwidth from reaching the predicted peak bandwidth, thereby achieving the effect of peak shaving of live bandwidth, and also ensuring the video quality of the transcoded live video stream.

[0247] In step 406, the live streaming server sends a second live video stream to the second terminal.

[0248] The second terminal can be any terminal that requests the first live video stream from the live streaming server.

[0249] For example, for any first live video stream, in response to any second terminal's request to obtain the first live video stream, the live streaming server sends a second live video stream with a reduced bitrate to the second terminal.

[0250] It should be noted that when the peak shaving strategy is enabled, the live streaming server reduces the bitrate of each received live video stream by an average of the first target bitrate adjustment magnitude before pushing the reduced bitrate live video stream to the terminal used by the viewer. This continues until the first bandwidth is less than the first peak bandwidth, and the ratio between the first bandwidth and the first peak bandwidth is less than the first target bitrate adjustment magnitude. If the difference between the first target bitrate adjustment magnitude and this ratio is greater than a third threshold, it indicates that the current moment is not a peak period for the live video service. In this case, the live streaming server disables the peak shaving strategy, that is, it does not reduce the bitrate of the first live video stream received at the current moment and sends the received first live video stream to the second terminal.

[0251] In step 407, the second terminal receives and plays the second live video stream.

[0252] The method provided in this disclosure reduces the bitrate of the received live video stream when the live streaming server determines the first bandwidth for sending the live video stream at the current moment and predicts the peak bandwidth of the live streaming server. If the first bandwidth is lower than the predicted peak bandwidth of the live streaming server, and the ratio between the two bandwidths is greater than or equal to the first target bitrate adjustment range, the bandwidth occupied when pushing the live video stream with the reduced bitrate to the terminal will also be reduced, so as to avoid the bandwidth occupied when pushing the video stream reaching the predicted peak bandwidth, thereby reducing the peak bandwidth of the live streaming service.

[0253] In other embodiments, when the current live streaming bandwidth is significantly lower than the predicted peak bandwidth, the live streaming server can also appropriately increase the occupied live streaming bandwidth by increasing the bitrate of the live video stream to be sent at the current moment, even if the peak bandwidth is not reached.

[0254] Next, we will provide a detailed explanation of the process of increasing the bitrate of the live video stream to be sent on the live streaming server:

[0255] Figure 5 This is an interactive flowchart illustrating a video processing method according to an exemplary embodiment. See also: Figure 5 The video processing method includes the following steps.

[0256] In step 501, the first terminal sends the first live video stream to the live streaming server.

[0257] Step 501 is similar to step 401, and will not be described again in this embodiment.

[0258] In step 502, the live streaming server receives the first live video stream from the first terminal.

[0259] Step 502 is similar to step 402, and will not be described again in this embodiment.

[0260] In step 503, the live streaming server determines the total bandwidth for sending the live video stream at the current moment, and obtains the first bandwidth.

[0261] Step 503 is similar to step 431, and will not be described again in this embodiment.

[0262] In step 504, the live streaming server predicts the peak bandwidth of the live streaming server to obtain the first peak bandwidth.

[0263] Step 504 is similar to step 404, and will not be described again in this embodiment.

[0264] In step 505, if the first bandwidth is less than the first peak bandwidth and the difference between the first bandwidth and the first peak bandwidth is greater than the target difference, the live streaming server increases the bitrate of the first live video stream by the second target bitrate adjustment range to obtain the third live video stream.

[0265] The target difference is a value greater than 0, and can be set according to the actual application scenario. This embodiment does not limit the target difference. The second target bitrate adjustment range is the bitrate adjustment range corresponding to the current time period. In some embodiments, the live streaming server stores the bitrate adjustment range corresponding to the target time period. The live streaming server first obtains the stored bitrate adjustment range corresponding to the target time period as the second target bitrate adjustment range, and then executes step 505. In other embodiments, the live streaming server does not store the bitrate adjustment range corresponding to the target time period. The second target bitrate adjustment range is obtained based on the first bandwidth, the first peak bandwidth, and a safety factor. The safety factor is used to mitigate the impact of the difference between the first bandwidth and the second bandwidth on bandwidth cost. The second bandwidth is the bandwidth required to send the third live video stream. The safety factor ranges from 0 to 1. For example, assuming the first bandwidth is A, the first peak bandwidth is C, and the safety factor is h, the second target bitrate adjustment range RX% = C / A*100*h. Understandably, the second target bitrate adjustment range generated based on the safety factor can prevent the bandwidth occupied by the live video stream with the increased bitrate from exceeding the peak bandwidth when increasing the bitrate of the live video stream.

[0266] In some embodiments, the live streaming server increases the bitrate of the first live video stream through transcoding. For example, for each received first live video stream, the live streaming server obtains a third target bitrate based on the bitrate of the first live video stream and the adjustment range of the second target bitrate. The third target bitrate is the bitrate of the first live video stream after the bitrate increase. Assuming the bitrate of the first live video stream is N1, the adjustment range of the second target bitrate is R2%, and the third target bitrate T2 = N1 * (1 - R2%), the live streaming server then inputs the first live video stream and the third target bitrate into a video transcoder, which converts the first live video stream into a third live video stream with the third target bitrate and outputs the third live video stream.

[0267] When increasing the bitrate of the first live video stream through transcoding, at least one transcoding parameter may be involved. The transcoding parameter values ​​required to transcode into the third live video stream can be predicted first, and then transcoding can be performed based on these transcoding parameter values. The following section, in conjunction with steps 5051-5053, will provide a detailed explanation of this method.

[0268] In step 5051, if the first bandwidth is less than the first peak bandwidth and the difference between the first bandwidth and the first peak bandwidth is greater than the target difference, the live streaming server extracts the video features of the first live video stream to obtain the first video feature, which is used to describe the live video stream.

[0269] In this embodiment, the live streaming server receives at least one first live video stream at the current moment. Step 5051 is performed for each first live video stream. For the process of extracting video features, please refer to step 4041 above. In this embodiment, step 5051 will not be described again.

[0270] In step 5052, the live streaming server predicts the transcoding parameter values ​​required for the first live streaming video stream to reach the second transcoding target based on the first video features, and obtains the second transcoding parameter values. The second transcoding target includes increasing the bitrate of the live streaming video stream by the second target bitrate adjustment range and ensuring that the video quality evaluation value of the transcoded live streaming video stream meets the target conditions.

[0271] In the process of transcoding video streams, at least one transcoding parameter may be involved that affects bitrate and video clarity. Based on the first video features, the live streaming server predicts at least one transcoding parameter value required for the first live video stream to reach the second transcoding target, and obtains at least one second transcoding target parameter value.

[0272] In some embodiments, transcoding parameter values ​​are predicted using a prediction model of transcoding parameters. Step 5052 is described below in conjunction with steps C1 and C2.

[0273] In step C1, the live streaming server obtains the second transcoding parameter prediction model corresponding to the second target bitrate adjustment magnitude.

[0274] The second transcoding parameter prediction model has the function of predicting the transcoding parameter values ​​required for the live video stream to reach the second transcoding target based on the input video features. In one possible implementation, the second transcoding parameter prediction model is a neural network model, such as a shallow neural network. In another possible implementation, the second transcoding parameter prediction model may also be a neural network model other than a shallow neural network. This disclosure does not limit other neural network models.

[0275] In some embodiments, the live streaming server stores at least one second transcoding parameter prediction model, each second transcoding parameter prediction model corresponding to a bitrate adjustment range, and the live streaming server obtains the second transcoding parameter prediction model corresponding to the second target bitrate adjustment range from the stored at least one first transcoding parameter prediction model.

[0276] The at least one second transcoding parameter prediction model can be trained by the live streaming server or by a device other than the live streaming server. The process of training the second transcoding parameter prediction model will be described in detail below, taking the second transcoding parameter prediction model corresponding to the second target bitrate adjustment magnitude as an example, in conjunction with the following steps D1-D4.

[0277] In step D1, the live streaming server receives third historical live video streams from multiple fourth terminals.

[0278] The fourth terminal refers to the terminal that conducted live video streaming during historical time periods prior to the current time period. The third historical live video stream is the live video stream sent by the fourth terminal to the live streaming server during that historical time period. The live streaming server can receive multiple third historical live video streams during that historical time period. The fourth terminal may be the third terminal mentioned above, or it may not be a third terminal. If it is a third terminal, the third historical live video stream is also the first historical live video stream.

[0279] In step D2, the live streaming server predicts the transcoding parameter values ​​required for the third historical live streaming video stream to reach the second transcoding target based on multiple second bitrate adjustment ranges, and obtains the second historical transcoding parameter values.

[0280] The multiple second bitrate adjustment magnitudes range from 0 to 1. The second historical transcoding parameter is the parameter value of at least one transcoding parameter predicted based on the multiple second bitrate adjustment magnitudes. If there is at least one transcoding parameter, there is also one second historical transcoding parameter value. If there are multiple transcoding parameters, there are multiple second historical transcoding parameter values. These multiple second historical transcoding parameter values ​​are combinations of the parameter values ​​of the multiple transcoding parameters, which can be understood as a set of second historical transcoding parameter values. This set of second historical transcoding parameter values ​​represents the combination of transcoding parameter values ​​that achieves the highest video quality evaluation for multiple fourth historical live video streams when the average bitrate of the multiple third historical live video streams is increased by the second target bitrate adjustment magnitude. The fourth historical live video stream is the third live video stream after the bitrate increase.

[0281] In one possible implementation, the live streaming server first transcodes multiple third historical live video streams using multiple second bitrate adjustment amplitudes, and then predicts the second historical transcoding parameter values ​​based on the transcoding results. This method is described below in conjunction with steps D21-D23.

[0282] In step D21, the live streaming server performs multiple second bitrate adjustments to transcode the third historical live streaming video stream multiple times, resulting in multiple fourth historical live streaming video streams, each with a bitrate higher than that of the third historical live streaming video stream.

[0283] In this process, the live streaming server executes step D21 for each third historical live video stream, thus each third historical live video stream corresponds to multiple fourth historical live video streams. Taking any third historical live video stream as an example, step D21 is described as follows:

[0284] For each of the multiple second bitrate adjustment increments, the live streaming server obtains a fourth target bitrate based on the bitrate of the third historical live video stream and the second bitrate adjustment increment. The fourth target bitrate is the bitrate of the third historical live video stream after bitrate incrementing. The method for obtaining the fourth target bitrate is the same as that for the third target bitrate, and will not be repeated here. Then, the live streaming server inputs the third historical live video stream and the fourth target bitrate into a video transcoder. The video transcoder converts the third historical live video stream into a fourth historical live video stream with the fourth target bitrate and outputs the fourth historical live video stream, completing one transcoding cycle. The live streaming server also obtains the parameter values ​​(i.e., a set of transcoding parameter values) of at least one transcoding parameter used by the video transcoder when converting the third historical live video stream into the fourth historical live video stream.

[0285] Accordingly, the live streaming server performs multiple transcodings on the third historical live streaming video stream based on multiple second bitrate adjustment ranges, resulting in multiple fourth historical live streaming video streams and a set of transcoding parameter values ​​used to transcode each fourth historical live streaming video stream.

[0286] In step D22, the live streaming server obtains transcoding information of multiple fourth historical live streaming video streams. The transcoding information of each fourth historical live streaming video stream includes the second bitrate adjustment range used when obtaining the fourth historical live streaming video stream and the transcoding parameter values.

[0287] In this process, the live streaming server executes step D22 on multiple fourth historical live streaming streams corresponding to each third historical live streaming stream, thereby obtaining the transcoding information of each fourth historical live streaming stream. Taking the acquisition of transcoding information of any fourth historical live streaming stream as an example, step D22 is described as follows:

[0288] The live streaming server will transcode the video stream into the second bitrate adjustment range used by the SI historical live streaming video stream and a set of transcoding parameter values, as the transcoding information for the fourth historical live streaming video stream.

[0289] In step D23, the live streaming server obtains the transcoding parameter values ​​required for the third historical live streaming video stream to reach the second transcoding target based on the transcoding information of multiple fourth historical live streaming video streams.

[0290] In some embodiments, the live streaming server uses a dynamic programming algorithm model to obtain the transcoding parameter values ​​required for the third historical live streaming video stream to reach the first transcoding target, which is a set of second historical transcoding parameter values, based on the transcoding information of the multiple fourth historical live streaming video streams. The dynamic programming algorithm model includes a knapsack algorithm model.

[0291] The process of obtaining the second historical transcoding parameter values ​​using a dynamic programming algorithm model is similar to that of obtaining the first historical transcoding parameter values. The difference lies in that step D23 requires planning the transcoding parameter values ​​needed to make the third historical live video stream reach the second transcoding target. Similarly, the parameter value combination planned in step D23 forms a set of second historical transcoding parameter values. When transcoding the live video stream using this set of second historical transcoding parameter values, the live video streams participating in the transcoding can reach or approach the second transcoding target. Specifically, this set of second historical transcoding parameter values ​​represents the combination of transcoding parameter values ​​that achieves the highest video quality evaluation among multiple fourth historical live video streams when the average bitrate of multiple third historical live video streams is increased by the second target bitrate adjustment magnitude.

[0292] For the process shown in steps D21-D23 above, the live streaming server first transcodes the third historical live video stream into multiple fourth historical indicator video streams. Based on the transcoding information of the multiple fourth historical live video streams, it obtains the transcoding parameters required for the third historical live video stream to reach the second transcoding target. This allows the video features of the multiple third historical live video streams to be used as training samples, and the transcoding parameters required for the multiple third historical live video streams to reach the second transcoding target to be used as the expected output. This trains a second transcoding parameter prediction model, enabling the trained first transcoding parameter prediction model to learn the logic for predicting and calculating the transcoding parameter values ​​required for the live video stream to reach the first transcoding target based on the video features of the live video stream.

[0293] In step D3, the live streaming server extracts the video features of the third historical live streaming video stream to obtain the second historical video features.

[0294] The second historical video feature is the video feature of the third historical live video stream. The extraction method can refer to step 4051 above. Here, this embodiment of the disclosure will not repeat step D3.

[0295] In step D4, the live streaming server trains a second transcoding parameter prediction model based on the second historical transcoding parameter values ​​and the second historical video features.

[0296] The obtained second transcoding parameter prediction model corresponds to the second target bitrate adjustment value.

[0297] For example, the second historical video features of multiple third-generation historical live video streams are used as input data to a shallow neural network. The resulting set of second historical transcoding parameter values ​​is used as the model's ground truth (correct data), i.e., the expected output. The shallow neural network is then iteratively trained. In each training process, the shallow neural network outputs a set of transcoding parameter values ​​based on the second historical video features of multiple third-generation historical live video streams using the current model parameters. If the error between the output set of transcoding parameter values ​​and the set of second historical transcoding parameter values ​​is greater than a second threshold, the current model parameters are updated, and the next training process begins. This continues until multiple transcoding parameter values ​​with an error less than the second threshold are generated consecutively, at which point training ends. Through training, the shallow neural network learns the logic for predicting and calculating the transcoding parameter values ​​required for the live video stream to achieve the second transcoding target based on the video features of the live video stream. The trained shallow neural network is thus the second transcoding parameter prediction model corresponding to the second target bitrate adjustment range.

[0298] In step C2, the live streaming server inputs the first video feature into the second transcoding parameter prediction model corresponding to the second target bitrate adjustment range. Based on the first video feature, the second transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live streaming video stream to reach the second transcoding target and outputs the second transcoding parameter values.

[0299] Since the second transcoding parameter prediction model corresponding to the second target bitrate adjustment range has learned the calculation logic of predicting the transcoding parameter values ​​required for the live video stream to reach the second transcoding target based on the video features of the live video stream, the live server inputs the second video features into the corresponding second transcoding parameter prediction model. Based on the first video features, the second transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target and outputs a set of second transcoding parameter values ​​that enable the first live video stream to reach the second transcoding target. This ensures that when the first live video stream is transcoded using this set of second transcoding parameter values, the transcoded live video stream can meet the second transcoding target of the first live video stream. In addition, when the second transcoding parameter model is a shallow neural network, the number of network layers in the second transcoding parameter model is small, which is convenient for real-time online prediction.

[0300] In step 5053, the live streaming server uses the second transcoding parameter value to transcode the first live video stream.

[0301] For example, for each first live video stream, the live server inputs a set of second transcoding parameter values ​​and the first live video stream into a video transcoder. The video transcoder then transcodes the first live video stream according to the input set of second transcoding parameter values ​​and outputs a third live video stream.

[0302] Since the bandwidth cost of a live streaming platform is related to the peak bandwidth consumed during peak periods, in order to reduce bandwidth costs, in related technologies, the live streaming server sets the resolution of the live video stream played by terminals (such as the first terminal and the second terminal) to the highest level. When the live streaming bandwidth is low (such as during off-peak periods), increasing the resolution will lead to an increase in the stuttering rate, and the improvement in resolution is limited by the bitrate, so the effect of improving resolution is limited. However, in this embodiment of the disclosure, since the second transcoding parameters can increase the average bitrate of multiple first live video streams by a second target bitrate adjustment, the video quality of the transcoded third live video stream reaches the highest level. Since the second target bitrate adjustment can control the live streaming bandwidth required by the transcoded third live video stream to not reach the peak bandwidth, the video quality of the live video stream is improved without increasing bandwidth costs.

[0303] In step 506, the live streaming server sends a third live video stream to the second terminal.

[0304] Step 506 is similar to step 406, and will not be described in detail here.

[0305] In some embodiments, if the first bandwidth is less than the first peak bandwidth and the difference between the first bandwidth and the first peak bandwidth is less than or equal to the target difference, it indicates that the current time is not a low period for the video live streaming service. In this case, the live streaming server does not increase the bit rate of the first live video received at the current time and sends the received first live video stream to the second terminal.

[0306] In step 507, the second terminal receives and plays the third live video stream.

[0307] The method provided in this disclosure determines the first bandwidth of the live video stream to be sent at the current moment and the predicted peak bandwidth of the live video server through the live streaming server. When the first bandwidth is less than the predicted peak bandwidth and the difference between the two bandwidths is greater than the target difference, the difference between the first bandwidth and the predicted peak bandwidth at the current moment is large. The current moment is obviously not in the peak period of the video live streaming service. The current moment may be in the off-peak period or not. By increasing the bitrate of the live video stream to be sent by a second target bitrate adjustment range, the video quality (such as image clarity) of the live video stream to be sent is improved. Furthermore, the second target bitrate adjustment range controls the live bandwidth required for the live video stream to be sent after the bitrate is increased to reach the predicted peak bandwidth. Thus, the quality of the sent live video stream is improved without increasing bandwidth costs.

[0308] Figure 6 This is a structural block diagram of a video processing apparatus according to an exemplary embodiment, with reference to... Figure 6 According to another aspect of the present disclosure, the apparatus 600 provides a video processing apparatus applied to a live streaming server, the apparatus 600 comprising:

[0309] The receiving unit 601 is configured to receive the first live video stream from the first terminal;

[0310] The determining unit 602 is configured to determine the total bandwidth of the live video stream sent by the live streaming server at the current moment, and obtain the first bandwidth;

[0311] The prediction unit 603 is configured to predict the peak bandwidth of the live streaming server to obtain a first peak bandwidth;

[0312] The bitrate adjustment unit 604 is configured to reduce the bitrate of the first live video stream by the first target bitrate adjustment range if the first bandwidth is less than the first peak bandwidth and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, thereby obtaining a second live video stream.

[0313] The sending unit 605 is configured to send the second live video stream to the second terminal.

[0314] Optionally, the bit rate adjustment unit 603 includes:

[0315] The first extraction subunit is configured to extract video features from the first live video stream if the first bandwidth is less than the first peak bandwidth and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, thereby obtaining a first video feature, which is used to describe the live video stream.

[0316] The first prediction subunit is configured to perform prediction of the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target based on the first video features, and obtain the first transcoding parameter values. The first transcoding target includes reducing the bitrate of the live video stream by the first target bitrate adjustment range and ensuring that the video quality evaluation value of the transcoded live video stream meets the target conditions.

[0317] The first transcoding subunit is configured to transcode the first live video stream using the first transcoding parameter value.

[0318] Optionally, the first prediction subunit is configured to perform:

[0319] Obtain the first transcoding parameter prediction model corresponding to the first target bitrate adjustment range, wherein the first transcoding parameter prediction model is a shallow neural network.

[0320] The first video feature is input into the first transcoding parameter prediction model. Based on the first video feature, the first transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target, and outputs the first transcoding parameter values.

[0321] Optionally, the device 600 further includes a first feature extraction unit and a first training unit;

[0322] The receiving unit 601 is also configured to receive a first historical live video stream from multiple third terminals;

[0323] The prediction unit 603 is further configured to perform a prediction based on multiple first bitrate adjustment magnitudes of the transcoding parameter values ​​required for the first historical live video stream to reach the first transcoding target, thereby obtaining the first historical transcoding parameter values.

[0324] The first feature extraction unit is configured to extract the video features from the first historical live video stream to obtain the first historical video features;

[0325] The first training unit is configured to train the first transcoding parameter prediction model based on the first historical transcoding parameter values ​​and the first historical video features.

[0326] Optionally, the prediction unit 603 is configured to perform:

[0327] Based on the multiple first bitrate adjustment ranges, the first historical live video stream is transcoded multiple times to obtain multiple second historical live video streams, each of which has a lower bitrate than the first historical live video stream.

[0328] Obtain transcoding information of the plurality of second historical live video streams, wherein the transcoding information of each second historical live video stream includes the first bitrate adjustment range and transcoding parameter values ​​used when obtaining the second historical live video stream;

[0329] Based on the transcoding information of the multiple second historical live video streams, the transcoding parameter values ​​required for the first historical live video stream to reach the first transcoding target are obtained.

[0330] Optionally, the bitrate adjustment unit 604 is further configured to perform the following: if the first bandwidth is less than the first peak bandwidth and the difference between the first bandwidth and the first peak bandwidth is greater than a target difference, increase the bitrate of the first live video stream by a second target bitrate adjustment amount to obtain a third live video stream.

[0331] The sending unit 605 is also configured to send the third live video stream to the second terminal.

[0332] Optionally, the bit rate adjustment unit 604 includes:

[0333] The second feature extraction subunit is configured to extract video features from the first live video stream if the first bandwidth is less than the first peak bandwidth and the difference between the first bandwidth and the first peak bandwidth is greater than a target difference, thereby obtaining a first video feature, which is used to describe the live video stream.

[0334] The second prediction subunit is configured to perform prediction of the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target based on the first video features, and obtain the second transcoding parameter values. The second transcoding target includes increasing the bitrate of the live video stream by the second target bitrate adjustment range and ensuring that the video quality evaluation value of the transcoded live video stream meets the target conditions.

[0335] The second transcoding subunit is configured to transcode the first live video stream using the second transcoding parameter value.

[0336] Optionally, the second prediction subunit is configured to perform:

[0337] Obtain the second transcoding parameter prediction model corresponding to the second target bitrate adjustment range, wherein the second transcoding parameter prediction model is a shallow neural network;

[0338] The first video feature is input into the second transcoding parameter prediction model. Based on the first video feature, the second transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target, and outputs the second transcoding parameter values.

[0339] Optionally, the method of device 600 further includes a second feature extraction unit and a second training unit;

[0340] The receiving unit 601 is also configured to receive third historical live video streams from multiple fourth terminals;

[0341] The prediction unit 603 is further configured to perform prediction of the transcoding parameter values ​​required for the third historical live video stream to reach the second transcoding target based on multiple second bitrate adjustment amplitudes, thereby obtaining the second historical transcoding parameter values;

[0342] The second feature extraction unit is configured to extract the video features from the third historical live video stream to obtain the second historical video features;

[0343] The second training unit is configured to train the second transcoding parameter prediction model based on the second historical transcoding parameter values ​​and the second historical video features.

[0344] Optionally, the prediction unit 603 is further configured to perform:

[0345] Based on the multiple second bitrate adjustment ranges, the third historical live video stream is transcoded multiple times to obtain multiple fourth historical live video streams, each of which has a higher bitrate than the third historical live video stream.

[0346] Obtain transcoding information of the plurality of fourth historical live video streams, wherein the transcoding information of each fourth historical live video stream includes the second bitrate adjustment range and transcoding parameter values ​​used when obtaining the fourth historical live video stream;

[0347] Based on the transcoding information of the multiple fourth historical live video streams, the transcoding parameter values ​​required for the third historical live video stream to reach the second transcoding target are obtained.

[0348] Optionally, the device 600 further includes:

[0349] The acquisition unit is configured to acquire the second target bitrate adjustment range based on the first bandwidth, the first peak bandwidth, and a safety factor. The safety factor is used to mitigate the impact of the difference between the first bandwidth and the second bandwidth on bandwidth cost. The second bandwidth is the bandwidth required when the third live video stream is sent.

[0350] Optionally, the prediction unit 603 is used for:

[0351] The second peak bandwidth and the historical bitrate adjustment range within the target historical time period are obtained. The second peak bandwidth is the maximum bandwidth occupied when sending live video streams within the target historical time period, and the historical bitrate adjustment range is the bitrate adjustment range used when adjusting the bitrate of live video streams within the target historical time period.

[0352] Based on the second peak bandwidth and the historical bitrate adjustment range, a third peak bandwidth is obtained. The third peak bandwidth is the maximum bandwidth required by the live streaming server to send a live video stream within the target historical time period without adjusting the live video stream bitrate.

[0353] It should be understood that the above-mentioned optional implementation methods can be combined, and the embodiments disclosed herein do not limit the combination methods.

[0354] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the video processing method, and will not be elaborated upon here.

[0355] Figure 7This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. The computer device 700 can be configured as the aforementioned live streaming server. The computer device 700 can vary significantly due to differences in configuration or performance. It may include one or more CPUs (Central Processing Units) 701 and one or more memories 702. The memory 702 stores at least one line of program code, which is loaded and executed by the processor 701 to implement the video processing methods provided in the various embodiments described above. Of course, the computer device 700 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The computer device 700 may also include other components for implementing device functions, which will not be elaborated upon here.

[0356] In an exemplary embodiment, a computer-readable storage medium including at least one instruction is also provided, such as a memory including at least one instruction, which can be executed by a processor in a computer device to perform the video processing method in the above embodiments. Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as ROM (Read-Only Memory), RAM (Random-Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk, and optical data storage device, etc.

[0357] In an exemplary embodiment, a computer program product is also provided, including one or more instructions that can be executed by a processor of a computer device to perform the video processing methods provided in the various embodiments described above.

[0358] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the live video streams involved in this disclosure were all obtained with full authorization.

[0359] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0360] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A video processing method, characterized in that, The method is applied to a live streaming server, and the method includes: Receive the first live video stream from the first terminal; Determine the total bandwidth of the live video stream sent by the live streaming server at the current moment to obtain the first bandwidth; The second peak bandwidth and the historical bitrate adjustment range within the target historical time period are obtained. The second peak bandwidth is the maximum bandwidth occupied by the live streaming server when sending the live video stream within the target historical time period, and the historical bitrate adjustment range is the bitrate adjustment range used when adjusting the bitrate of the live video stream within the target historical time period. Based on the second peak bandwidth and the historical bitrate adjustment range, a third peak bandwidth is obtained, which is the maximum bandwidth required by the live streaming server to send the live video stream within the target historical time period without adjusting the live video stream bitrate. The first peak bandwidth is determined based on the third peak bandwidth. The first peak bandwidth is the peak bandwidth reached by the live streaming server within the target duration after the predicted current moment without adjusting the live streaming bitrate. If the first bandwidth is less than the first peak bandwidth, and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, the bitrate of the first live video stream is reduced by the first target bitrate adjustment range to obtain the second live video stream. The second live video stream is sent to the second terminal.

2. The video processing method according to claim 1, characterized in that, If the first bandwidth is less than the first peak bandwidth, and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, reducing the bitrate of the first live video stream by the first target bitrate adjustment range includes: If the first bandwidth is less than the first peak bandwidth, and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, the video features of the first live video stream are extracted to obtain the first video features, which are used to describe the live video stream. Based on the first video features, the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target are predicted to obtain the first transcoding parameter values. The first transcoding target includes reducing the bitrate of the live video stream by the first target bitrate adjustment range and ensuring that the video quality evaluation value of the transcoded live video stream meets the target conditions. The first live video stream is transcoded using the first transcoding parameter value.

3. The video processing method according to claim 2, characterized in that, The step of predicting the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target based on the first video features, and obtaining the first transcoding parameter values, includes: Obtain the first transcoding parameter prediction model corresponding to the first target bitrate adjustment range, wherein the first transcoding parameter prediction model is a shallow neural network. The first video feature is input into the first transcoding parameter prediction model. Based on the first video feature, the first transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live video stream to reach the first transcoding target, and outputs the first transcoding parameter values.

4. The video processing method according to claim 3, characterized in that, Before receiving the first live video stream from the first terminal, the method further includes: Receive the first historical live video streams from multiple third-party terminals; Based on multiple first bitrate adjustment ranges, the transcoding parameter values ​​required for the first historical live video stream to reach the first transcoding target are predicted to obtain the first historical transcoding parameter values. The video features of the first historical live video stream are extracted to obtain the first historical video features; The first transcoding parameter prediction model is trained based on the first historical transcoding parameter value and the first historical video features.

5. The video processing method according to claim 4, characterized in that, The step of predicting the transcoding parameter values ​​required for the first historical live video stream to reach the first transcoding target based on multiple first bitrate adjustment amplitudes, and obtaining the first historical transcoding parameter values, includes: Based on the multiple first bitrate adjustment ranges, the first historical live video stream is transcoded multiple times to obtain multiple second historical live video streams, each of which has a lower bitrate than the first historical live video stream. Obtain transcoding information of the plurality of second historical live video streams, wherein the transcoding information of each second historical live video stream includes the first bitrate adjustment range and transcoding parameter values ​​used when obtaining the second historical live video stream; Based on the transcoding information of the multiple second historical live video streams, the transcoding parameter values ​​required for the first historical live video stream to reach the first transcoding target are obtained.

6. The video processing method according to claim 1, characterized in that, After determining the first peak bandwidth based on the third peak bandwidth, the method further includes: If the first bandwidth is less than the first peak bandwidth, and the difference between the first bandwidth and the first peak bandwidth is greater than the target difference, the bitrate of the first live video stream is increased by the second target bitrate adjustment range to obtain the third live video stream. The third live video stream is sent to the second terminal.

7. The video processing method according to claim 6, characterized in that, If the first bandwidth is less than the first peak bandwidth, and the difference between the first bandwidth and the first peak bandwidth is greater than the target difference, increasing the bitrate of the first live video stream by the first target bitrate adjustment amount includes: If the first bandwidth is less than the first peak bandwidth, and the difference between the first bandwidth and the first peak bandwidth is greater than the target difference, the video features of the first live video stream are extracted to obtain the first video features, which are used to describe the live video stream. Based on the first video features, the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target are predicted to obtain the second transcoding parameter values. The second transcoding target includes increasing the bitrate of the live video stream by the second target bitrate adjustment range and ensuring that the video quality evaluation value of the transcoded live video stream meets the target conditions. The first live video stream is transcoded using the second transcoding parameter value.

8. The video processing method according to claim 7, characterized in that, The step of predicting the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target based on the first video features, and obtaining the second transcoding parameter values, includes: Obtain the second transcoding parameter prediction model corresponding to the second target bitrate adjustment range, wherein the second transcoding parameter prediction model is a shallow neural network; The first video feature is input into the second transcoding parameter prediction model. Based on the first video feature, the second transcoding parameter prediction model predicts the transcoding parameter values ​​required for the first live video stream to reach the second transcoding target, and outputs the second transcoding parameter values.

9. The video processing method according to claim 8, characterized in that, Before receiving the first live video stream from the first terminal, the method further includes: Receive third-generation historical live video streams from multiple fourth terminals; Based on multiple second bitrate adjustment ranges, the transcoding parameter values ​​required for the third historical live video stream to reach the second transcoding target are predicted to obtain the second historical transcoding parameter values. The video features of the third historical live video stream are extracted to obtain the second historical video features; The second transcoding parameter prediction model is trained based on the second historical transcoding parameter values ​​and the second historical video features.

10. The video processing method according to claim 9, characterized in that, The step of predicting the transcoding parameter values ​​required for the third historical live video stream to reach the second transcoding target based on multiple second bitrate adjustment amplitudes, and obtaining the second historical transcoding parameter values, includes: Based on the multiple second bitrate adjustment ranges, the third historical live video stream is transcoded multiple times to obtain multiple fourth historical live video streams, each of which has a higher bitrate than the third historical live video stream. Obtain transcoding information of the plurality of fourth historical live video streams, wherein the transcoding information of each fourth historical live video stream includes the second bitrate adjustment range and transcoding parameter values ​​used when obtaining the fourth historical live video stream; Based on the transcoding information of the multiple fourth historical live video streams, the transcoding parameter values ​​required for the third historical live video stream to reach the second transcoding target are obtained.

11. The video processing method according to any one of claims 6-10, characterized in that, Before increasing the bitrate of the first live video stream by the second target bitrate adjustment amount, the method further includes: Based on the first bandwidth, the first peak bandwidth, and the safety factor, the second target bitrate adjustment range is obtained. The safety factor is used to mitigate the impact of the difference between the first bandwidth and the second bandwidth on bandwidth cost. The second bandwidth is the bandwidth required when the third live video stream is sent.

12. A video processing apparatus, characterized in that, The device is used in a live streaming server, and the device includes: The receiving unit is configured to receive the first live video stream from the first terminal; The determining unit is configured to determine the total bandwidth of the live video stream sent by the live streaming server at the current moment, and obtain the first bandwidth; The prediction unit is configured to: obtain a second peak bandwidth and a historical bitrate adjustment range within a target historical time period, wherein the second peak bandwidth is the maximum bandwidth occupied by the live streaming server when sending a live video stream within the target historical time period, and the historical bitrate adjustment range is the bitrate adjustment range used when adjusting the bitrate of the live video stream within the target historical time period; based on the second peak bandwidth and the historical bitrate adjustment range, obtain a third peak bandwidth, wherein the third peak bandwidth is the maximum bandwidth required by the live streaming server to send a live video stream within the target historical time period without adjusting the bitrate of the live video stream; and determine a first peak bandwidth based on the third peak bandwidth, wherein the first peak bandwidth is the peak bandwidth reached by the live streaming server within a target duration after the predicted current moment without adjusting the bitrate of the live stream. The bitrate adjustment unit is configured to reduce the bitrate of the first live video stream by the first target bitrate adjustment range if the first bandwidth is less than the first peak bandwidth and the ratio between the first bandwidth and the first peak bandwidth is greater than or equal to the first target bitrate adjustment range, thereby obtaining a second live video stream. The sending unit is configured to send the second live video stream to the second terminal.

13. A computer device, characterized in that, include: One or more processors; One or more memories for storing the one or more processor-executable instructions; The one or more processors are configured to execute the instructions to implement the video processing method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, When at least one instruction in the computer-readable storage medium is executed by one or more processors of a computer device, the computer device is enabled to perform the video processing method as described in any one of claims 1 to 11.

Images