Encoding processing method, transcoding server and storage medium

Abstract

This application provides an encoding processing method, a transcoding server, and a storage medium. In this embodiment, when the transcoding server encodes any video frame after decoding from the video data to be transcoded, it dynamically adjusts the encoding parameters according to the current resource utilization of the transcoding server. The dynamically adjusted encoding parameters are used to encode the video frame, dynamically changing the encoding complexity, reducing the load on the transcoding server, ensuring smooth playback of the video data, reducing the probability of stuttering, and guaranteeing video quality.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to an encoding processing method, a transcoding server, and a storage medium. Background Technology

[0002] With the development of internet technology, live streaming has become a trend, with more and more users initiating and watching live streams via mobile devices. Video data transmitted from the broadcaster to the viewer needs to undergo transcoding processing by a transcoding server. During transcoding, the server first decodes the video data, obtaining multiple consecutive video frames. Then, it re-encodes these frames using fixed encoding parameters to obtain the transcoded video data.

[0003] However, some video frames have complex content, and the transcoding server needs a long time to complete the encoding operation for these video frames, resulting in low encoding efficiency of the transcoding server and affecting the smoothness of video data playback. Summary of the Invention

[0004] This application provides an encoding processing method, a transcoding server, and a storage medium to improve the encoding efficiency of the transcoding server and ensure smooth playback of video data.

[0005] This application provides an encoding processing method applied to a transcoding server. The method includes: obtaining the current video frame to be encoded at the current moment, wherein the current video frame is one of a plurality of consecutive video frames obtained by decoding the video data to be transcoded; obtaining the current resource utilization rate of the target resource in the transcoding server; adjusting the current encoding parameters of the transcoding server according to the current resource utilization rate; and encoding the current video frame according to the adjusted encoding parameters.

[0006] This application also provides a transcoding server, including: a memory and a processor; the memory for storing a computer program; the processor coupled to the memory for executing the computer program to perform steps in an encoding processing method.

[0007] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the coded processing method.

[0008] In this embodiment, when the transcoding server encodes any video frame after decoding in the video data to be transcoded, it dynamically adjusts the encoding parameters according to the current resource utilization of the transcoding server. The dynamically adjusted encoding parameters are used to encode the video frame, dynamically changing the encoding complexity, reducing the load pressure on the transcoding server, ensuring the smoothness of video data playback, reducing the probability of stuttering, and ensuring video quality. Attached Figure Description

[0009] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0010] Figure 1 This is an example application scenario diagram;

[0011] Figure 2 A flowchart illustrating an encoding processing method provided in this application embodiment;

[0012] Figure 3 A graph showing the relationship between multiple usage thresholds;

[0013] Figure 4 A flowchart illustrating another encoding processing method provided in this application embodiment;

[0014] Figure 5 A flowchart illustrating another encoding processing method provided in this application embodiment;

[0015] Figure 6 This is a schematic diagram of the structure of an encoding processing device provided in an embodiment of this application;

[0016] Figure 7 This is a schematic diagram of the structure of a transcoding server provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the access relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates that the preceding and following associated objects have an "or" relationship. Furthermore, in the embodiments of this application, "first," "second," "third," etc., are only used to distinguish the content of different objects and have no other special meaning.

[0019] Figure 1 This diagram illustrates an exemplary application scenario. In live streaming, due to the diverse types of playback terminals and varying network quality, content service providers need to offer live streams with multiple bitrates, encoding formats, or resolutions for users to choose from to meet their viewing experience. Taking bitrate control by a transcoding server as an example, with the support of the transcoding server's transcoding function, a live stream with a bitrate of 3000kbps (kilobits per second) provided by the streaming end (which can be understood as the live streamer in a live streaming scenario) is converted into a live stream with a bitrate of 2000kbps and a live stream with a bitrate of 1000kbps. Thus, the transcoding server can provide the playback end (which can be understood as the viewer in a live streaming scenario) with multiple live streams: a 3000kbps stream, a 2000kbps stream, and a 1000kbps stream.

[0020] Currently, when transcoding servers perform transcoding, they first decode the live stream to obtain multiple consecutive video frames. Then, they re-encode these video frames using fixed encoding parameters to obtain the transcoded video data, i.e., the transcoded live stream. These encoding parameters include, but are not limited to: CBR (Constant Bitrate), ABR (Average Bitrate), VBR (Variable Bitrate), bitrate control buffer duration, reference frame count, quantization parameters, and resolution. When encoding video frames, the transcoding server can use CPU (Central Processing Unit) resources for software encoding, or it can use resources such as GPU (Graphics Processing Unit) for hardware encoding; there are no restrictions on which method is used. Regardless of whether software or hardware encoding is used, the resource utilization of the transcoding server, such as CPU or GPU utilization, directly affects the transcoding server's encoding efficiency.

[0021] In practical applications, some video frames in the same live stream are static, while others are dynamic; some frames contain more content, while others contain less, resulting in varying content complexity. With fixed encoding parameters, encoding lower-complexity video frames requires fewer transcoding server resources and takes less time. However, encoding higher-complexity frames requires more resources and takes longer, leading to lower overall transcoding efficiency, which affects the smoothness of video playback and can cause stuttering.

[0022] It should be noted that, Figure 1 The live streaming application scenario shown is merely an exemplary scenario, and this application embodiment does not limit the application scenario, such as video-on-demand scenarios or real-time audio and video scenarios. This application embodiment does not... Figure 1 The included equipment is not limited, nor is it restricted. Figure 1 The positional relationships between the devices are defined.

[0023] In this embodiment, the playback end and the streaming end can interact with the transcoding server via a wired network or a wireless network. For example, the wired network can include coaxial cable, twisted pair, and fiber optic cable, while the wireless network can be a 2G (2nd Generation), 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), or Wireless Fidelity (WIFI) network. This application does not limit the specific type or form of interaction, as long as it enables the playback end and the streaming end to interact with the transcoding server.

[0024] The playback or streaming end can be either hardware or software. When the playback or streaming end is hardware, it can be, for example, a mobile phone, tablet computer, desktop computer, wearable smart device, or smart home device. When the playback or streaming end is software, it can be installed on the aforementioned hardware devices. In this case, the playback or streaming end can be, for example, multiple software modules or a single software module; this embodiment of the application is not limited in this regard. The transcoding server can also be either hardware or software. When the transcoding server is hardware, it can be a single server or a distributed server cluster composed of multiple servers. When the transcoding server is software, it can be multiple software modules or a single software module; this embodiment of the application is not limited in this regard.

[0025] Based on the above, this application provides an encoding processing method, a transcoding server, and a storage medium. In this embodiment, when the transcoding server encodes any video frame after decoding in the video data to be transcoded, it dynamically adjusts the encoding parameters according to the current resource utilization of the transcoding server. The dynamically adjusted encoding parameters are used to encode the video frame, dynamically changing the encoding complexity, reducing the load on the transcoding server, ensuring smooth playback of the video data, reducing the probability of stuttering, and guaranteeing video quality.

[0026] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0027] Figure 2 This is a flowchart illustrating an encoding processing method provided in an embodiment of this application. The method can be executed by an encoding processing device, which may consist of software and / or hardware, and is generally configured in a transcoding server. See also... Figure 2 The method may include the following steps:

[0028] 201. Obtain the current video frame to be encoded at the current moment. The current video frame is one of multiple consecutive video frames obtained by decoding the video data to be transcoded.

[0029] 202. Obtain the current resource utilization rate of the target resource in the transcoding server.

[0030] 203. Adjust the current encoding parameters of the transcoding server according to the current resource utilization rate.

[0031] 204. Encode the current video frame according to the adjusted encoding parameters.

[0032] In this embodiment, for the video data to be transcoded, firstly, the transcoding server decodes the video data to be transcoded, obtaining multiple consecutive video frames. Then, for any current video frame that needs to be encoded among the multiple consecutive video frames, suitable encoding parameters are determined for the current video frame. Specifically, for any video frame to be encoded, the current resource utilization rate of the target resource in the transcoding server is obtained, and the current encoding parameters of the transcoding server are adjusted according to the current resource utilization rate to obtain adjusted encoding parameters suitable for the current video frame. In practical applications, when it is determined that encoding processing of any video frame to be encoded will be initiated, the current resource utilization rate of the target resource at the current time is obtained. The current resource utilization rate can also be the average resource utilization rate, maximum resource utilization rate, or minimum resource utilization rate over a short period of time, including the current time; there is no limitation on this.

[0033] It is worth noting that if the transcoding server uses software encoding to encode video frames, the target resource can be the transcoding server's CPU; if the transcoding server uses hardware encoding to encode video frames, the target resource can be the transcoding server's GPU.

[0034] In some optional embodiments, when adjusting the current encoding parameters of the transcoding server based on the current resource utilization rate, the current resource utilization rate can be compared with a preset utilization rate threshold. If the current resource utilization rate is greater than or equal to the preset utilization rate threshold, the current encoding parameters can be adjusted in the direction of reducing encoding complexity. If the current resource utilization rate is less than the preset utilization rate threshold, the current encoding parameters can be adjusted in the direction of increasing encoding complexity or remain unchanged.

[0035] It's worth noting that if the current resource utilization rate is greater than or equal to the preset utilization rate threshold, it indicates that the transcoding server has limited remaining available resources. Adjusting the current encoding parameters to reduce encoding complexity results in less resource consumption compared to the original parameters, thus improving encoding efficiency.

[0036] If the current resource utilization rate is less than the preset utilization rate threshold, it indicates that the transcoding server has sufficient remaining resources. In this case, the current encoding parameters can be adjusted to increase encoding complexity or the current resource utilization rate can be kept unchanged. When adjusting the current encoding parameters to increase encoding complexity, encoding with the adjusted parameters requires more resources compared to the original parameters. However, with sufficient remaining resources on the transcoding server, encoding efficiency can still be maintained at a high level.

[0037] Taking encoding parameters including CBR, AB, VBR, bitrate control buffer duration, reference frame count, quantization parameters, and resolution as an example, when adjusting encoding parameters to reduce encoding complexity, the values ​​of parameters such as CBR, AB, VBR, bitrate control buffer duration, reference frame count, quantization parameters, and resolution are decreased, and / or the total number of parameters is reduced (i.e., some encoding parameters are reduced); when adjusting encoding parameters to increase encoding complexity, the values ​​of parameters such as CBR, AB, VBR, bitrate control buffer duration, reference frame count, quantization parameters, and resolution are increased, and / or the total number of parameters is increased (i.e., some additional encoding parameters are added), and so on. Of course, how to adjust encoding parameters in the direction of reducing or increasing encoding complexity can be flexibly defined as needed, and there are no restrictions on this.

[0038] In some alternative embodiments, when adjusting the current encoding parameters of the transcoding server based on the current resource utilization rate, the current resource utilization rate can be compared with a preset utilization rate range. If the current resource utilization rate falls within the preset utilization rate range, it indicates that the remaining available resources of the transcoding server are relatively small, and the current encoding parameters can be adjusted in the direction of reducing encoding complexity. If the current resource utilization rate does not fall within the preset utilization rate range, it indicates that the remaining available resources of the transcoding server are relatively sufficient, and the current encoding parameters can be adjusted in the direction of increasing encoding complexity or remain unchanged.

[0039] Optionally, in order to adjust the encoding parameters more reasonably and improve encoding efficiency, when adjusting the current encoding parameters of the transcoding server based on the current resource utilization rate, the current encoding parameters of the transcoding server can be adjusted based on the comparison results of the current resource utilization rate and multiple utilization rate thresholds; wherein, the multiple utilization rate thresholds are ordered in ascending order as the first utilization rate threshold, the second utilization rate threshold, and the third utilization rate threshold.

[0040] In this embodiment, the first utilization rate threshold, the second utilization rate threshold, and the third utilization rate threshold are flexibly set as needed. See also Figure 3 Assuming the first utilization threshold is denoted as A, the second as B, and the third as C, these thresholds divide resource utilization into four numerical ranges: the first range is [0, A), the second as [A, B), the third as [B, C], and the fourth as [C, 100%]. As resource utilization increases, the resource load on the transcoding server also increases. Different encoding parameters are used for different numerical ranges, thus utilizing the first, second, and third utilization thresholds to implement a multi-level adjustment mechanism for the encoding parameters.

[0041] In this embodiment, the encoding parameters supported by the transcoding server are arranged in descending order of encoding complexity: a first encoding parameter, a second encoding parameter, and a third encoding parameter. Higher encoding complexity results in better image quality for the encoded video frames. The first, second, and third encoding parameters can be flexibly set as needed and are not restricted. For example, compared to the first encoding parameter, the total number of parameters or the parameter values ​​in the second encoding parameter may be reduced; similarly, the total number of parameters or the parameter values ​​in the third encoding parameter may be reduced. In other words, with the first encoding parameter as a reference, the second encoding parameter is a first-level downgraded encoding parameter that reduces encoding complexity relative to the first encoding parameter; the third encoding parameter is a second-level downgraded encoding parameter that reduces encoding complexity relative to the first encoding parameter.

[0042] In practical applications, when adjusting the current encoding parameters of the transcoding server based on a comparison of the current resource utilization rate and multiple utilization thresholds, the following situations can be considered:

[0043] Case 1: If the current resource utilization rate is less than the first utilization rate threshold, the current encoding parameters of the transcoding server are adjusted according to the historical resource utilization rate. The historical resource utilization rate refers to the resource utilization rate of the transcoding server when encoding the previous video frame of the current video frame.

[0044] Specifically, if the current resource utilization rate is less than the first utilization rate threshold, it indicates that the transcoding server's current resource load is relatively low, and the remaining available resources are sufficient. The decision to adjust the current encoding parameters to the highest-complexity first encoding parameter can be made by considering the historical resource utilization rate of the previous video frame. The historical resource utilization rate can be the maximum, minimum, or average resource utilization rate during the period when the transcoding server encoded the previous video frame; there are no restrictions on this.

[0045] As an example, in order to better adjust the encoding parameters, if the current resource utilization rate is less than the first utilization rate threshold, then if the historical resource utilization rate is greater than or equal to the first utilization rate threshold, the current encoding parameter is adjusted to the first encoding parameter; if the historical resource utilization rate is less than the first utilization rate threshold, the current encoding parameter is determined to be the adjusted encoding parameter.

[0046] It is worth noting that if the historical resource utilization rate of the previous video frame is greater than or equal to the first utilization rate threshold, it indicates that the transcoding server is more likely to use the second or third encoding parameters to encode the previous video frame. In this case, the current encoding parameters can be adjusted to the first encoding parameters.

[0047] If the historical resource utilization rate is less than the first utilization rate threshold, it means that the encoding parameters used by the transcoding server to encode the previous video frame of the current video frame can keep the resource utilization rate of the transcoding server at a low level. Therefore, it can be determined that the current encoding parameters are the adjusted encoding parameters, that is, keep the current encoding parameters unchanged and do not adjust the current encoding parameters, so as to better balance the encoding complexity and the picture quality.

[0048] Case 2: If the current resource utilization rate is greater than or equal to the first utilization rate threshold and less than the second utilization rate threshold, then the current encoding parameter is determined to be the adjusted encoding parameter.

[0049] Specifically, if the current resource utilization rate is greater than or equal to the first utilization rate threshold and less than the second utilization rate threshold, it means that the remaining available resources of the transcoding server are relatively sufficient, and the current encoding parameters can be maintained unchanged to better balance encoding complexity and image quality.

[0050] Case 3: If the current resource utilization rate is greater than the third utilization rate threshold, then adjust the current encoding parameter to the third encoding parameter.

[0051] Specifically, if the current resource utilization rate is greater than the third utilization rate threshold, it indicates that the transcoding server is currently under heavy resource load. The current encoding parameters are then adjusted to the third encoding parameter with the lowest encoding complexity to reduce the current resource load on the transcoding server.

[0052] Case 4: If the current resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, then the current encoding parameters of the transcoding server are adjusted according to the historical resource utilization rate.

[0053] Specifically, if the current resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, it indicates that the current resource load pressure of the transcoding server is relatively high and the remaining available resources of the transcoding server are relatively tight. The decision on whether to adjust the current encoding parameter to the second encoding parameter with the second highest encoding complexity can be made by considering the historical resource utilization rate corresponding to the previous video frame of the current video frame.

[0054] As an example, to better adjust the encoding parameters, if the current resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, then if the historical resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, the current encoding parameter will be adjusted to the second encoding parameter; if the historical resource utilization rate is greater than the third utilization rate threshold or less than the second utilization rate threshold, then the current encoding parameter will be determined to be the adjusted encoding parameter.

[0055] Further, optionally, to better adjust the encoding parameters, one implementation of adjusting the current encoding parameters to the second encoding parameters when the historical resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold is as follows: when the historical resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, obtain the encoding complexity of the current video frame; determine whether the encoding complexity of the current video frame is greater than the encoding complexity corresponding to the second encoding parameter; if the determination result is no, then adjust the current encoding parameters to the second encoding parameter; if the determination result is yes, then adjust the current encoding parameters to the third encoding parameter.

[0056] Specifically, when both the current resource utilization rate and the historical resource utilization rate are greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, if the encoding complexity of the current video frame is less than or equal to the encoding complexity corresponding to the second encoding parameter, then adjusting the current encoding parameter to the second encoding parameter can reliably reduce the resource utilization rate of the transcoding server and improve encoding efficiency. If the encoding complexity of the current video frame is greater than the encoding complexity corresponding to the second encoding parameter, then adjusting the current encoding parameter to the third encoding parameter can reliably reduce the resource utilization rate of the transcoding server and improve encoding efficiency.

[0057] Alternatively, in order to accurately determine the encoding complexity of the current video frame, the content complexity of the current video frame can be obtained when acquiring the encoding complexity; based on the content complexity of the current video frame, the correspondence between the content complexity and the encoding complexity can be queried to determine the encoding complexity of the current video frame.

[0058] Specifically, the content complexity of the current video frame is directly proportional to its encoding complexity. The content complexity of the current video frame can be quantitatively assessed through its static / dynamic attributes and / or content richness, but is not limited to these. Static / dynamic attributes indicate whether the current video frame is static or dynamic, while content richness indicates the amount of content included in the video frame.

[0059] Further, optionally, in order to accurately determine the content complexity of the current video frame, a machine learning model can be used to detect the content complexity of the current video frame to obtain the content complexity of the current video frame; wherein, the machine learning model is trained using multiple sample video frames and their labeled content complexity.

[0060] In practical applications, when training machine learning models, a massive amount of sample video frames can be prepared. These sample video frames include both static and dynamic frames, and the content contained in multiple sample video frames is different, thereby improving the recognition accuracy and precision of the machine learning model. Furthermore, the content complexity of sample video frames can be labeled by combining the static / dynamic attributes and / or content richness.

[0061] Optionally, to improve the recognition accuracy and precision of the machine learning model, the model may include a feature extraction network and a classification network connected sequentially. The feature extraction network is a neural network structure with feature extraction capabilities, such as, but not limited to, DarkNet (anonymous network), ResNet (residual network), and AlexNet. AlexNet, for example, has eight neural network layers: the first five are convolutional layers, and the last three are fully connected layers. The classification network is a neural network structure with classification capabilities, such as, but not limited to, a multilayer perceptron (MLP).

[0062] In this embodiment, model training can be performed based on whether the classification loss function satisfies the convergence condition of the machine learning model. The classification loss function is a loss function that measures the degree of inconsistency between the predicted classification result output by the machine learning model (i.e., the predicted content complexity) and the true classification result of the sample video frame annotations (i.e., the annotation content complexity). Classification loss functions include, but are not limited to: log-log loss function, L1 distance loss function, cross-entropy loss function, and Focal loss function used to address data imbalance problems.

[0063] Based on the above, one possible training method for the machine learning model is as follows: acquire multiple sample video frames; input each sample video frame into the feature extraction network of the machine learning model to be trained for feature extraction, and obtain the feature map corresponding to each sample video frame; use the classification network in the machine learning model to classify the feature map corresponding to each sample video frame, and obtain the predicted content complexity corresponding to each sample video frame; calculate the classification loss function of each of the multiple sample video frames based on their respective predicted content complexity and their labeled content complexity; and adjust the network parameters of the machine learning model based on the respective classification loss functions of the multiple sample video frames.

[0064] It's worth noting that when adjusting the network parameters of a machine learning model based on the classification loss functions of multiple sample video frames, one approach is to sum the classification loss functions of each frame to obtain a total classification loss function, which can then be used to adjust the network parameters. Alternatively, the classification loss functions of each frame can be summed in a weighted manner to obtain a weighted classification loss function, which can then be used to adjust the network parameters. Another approach is to average the classification loss functions of each frame to obtain an average classification loss function, which can then be used to adjust the network parameters. However, this method is not limited to these approaches.

[0065] The goal is to minimize various classification loss functions, such as the sum of classification loss functions, weighted classification loss functions, or average classification loss functions. Model training is iteratively performed, continuously adjusting the network parameters of the machine learning model until the model meets the convergence criteria. The convergence criteria are set according to the actual application requirements. For example, a convergence criterion might be that the current model has reached its maximum training iterations, or that the classification loss function has reached its minimum value, or that the classification loss function has reached its minimum value even after reaching the maximum training iterations. However, these criteria are not limited to these.

[0066] In this embodiment, after adjusting the current encoding parameters of the transcoding server according to the current resource utilization, the current video frame is encoded according to the adjusted encoding parameters. The encoded video frame can be transmitted to the playback end for playback. In practical applications, the encoded video frames can be transmitted to the playback end in real time, or a certain number of encoded video frames can be transmitted to the playback end after a certain number of frames have been encoded; there is no limitation on this.

[0067] The technical solution provided in this application embodiment allows the transcoding server to dynamically adjust the encoding parameters based on the current resource utilization rate of the transcoding server when encoding any video frame after decoding in the video data to be transcoded. The dynamically adjusted encoding parameters are used to encode the video frame, dynamically changing the encoding complexity, reducing the load pressure on the transcoding server, ensuring the smoothness of video data playback, reducing the probability of stuttering, and ensuring video quality.

[0068] To better understand the technical solutions provided in the embodiments of this application, several scenario embodiments are described below:

[0069] Scenario Example 1:

[0070] See Figure 4 In live streaming scenarios, the process of software encoding using CPU resources by the transcoding server includes the following steps:

[0071] 401. For the current video frame to be encoded, obtain the current CPU utilization rate of the CPU in the transcoding server.

[0072] 402. Compare the current CPU utilization rate with multiple utilization rate thresholds, and execute steps 403, 406, 407 and 410.

[0073] 403. If the current CPU utilization is less than the first utilization threshold, determine whether the historical CPU utilization of the transcoding server when encoding the previous video frame of the current video frame was greater than or equal to the first utilization threshold. If the determination result is yes, proceed to step 404; if the determination result is no, proceed to step 405.

[0074] 404. Adjust the current encoding parameter to the first encoding parameter and proceed to step 411.

[0075] 405. Determine that the current encoding parameters are the adjusted encoding parameters, and proceed to step 411.

[0076] 406. If the current CPU utilization rate is greater than or equal to the first utilization rate threshold and less than the second utilization rate threshold, then determine that the current encoding parameter is the adjusted encoding parameter and proceed to step 411.

[0077] 407. If the current CPU utilization is greater than or equal to the second utilization threshold and less than the third utilization threshold, then determine whether the historical CPU utilization is greater than or equal to the second utilization threshold and less than the third utilization threshold. If the determination result is yes, proceed to step 408; if the determination result is no, proceed to step 409.

[0078] 408. Adjust the current encoding parameters to the second encoding parameters and proceed to step 411.

[0079] 409. If the historical CPU utilization rate is greater than the third utilization rate threshold or less than the second utilization rate threshold, then the current encoding parameter is determined to be the adjusted encoding parameter, and step 411 is executed.

[0080] 410. If the current CPU utilization rate is greater than the third utilization rate threshold, adjust the current encoding parameter to the third encoding parameter and execute step 411.

[0081] 411. Encode the current video frame according to the adjusted encoding parameters.

[0082] The implementation details of each step in the transcoding server's use of CPU resources for software encoding can be found in the aforementioned embodiments and will not be repeated here.

[0083] The technical solution provided in this application embodiment is that when the transcoding server encodes any video frame after decoding in the video data to be transcoded, it dynamically adjusts the encoding parameters according to the current CPU utilization of the transcoding server. The dynamically adjusted encoding parameters are used to encode the video frame, dynamically changing the encoding complexity, reducing the CPU load pressure of the transcoding server, ensuring the smoothness of video data playback, reducing the probability of stuttering, and ensuring video quality.

[0084] Scenario Example 2:

[0085] See Figure 5 In video-on-demand scenarios, the process of a transcoding server using GPU resources for hard encoding includes the following steps:

[0086] 501. For the current video frame to be encoded, obtain the current GPU utilization rate of the GPU in the transcoding server.

[0087] 502. Compare the current GPU utilization rate with multiple utilization rate thresholds, and execute steps 503, 506, 507 and 510.

[0088] 503. If the current GPU utilization is less than the first utilization threshold, determine whether the historical GPU utilization when the transcoding server encoded the previous video frame of the current video frame was greater than or equal to the first utilization threshold. If the determination result is yes, proceed to step 504; if the determination result is no, proceed to step 505.

[0089] 504. Adjust the current encoding parameters to the first encoding parameters and proceed to step 511.

[0090] 505. Determine that the current encoding parameters are the adjusted encoding parameters, and proceed to step 511.

[0091] 506. If the current GPU utilization rate is greater than or equal to the first utilization rate threshold and less than the second utilization rate threshold, then determine that the current encoding parameters are the adjusted encoding parameters and proceed to step 511.

[0092] 507. If the current GPU utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, then determine whether the historical GPU utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold. If the determination result is yes, proceed to step 508; if the determination result is no, proceed to step 509.

[0093] 508. Adjust the current encoding parameters to the second encoding parameters and proceed to step 511.

[0094] 509. If the historical GPU utilization rate is greater than the third utilization rate threshold or less than the second utilization rate threshold, then the current encoding parameters are determined to be the adjusted encoding parameters, and step 511 is executed.

[0095] 510. If the current GPU utilization rate is greater than the third utilization rate threshold, adjust the current encoding parameters to the third encoding parameters and execute step 511.

[0096] 511. Encode the current video frame according to the adjusted encoding parameters.

[0097] The implementation details of each step in the transcoding server's use of GPU resources for hard encoding can be found in the aforementioned embodiments and will not be repeated here.

[0098] The technical solution provided in this application embodiment is that when the transcoding server encodes any video frame after decoding in the video data to be transcoded, it dynamically adjusts the encoding parameters according to the current GPU utilization of the transcoding server. The dynamically adjusted encoding parameters are used to encode the video frame, dynamically changing the encoding complexity, reducing the GPU load pressure of the transcoding server, ensuring the smoothness of video data playback, reducing the probability of stuttering, and ensuring video quality.

[0099] Figure 6 This is a schematic diagram of an encoding processing apparatus provided in an embodiment of this application. This apparatus can be configured into a transcoding server; see [link / reference]. Figure 6 The device may include:

[0100] The first acquisition module 61 is used to acquire the current video frame to be encoded, which is one of multiple consecutive video frames obtained by decoding the video data to be transcoded.

[0101] The second acquisition module 62 is used to acquire the current resource utilization rate of the target resource in the transcoding server;

[0102] Adjustment module 63 is used to adjust the current encoding parameters of the transcoding server according to the current resource utilization rate;

[0103] Encoding module 64 is used to encode the current video frame according to the adjusted encoding parameters.

[0104] Optionally, when adjusting the current encoding parameters of the transcoding server based on the current resource utilization rate, the adjustment module 63 is specifically used to: adjust the current encoding parameters of the transcoding server based on the comparison results of the current resource utilization rate and multiple utilization rate thresholds; wherein, the multiple utilization rate thresholds are ordered in ascending order as the first utilization rate threshold, the second utilization rate threshold, and the third utilization rate threshold.

[0105] Optionally, when adjusting the current encoding parameters of the transcoding server based on the comparison results of the current resource utilization rate and multiple utilization rate thresholds, the adjustment module 63 specifically performs the following: if the current resource utilization rate is less than the first utilization rate threshold, the current encoding parameters of the transcoding server are adjusted according to the historical resource utilization rate, where the historical resource utilization rate refers to the resource utilization rate when the transcoding server encodes the previous video frame of the current video frame; if the current resource utilization rate is greater than or equal to the first utilization rate threshold and less than the second utilization rate threshold, the current encoding parameters are determined to be the adjusted encoding parameters; if the current resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, the current encoding parameters of the transcoding server are adjusted according to the historical resource utilization rate; if the current resource utilization rate is greater than the third utilization rate threshold, the current encoding parameters are adjusted to the third encoding parameters.

[0106] Optionally, if the current resource utilization rate is less than the first utilization rate threshold, when adjusting the current encoding parameters of the transcoding server based on the historical resource utilization rate, the adjustment module 63 is specifically used as follows: if the current resource utilization rate is less than the first utilization rate threshold, and if the historical resource utilization rate is greater than or equal to the first utilization rate threshold, the current encoding parameters are adjusted to the first encoding parameters, wherein the encoding complexity corresponding to the first encoding parameters is greater than the encoding complexity corresponding to the third encoding parameters; if the historical resource utilization rate is less than the first utilization rate threshold, the current encoding parameters are determined to be the adjusted encoding parameters.

[0107] Optionally, when adjusting the current encoding parameters of the transcoding server based on historical resource utilization, if the current resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, the adjustment module 63 is specifically used as follows: if the current resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, then if the historical resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, the current encoding parameter is adjusted to the second encoding parameter; if the historical resource utilization rate is greater than the third utilization rate threshold or less than the second utilization rate threshold, then the current encoding parameter is determined to be the adjusted encoding parameter; wherein, the encoding complexity corresponding to the second encoding parameter is greater than the encoding complexity corresponding to the third encoding parameter, and the encoding complexity corresponding to the second encoding parameter is less than the first encoding parameter.

[0108] Optionally, when the adjustment module 63 adjusts the current encoding parameter to the second encoding parameter when the historical resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, it is specifically used to: obtain the encoding complexity of the current video frame when the historical resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold; determine whether the encoding complexity of the current video frame is greater than the encoding complexity corresponding to the second encoding parameter; if the determination result is no, then adjust the current encoding parameter to the second encoding parameter.

[0109] Further optionally, the adjustment module 63 is also used to: if the judgment result is yes, adjust the current encoding parameter to the third encoding parameter.

[0110] Optionally, when adjusting module 63 obtains the encoding complexity of the current video frame, it is specifically used to: obtain the content complexity of the current video frame; and based on the content complexity of the current video frame, query the correspondence between content complexity and encoding complexity to determine the encoding complexity of the current video frame.

[0111] Further optionally, when adjusting module 63 obtains the content complexity of the current video frame, it is specifically used to: use a machine learning model to detect the content complexity of the current video frame and obtain the content complexity of the current video frame; wherein, the machine learning model is trained using multiple sample video frames and their labeled content complexity.

[0112] Further optionally, the above apparatus further includes: a training module, configured to: acquire multiple sample video frames and their labeled content complexity; input each sample video frame into the feature extraction network of the machine learning model to be trained for feature extraction, obtaining a feature map corresponding to each sample video frame; use the classification network in the machine learning model to classify the feature map corresponding to each sample video frame, obtaining the predicted content complexity corresponding to each sample video frame; calculate the classification loss function of each of the multiple sample video frames based on their respective predicted content complexity and labeled content complexity; and adjust the network parameters of the machine learning model based on the respective classification loss functions of the multiple sample video frames.

[0113] Figure 6 The device shown can perform Figure 2 The implementation principle and technical effects of the methods in the illustrated embodiments will not be elaborated further. Regarding the methods in the above embodiments... Figure 6 The specific ways in which each module and unit of the device performs operations have been described in detail in the embodiments of the method, and will not be elaborated here.

[0114] For a detailed description of the implementation process of each action by the processor, please refer to the relevant descriptions in the foregoing method embodiments or device embodiments, which will not be repeated here.

[0115] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 201 to 204 can be device A; or the execution subject of steps 201 and 202 can be device A, and the execution subject of steps 203 and 204 can be device B; and so on.

[0116] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 201, 202, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0117] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0118] Figure 7 This is a schematic diagram of the structure of a transcoding server provided in an embodiment of this application. Figure 7 As shown, the transcoding server includes: a memory 71 and a processor 72;

[0119] Memory 71 is used to store computer programs and can be configured to store various other data to support operation on the computing platform. Examples of this data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc.

[0120] The memory 71 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0121] The processor 72, coupled to the memory 71, is used to execute the computer program in the memory 71 for: performing steps in the encoding processing method.

[0122] Furthermore, such as Figure 7 As shown, the transcoding server also includes other components such as a communication component 73, a display 74, a power supply component 75, and an audio component 76. Figure 7 The diagram only shows some components and does not mean that the transcoding server only includes... Figure 7 The components shown. Additionally... Figure 7 The components within the dashed box are optional, not mandatory, and their specific requirements depend on the product type of the transcoding server.

[0123] For a detailed description of the processor's execution of each action, please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.

[0124] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can perform the steps that can be executed by the transcoding server in the above method embodiments.

[0125] Accordingly, this application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, enables the processor to perform the steps that can be executed by a transcoding server in the above method embodiments.

[0126] The aforementioned communication components are configured to facilitate wired or wireless communication between the device containing the communication components and other devices. The device containing the communication components can access wireless networks based on communication standards, such as WiFi (Wireless Fidelity), 2G (2nd Generation), 3G (3rd Generation), 4G (4th Generation) / LTE (long Term Evolution), 5G (5th Generation), or combinations thereof. In one exemplary embodiment, the communication components receive broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication components also include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth, and other technologies.

[0127] The aforementioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0128] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0129] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0130] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interfaces, network interfaces, and memory.

[0135] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0136] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. As defined in this article, computer-readable media do not include transient media, such as modulated data signals and carrier waves.

[0137] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0138] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An encoding processing method, characterized in that, Applied to a transcoding server, the method includes: Obtain the current video frame to be encoded, wherein the current video frame is one of multiple consecutive video frames obtained by decoding the video data to be transcoded; Obtain the current resource utilization rate of the target resource in the transcoding server; If the current resource utilization rate is less than the first utilization rate threshold, then if the historical resource utilization rate is greater than or equal to the first utilization rate threshold, the current encoding parameters of the transcoding server are adjusted to the first encoding parameters; if the historical resource utilization rate is less than the first utilization rate threshold, the current encoding parameters are determined to be the adjusted encoding parameters; the historical resource utilization rate refers to the resource utilization rate of the transcoding server when encoding the previous video frame of the current video frame; If the current resource utilization rate is greater than or equal to the first utilization rate threshold and less than the second utilization rate threshold, then the current encoding parameter is determined to be the adjusted encoding parameter; If the current resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, then if the historical resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, the current encoding parameter is adjusted to the second encoding parameter; if the historical resource utilization rate is greater than the third utilization rate threshold or less than the second utilization rate threshold, then the current encoding parameter is determined to be the adjusted encoding parameter; wherein, the encoding complexity corresponding to the second encoding parameter is greater than the encoding complexity corresponding to the third encoding parameter, and the encoding complexity corresponding to the second encoding parameter is less than the encoding complexity of the first encoding parameter; If the current resource utilization rate is greater than the third utilization rate threshold, then the current encoding parameter is adjusted to the third encoding parameter; wherein, in ascending order, the first utilization rate threshold, the second utilization rate threshold, and the third utilization rate threshold are respectively; Adjust the current encoding parameters of the transcoding server based on the current resource utilization rate; The current video frame is encoded according to the adjusted encoding parameters.

2. The method according to claim 1, characterized in that, When the historical resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, adjusting the current encoding parameter to the second encoding parameter includes: If the historical resource utilization rate is greater than or equal to the second utilization rate threshold and less than the third utilization rate threshold, the encoding complexity of the current video frame is obtained. Determine whether the encoding complexity of the current video frame is greater than the encoding complexity corresponding to the second encoding parameter; If the judgment result is negative, the current encoding parameter is adjusted to the second encoding parameter.

3. The method according to claim 2, characterized in that, Also includes: If the judgment result is yes, then the current encoding parameter is adjusted to the third encoding parameter.

4. The method according to claim 2, characterized in that, The encoding complexity of obtaining the current video frame includes: Obtain the content complexity of the current video frame; Based on the content complexity of the current video frame, the correspondence between content complexity and encoding complexity is queried to determine the encoding complexity of the current video frame.

5. The method according to claim 4, characterized in that, The complexity of obtaining the content of the current video frame includes: The content complexity of the current video frame is obtained by using a machine learning model to detect the content complexity of the current video frame. The machine learning model is trained using multiple sample video frames and the complexity of their labeled content.

6. The method according to claim 5, characterized in that, The training method for the machine learning model is as follows: The complexity of acquiring multiple sample video frames and their annotations; Each sample video frame is input into the feature extraction network in the machine learning model to be trained to extract features, thereby obtaining the feature map corresponding to each sample video frame. The classification network in the machine learning model is used to classify the feature map corresponding to each sample video frame to obtain the predicted content complexity for each sample video frame. Based on the predicted content complexity and the labeled content complexity of each of the multiple sample video frames, calculate the classification loss function for each of the multiple sample video frames. The network parameters of the machine learning model are adjusted based on the classification loss function of each of the multiple sample video frames.

7. A transcoding server, characterized in that, include: Memory and processor; The memory is used to store computer programs; The processor is coupled to the memory for executing the computer program to perform the steps of the method according to any one of claims 1-6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 1-6.

Images