DVFS method and device of hardware video decoder

Abstract

The invention discloses a DVFS method of a hardware video decoder. The DVFS method comprises the following steps. And S31, counting the number of to-be-decoded code stream caches and the number of to-be-filled video frame caches in the video decoding system, calling a'to-be-decoded code stream cache 'and a'to-be-filled video frame cache' as a'to-be-decoded cache pair ', and representing the workload of a hardware video decoder at a moment by using the number of the'to-be-decoded cache pair' at the moment. And S32, according to the real-time working load of the hardware video decoder, determining the relative relationship between the decoding performance and the decoding demand of the hardware video decoder at the moment, and calculating the new working frequency of the hardware video decoder according to the relative relationship. And step S33, adjusting the working frequency of the hardware video decoder to a new working frequency. The method has the advantages of being small in calculation amount, good in adaptability and high in real-time performance.

Description

Technical Field

[0001] The present application relates to a power consumption control method, and in particular to a DVFS method suitable for a hardware video decoder. Background Art

[0002] DVFS (dynamic voltage and frequency scaling) is an energy management technology widely used to reduce energy consumption in processors and computing devices. DVFS adjusts voltage and frequency in real time based on the processor workload. It reduces voltage and frequency to save energy when the processor workload is low, and increases voltage and frequency to provide higher performance when the processor workload is high. By dynamically adjusting processor voltage and frequency to adapt to varying workloads, DVFS achieves energy savings and thermal management.

[0003] Video coding is a technology that compresses redundant components in video images and uses the least amount of data possible to represent video information. Common video coding standards include HEVC (high-efficiency video coding, also known as H.265) and AVC (advanced video coding, also known as H.264).

[0004] Video decoding is the process of decoding and restoring an encoded digital video. This process is typically performed by a video decoder. Specifically, video decoding is the inverse of video encoding. This involves parsing, dequantizing, and inversely transforming the encoded video data (called the decoded bitstream) according to the encoding rules, ultimately restoring the original video signal.

[0005] Because video decoding algorithms require a lot of computation, it's common practice in the industry to use an application-specific integrated circuit (ASIC) to accelerate the video decoding process in order to increase decoding speed and achieve real-time decoding and display. This ASIC is often referred to as a hardware video decoder, or VPU (video processing unit). "VPU" will be used to refer to "hardware video decoder" in this article.

[0006] The VPU can operate at different clock frequencies and voltages. Generally speaking, the higher the VPU's operating frequency, the stronger its decoding performance and the higher the required voltage. The lower the VPU's operating frequency, the weaker its decoding performance and the lower the required voltage.

[0007] See also Figure 1A typical video decoding system includes a stream extraction module 11, a video decoding module 12, and a video display module 13, which are connected in sequence. The core component of the stream extraction module 11 is typically a stream parser, responsible for parsing multimedia files and extracting the stream to be decoded. The core component of the video decoding module 12 is the VPU, responsible for decoding the stream to be decoded and generating decoded video frames. The core component of the video display module 13 is typically a display, responsible for displaying the decoded video frames.

[0008] In video decoding systems, the VPU must not only meet real-time decoding requirements but also minimize power consumption. If the VPU performance is too low to meet the system's requirements, it will not only increase video decoding latency but, in severe cases, cause frame loss, significantly reducing display quality. If the VPU performance is too high, exceeding the system's requirements, it will increase power consumption and cost.

[0009] Therefore, it is necessary to design a DVFS method suitable for the VPU based on the characteristics of the video decoding system. When the VPU workload is low, its operating frequency is reduced to save energy. When the VPU workload is high, its operating frequency is increased to provide better performance.

[0010] Currently, there are some DVFS methods suitable for VPUs, the most typical of which are the DVFS method based on video decoding configuration parameters and the DVFS method based on the VPU duty cycle.

[0011] The DVFS method based on video decoding configuration parameters pre-measures the impact of different video decoding configuration parameters on VPU performance. Before the VPU operates, the VPU's operating frequency is selected based on the video decoding parameter information extracted from the multimedia file by the bitstream extraction module to meet the performance requirements of the video decoding system. These video decoding parameters include, for example, the number of video decoding channels, the video decoding standard used by each decoding channel, video resolution, and decoding frame rate. The essence of this DVFS method is to use video decoding configuration parameters to characterize the VPU workload. Different video decoding configuration parameters represent different VPU workloads and, therefore, correspond to different VPU operating frequencies. This DVFS method has the following disadvantages. First, the mapping relationship between different video decoding configuration parameters and VPU performance must be measured in advance. The more video decoding configuration parameters there are, the greater the measurement workload. Furthermore, the mapping relationship will not be identical for different VPUs, requiring re-measurement, making adaptive use impossible. Second, it is not effective on complex embedded systems (such as smartphones). This is because the VPU's performance is affected not only by its own frequency but also by available resources such as the CPU and memory. In complex embedded systems, due to competition from other hardware devices, the CPU, memory and other resources available to the VPU are often unstable, resulting in a large difference between the actual performance of the VPU and the measured performance, making it difficult to match the requirements of the video decoding system.

[0012] The VPU duty cycle-based DVFS method periodically monitors the VPU's duty cycle—the ratio of active operating time to total operating time—while the VPU is operating, and adjusts the VPU frequency based on the duty cycle. Essentially, this DVFS method uses the VPU's duty cycle to characterize the VPU's workload. A high VPU duty cycle indicates a heavy workload, requiring an increase in the VPU frequency to provide better video decoding performance. A low VPU duty cycle indicates a light workload, requiring a decrease in the VPU frequency to conserve system power. Compared to DVFS methods based on video decoding configuration parameters, this DVFS method hides system complexity by normalizing the relative relationship between VPU workload and VPU decoding performance to the VPU's duty cycle. It does not consider the impact of video decoding configuration parameters and available system resources such as CPU and memory on the VPU's actual decoding performance, enabling adaptive adjustment of the VPU frequency. This DVFS method has the following disadvantages. First, VPU duty cycle information must rely on statistics collected from the past period of VPU operation, resulting in an unavoidable lag. Using lagged duty cycle information to predict the current VPU workload inevitably introduces certain errors, especially when the system's instantaneous state undergoes significant changes. Predicting the current VPU workload based on duty cycle information can be highly inaccurate. Second, this DVFS method cannot effectively handle situations where the stream extraction module actively drops frames. Because this DVFS method requires statistics on VPU performance over a period of time, if the system state fluctuates dramatically during this statistical period, resulting in a significant drop in video decoding system performance, the stream extraction module may resort to active frame drops (typically discarding non-reference frames) to prevent significant frame drops. Consequently, during this statistical period, the VPU's duty cycle may actually drop significantly, causing the DVFS method to mistakenly believe that the VPU workload is low and further reduce the VPU frequency. This drop in VPU frequency further degrades video decoding performance, causing the stream extraction module to actively drop frames even more aggressively, creating an unrecoverable vicious cycle.

[0013] As can be seen, due to the inherent complexity of video decoding systems, the actual decoding performance of the VPU is not only related to its own operating frequency but also affected by available system resources such as the CPU and memory. This makes DVFS methods based on video decoding configuration parameters poorly applicable in complex embedded systems. While DVFS methods based on the VPU duty cycle can hide system complexity, they rely on statistics of VPU operating conditions over a period of time, resulting in an inevitable lag and inability to accurately assess the VPU workload in real time. Summary of the Invention

[0014] The technical problem to be solved by this application is to provide a DVFS method suitable for hardware video decoders. The method can hide the complexity of the system in a video decoding system where the system's available resources (such as CPU and memory) fluctuate frequently, and use clear and easily statistically analyzed data indicators to adaptively characterize the mapping relationship between the decoding performance of the hardware video decoder and the requirements of the video decoding system in real time. The method can then adjust the frequency of the hardware video decoder accordingly to meet the performance and power consumption requirements of the video decoding system.

[0015] In order to solve the above technical problems, the present application discloses a DVFS method for a hardware video decoder, comprising the following steps. Step S31: Count the number of code stream caches to be decoded and video frame caches to be filled in the video decoding system, and call a "code stream cache to be decoded" and a "video frame cache to be filled" a "cache pair to be decoded", and use the number of "cache pairs to be decoded" at a certain moment to represent the workload of the hardware video decoder at that moment. Step S32: According to the real-time workload of the hardware video decoder, determine the relative relationship between the decoding performance and decoding requirements of the hardware video decoder at that moment, and calculate the new operating frequency of the hardware video decoder accordingly. Step S33: Adjust the hardware video decoder to the new operating frequency.

[0016] Furthermore, in step S31, when the number of "buffers to be decoded" and "buffers to be filled video frames" in the video decoding system are not equal at a certain moment, the smaller one is taken as the number of "buffer pairs to be decoded".

[0017] Furthermore, in step S31, the workload of the hardware video decoder at a certain moment is the number of "buffer pairs to be decoded" in the video decoding system at that moment.

[0018] Furthermore, in the step S31, the following three situations may occur. Situation 1: The speed at which "cache pairs to be decoded" are generated in the video decoding system is greater than the speed at which the hardware video decoder processes the "cache pairs to be decoded". At this time, the decoding performance of the hardware video decoder cannot meet the requirements of the video decoding system, and the number of "cache pairs to be decoded" gradually increases. Situation 2: The speed at which "cache pairs to be decoded" are generated in the video decoding system is less than the speed at which the hardware video decoder processes the "cache pairs to be decoded". At this time, the decoding performance of the hardware video decoder exceeds the requirements of the video decoding system, and the number of "cache pairs to be decoded" gradually decreases. Situation 3: The speed at which "cache pairs to be decoded" are generated in the video decoding system is equal to the speed at which the hardware video decoder processes the "cache pairs to be decoded". At this time, the decoding performance of the hardware video decoder just meets the requirements of the video decoding system, and the number of "cache pairs to be decoded" remains basically unchanged.

[0019] Furthermore, in step S31, the greater the number of "cache pairs to be decoded", the greater the workload of the hardware video decoder and the greater the decoding delay; the fewer the number of "cache pairs to be decoded", the smaller the workload of the hardware video decoder and the smaller the decoding delay.

[0020] Furthermore, in step S32, firstly, it is determined whether the current encoding performance of the hardware video decoder is surplus, insufficient, or suitable; and secondly, a new operating frequency of the hardware video decoder is calculated.

[0021] Furthermore, in step S32, M represents a threshold for determining excess decoding performance of the hardware video decoder, and N represents a threshold for determining insufficient decoding performance of the hardware video decoder, where 0≤M<N. If the number of "buffer pairs to be decoded" in the current video decoding system is ≤M, then the current decoding performance of the hardware video decoder is determined to be excess. If the number of "buffer pairs to be decoded" in the current video decoding system is ≥N, then the current decoding performance of the hardware video decoder is determined to be insufficient. If the number of "buffer pairs to be decoded" in the current video decoding system is >M and <N, then the current decoding performance of the hardware video decoder is determined to be adequate.

[0022] Furthermore, in step S32, the optional operating frequency values of the hardware video decoder are arranged in ascending order, and a unique frequency number is assigned to each operating frequency value starting from 0 and incremented by 1, thereby obtaining a mapping table between frequency numbers and optional operating frequency values of the hardware video decoder; min_freq_idx is used to represent the lowest frequency number, max_freq_idx is used to represent the highest frequency number, cur_freq_idx is used to represent the frequency number corresponding to the current operating frequency value of the hardware video decoder, freq_up_step is used to represent the frequency increase step size, and freq_down_step is used to represent the frequency decrease step size; min_freq_idx ≤ cur_freq_idx ≤ max_freq_idx. When the decoding performance of the hardware video decoder is excessive, the difference between cur_freq_idx and freq_down_step is compared with the value of min_freq_idx, and the larger value is used as the new frequency number of the hardware video decoder. When the hardware video decoder's decoding performance is insufficient, the sum of cur_freq_idx + freq_up_step is compared with the value of max_freq_idx, and the smaller value is used as the new frequency number of the hardware video decoder. When the hardware video decoder's decoding performance is adequate, the current frequency number cur_freq_idx of the hardware video decoder is directly used as the new frequency number of the hardware video decoder. After calculating the new frequency number of the hardware video decoder, the mapping table between the frequency number and the optional operating frequency value of the hardware video decoder is searched to obtain the new operating frequency of the hardware video decoder.

[0023] Furthermore, in step S32, the value of cur_freq_idx is updated using the calculated new frequency number of the hardware video decoder.

[0024] The present application also discloses a DVFS device for a hardware video decoder, including a decoder workload calculation module, a decoder frequency calculation module, and a decoder frequency adjustment module. The decoder workload calculation module is used to count the number of code stream caches to be decoded and video frame caches to be filled in the video decoding system, and a "code stream cache to be decoded" and a "video frame cache to be filled" are called a "cache pair to be decoded", and the number of "cache pairs to be decoded" at a certain moment is used to characterize the workload of the hardware video decoder at that moment. The decoder frequency calculation module is used to determine the relative relationship between the decoding performance and decoding requirements of the hardware video decoder at that moment according to the real-time workload of the hardware video decoder, and calculate the new operating frequency of the hardware video decoder accordingly. The decoder frequency adjustment module is used to adjust the hardware video decoder to a new operating frequency.

[0025] The technical effect achieved by this application is: compared with the existing DVFS method of hardware video decoders, this application has the characteristics of small computational complexity, good adaptability, and strong real-time performance, so that the performance of the hardware video decoder can not only meet the real-time requirements of decoding, but also minimize the decoding power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a schematic diagram of the structure of a typical video decoding system.

[0027] Figure 2 yes Figure 1 The flowchart of the video decoding method of the video decoding system is shown.

[0028] Figure 3 This is a flowchart of the DVFS method suitable for hardware video decoders proposed in this application.

[0029] Figure 4 Schematic diagram of the structure of the DVFS device suitable for hardware video decoder proposed in this application.

[0030] Explanation of the reference numerals in the figure: code stream extraction module 11, video decoding module 12, video display module 13, decoder workload calculation module 31, decoder frequency calculation module 32, decoder frequency adjustment module 33. DETAILED DESCRIPTION

[0031] See also Figure 2 , Figure 1 The video decoding method of the video decoding system shown specifically includes the following steps.

[0032] Step S21: The stream extraction module 11 extracts the stream to be decoded and stores it in a "stream to be decoded buffer" for the video decoding module 12 to read and decode. The stream extraction module 11 maintains a software-level buffer queue, which contains multiple "stream to be decoded buffers" of equal size. Each "stream to be decoded buffer" is used to fill the stream to be decoded of a video frame to be decoded.

[0033] The video display module 13 provides a "video frame buffer to be filled" for the video decoding module 12 to write (fill) the decoded video frames (hereinafter referred to as decoded video frames) during decoding. The video display module 13 also maintains a software-level buffer queue, which contains multiple "video frame buffers to be filled" of equal size. Each "video frame buffer to be filled" is used to fill the decoded data of a decoded video frame (i.e., a decoded video frame).

[0034] The purpose of providing multiple "buffers for codestreams to be decoded" and multiple "buffers for video frames to be filled" is to enable codestream extraction module 11, video decoding module 12, and video display module 13 to operate in parallel, thereby improving the performance of the video decoding system. For example, at a given moment, while video decoding module 12 uses a portion of the "buffer for codestreams to be decoded" for decoding, codestream extraction module 11 can still concurrently fill other "buffers for codestreams to be decoded" with codestreams to be decoded. Similarly, at a given moment, while video decoding module 12 uses a portion of the "buffer for video frames to be filled" for decoding, video display module 13 can still concurrently display decoded video frames from other "buffers for video frames to be filled" (see "Buffers for video frames to be filled").

[0035] When performing video decoding, the video decoding module 12 (e.g., a VPU) needs to read a stream to be decoded from a stream buffer to be decoded and simultaneously write the corresponding decoded video frame to a video frame buffer to be filled. In other words, having at least one stream buffer to be decoded and one video frame buffer to be filled is a prerequisite for the video decoding module 12 to be able to decode video frames. In a video decoding system, multiple "stream buffers to be decoded" and multiple "video frame buffers to be filled" are generally allocated. Their number changes in real time with the state of the video decoding system and is not necessarily equal.

[0036] Step S22: The video decoding module 12 reads the code stream to be decoded stored in the "code stream buffer to be decoded" and decodes it, and stores the generated decoded video frames in the "video frame buffer to be filled." When the code stream to be decoded in the "code stream buffer to be decoded" is decoded by the video decoding module 12, the original "code stream buffer to be decoded" is renamed the "decoded code stream buffer." When the "video frame buffer to be filled" is filled with decoded video frames by the video decoding module 12, the original "video frame buffer to be filled" is renamed the "filled video frame buffer."

[0037] Step S23: After decoding the undecoded bitstream, the bitstream extraction module 11 reclaims the corresponding "decoded bitstream buffer" to store the new undecoded bitstream. The video display module 13 displays the decoded video frames stored in the "filled video frame buffer" and then reclaims the corresponding "filled video frame buffer" to fill it with new decoded video frames.

[0038] See also Figure 3 The DVFS method suitable for hardware video decoders proposed in this application includes the following steps.

[0039] Step S31: Count the number of to-be-decoded code stream buffers and to-be-filled video frame buffers in the video decoding system. A "to-be-decoded code stream buffer" and a "to-be-filled video frame buffer" are called a "to-be-decoded cache pair." The number of "to-be-decoded cache pairs" at a certain moment is used to represent the workload of the video decoding module 12 (i.e., the hardware video decoder) at that moment.

[0040] For example, at a certain moment, if there are 5 "stream buffers to be decoded" and 3 "video frame buffers to be filled" in the video decoding system, then there are 3 "buffer pairs to be decoded". For another example, at a certain moment, if there are 3 "stream buffers to be decoded" and 0 "video frame buffers to be filled" in the video decoding system, then there are 0 "buffer pairs to be decoded".

[0041] There are three possible situations in this step:

[0042] Case 1: The speed at which the code stream extraction module 11 and the video display module 13 generate "buffer pairs to be decoded" is faster than the speed at which the video decoding module 12 processes "buffer pairs to be decoded". At this time, the decoding performance of the video decoding module 12 cannot meet the requirements of the video decoding system, and the number of "buffer pairs to be decoded" gradually increases.

[0043] Case 2: The speed at which the code stream extraction module 11 and the video display module 13 generate "buffer pairs to be decoded" is slower than the speed at which the video decoding module 12 processes "buffer pairs to be decoded". At this time, the decoding performance of the video decoding module 12 exceeds the requirements of the video decoding system, and the number of "buffer pairs to be decoded" gradually decreases.

[0044] Case 3: The speed at which the code stream extraction module 11 and the video display module 13 generate "buffer pairs to be decoded" is exactly equal to the speed at which the video decoding module 12 processes "buffer pairs to be decoded". At this time, the decoding performance of the video decoding module 12 just meets the requirements of the video decoding system, and the number of "buffer pairs to be decoded" remains basically unchanged.

[0045] From the above three situations, it can be seen that the number of “buffer pairs to be decoded” in the video decoding system can reflect in real time the mapping relationship between the decoding performance of the current video decoding module 12 and the requirements of the video decoding system.

[0046] This application uses the number of "cache pairs to be decoded" to characterize the workload of the video decoding module 12. The more "cache pairs to be decoded", the greater the workload of the video decoding module 12, and the greater the decoding delay. The fewer "cache pairs to be decoded", the smaller the workload of the video decoding module 12, and the smaller the decoding delay. At the same time, the change in the number of "cache pairs to be decoded" also reflects the matching situation (relative relationship) between the decoding performance of the video decoding module 12 and the requirements of the video decoding system. The number of "cache pairs to be decoded" gradually increases, indicating that the decoding performance of the video decoding module 12 is lower than the requirements of the video decoding system. The number of "cache pairs to be decoded" gradually decreases, indicating that the decoding performance of the video decoding module 12 exceeds the requirements of the video decoding system. The number of "cache pairs to be decoded" remains unchanged, indicating that the decoding performance of the video decoding module 12 just meets the requirements of the video decoding system.

[0047] As can be seen, selecting the number of "buffer pairs to be decoded" in the video decoding system to represent the real-time workload of video decoding module 12 conceals the complexity of the video decoding system and normalizes the relationship between the decoding performance of video decoding module 12 and the system's requirements into a simple data metric. This metric is highly adaptable, clear, and easy to statistically analyze. Furthermore, the above analysis shows that the current number of "buffer pairs to be decoded" in the video decoding system can reflect the current workload of video decoding module 12 in real time without lag, effectively addressing significant fluctuations in the system's transient state.

[0048] As an example, the workload of the video decoding module 12 at a certain moment is the number of “buffer pairs to be decoded” in the video decoding system at that moment.

[0049] Step S32 : determining the relative relationship between the decoding performance and the decoding requirement of the video decoding module 12 at that moment according to the real-time workload of the video decoding module 12 , and calculating a new operating frequency of the video decoding module 12 accordingly.

[0050] First, determine whether the current encoding performance of the video decoding module 12 is excessive, insufficient, or appropriate. M represents the threshold for determining whether the decoding performance of the video decoding module 12 is excessive, and N represents the threshold for determining whether the decoding performance of the video decoding module 12 is insufficient, with 0 ≤ M < N. The specific values of M and N can be adjusted based on the sensitivity of the video decoding system to decoding delay and system power consumption. If the video decoding system is more sensitive to decoding delay, M and N can be selected as smaller integers. If the video decoding system is more sensitive to system power consumption, M and N can be selected as larger integers.

[0051] If the number of "buffer pairs to be decoded" in the current video decoding system is ≤ M, it is determined that the decoding performance of the current video decoding module 12 is excessive and exceeds the requirements of the current video decoding system, and the operating frequency of the video decoding module 12 needs to be reduced.

[0052] If the number of "buffer pairs to be decoded" in the current video decoding system is ≥ N, it is determined that the decoding performance of the current video decoding module 12 is insufficient and lower than the requirements of the current video decoding system, and the operating frequency of the video decoding module 12 needs to be increased.

[0053] If the number of "buffer pairs to be decoded" in the current video decoding system is greater than M and less than N, it is determined that the decoding performance of the current video decoding module 12 is appropriate, and the operating frequency of the video decoding module 12 is maintained unchanged.

[0054] Next, calculate the new operating frequency of the video decoding module 12. Arrange the optional operating frequency values of the video decoding module 12 in ascending order in advance, and assign a unique frequency number, starting from 0 and incrementing by 1, to each operating frequency value. For example, if the video decoding module 12 has N optional operating frequency values, then arrange these N optional operating frequency values in ascending order and, starting from the lowest operating frequency value, assign frequency numbers of 0, 1, 2, ... N-1. The lowest operating frequency value is numbered 0, and the highest operating frequency value is numbered N-1. This creates a mapping table between frequency numbers and the optional operating frequency values of the video decoding module 12. Hereinafter, min_freq_idx will be used to represent the lowest frequency number, max_freq_idx will be used to represent the highest frequency number, cur_freq_idx will be used to represent the frequency number corresponding to the current operating frequency value of the video decoding module 12, freq_up_step will be used to represent the frequency increase step, and freq_down_step will be used to represent the frequency decrease step. min_freq_idx≤cur_freq_idx≤max_freq_idx. freq_up_step indicates the step size by which the frequency number of the video decoding module 12 needs to be increased when the decoding performance of the video decoding module 12 is insufficient, and is a positive integer greater than 0. freq_down_step indicates the step size by which the frequency number of the video decoding module 12 needs to be decreased when the decoding performance of the video decoding module 12 is excessive, and is a positive integer greater than 0. The user can adjust the speed at which the frequency of the video decoding module 12 is increased or decreased by adjusting the values of freq_up_step and freq_down_step. The larger the values of freq_up_step and freq_down_step, the faster the frequency of the video decoding module 12 is increased or decreased. The smaller the values of freq_up_step and freq_down_step, the slower the frequency of the video decoding module 12 is increased or decreased.

[0055] When the decoding performance of the video decoding module 12 is excessive, the difference between cur_freq_idx and freq_down_step is compared with the value of min_freq_idx, and the larger one is used as the new frequency index of the video decoding module 12 .

[0056] When the decoding performance of the video decoding module 12 is insufficient, the sum of cur_freq_idx+freq_up_step and the value of max_freq_idx are compared, and the smaller one is taken as the new frequency number of the video decoding module 12 .

[0057] When the decoding performance of the video decoding module 12 is suitable, the current frequency number cur_freq_idx of the video decoding module 12 is directly used as the new frequency number of the video decoding module 12 .

[0058] After calculating the new frequency number for the video decoding module 12, a mapping table between the frequency number and the optional operating frequency values for the video decoding module 12 is searched to obtain the new operating frequency for the video decoding module 12. At the same time, the value of cur_freq_idx needs to be updated for subsequent adjustments to the operating frequency of the video decoding module 12.

[0059] Step S33: The video decoding module 12 is adjusted to a new operating frequency, so that the performance of the video decoding module 12 can meet the requirements of real-time decoding and can minimize the power consumption of decoding.

[0060] See also Figure 4 The DVFS device suitable for a hardware video decoder proposed in this application includes a decoder workload calculation module 31 , a decoder frequency calculation module 32 , and a decoder frequency adjustment module 33 . Figure 4 The device shown corresponds to Figure 3 The method shown.

[0061] The decoder workload calculation module 31 is used to count the number of code stream caches to be decoded and video frame caches to be filled in the video decoding system. A "code stream cache to be decoded" and a "video frame cache to be filled" are called a "cache pair to be decoded", and the number of "cache pairs to be decoded" at a certain moment is used to represent the workload of the video decoding module (i.e., the hardware video decoder) at that moment.

[0062] The decoder frequency calculation module 32 is used to determine the relative relationship between the decoding performance and decoding requirements of the video decoding module at that moment according to the real-time workload of the video decoding module, and calculate the new operating frequency of the video decoding module accordingly.

[0063] The decoder frequency adjustment module 33 is used to adjust the video decoding module to a new operating frequency.

[0064] This application proposes a DVFS method suitable for hardware video decoders. It selects the number of "pairs of caches to be decoded" of a VPU (hardware video decoder) to characterize the VPU's workload. This method hides the complexity of the video decoding system and normalizes the relationship between the VPU's decoding performance and the system's requirements into a simple data metric. This method is highly adaptable, clear, and easy to calculate. Furthermore, the number of "pairs of caches to be decoded" of the current VPU can reflect the current VPU's workload in real time without lag, effectively coping with large fluctuations in the system's instantaneous state and avoiding the occurrence of significant frame drops.

[0065] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A DVFS method for a hardware video decoder, characterized in that: The method includes the following steps: Step S31: Counting the number of to-be-decoded stream buffers and to-be-filled video frame buffers in the video decoding system. A "to-be-decoded stream buffer" and a "to-be-filled video frame buffer" are referred to as a "to-be-decoded buffer pair." The number of "to-be-decoded buffer pairs" at a given moment represents the workload of the hardware video decoder at that moment. Step S32: determining the relative relationship between the decoding performance of the hardware video decoder and the decoding demand at that moment based on the real-time workload of the hardware video decoder, and calculating a new operating frequency of the hardware video decoder accordingly; Step S33: Adjust the hardware video decoder to a new operating frequency.

2. The DVFS method of the hardware video decoder according to claim 1, wherein: In step S31, at a certain moment, when the number of "buffers to be decoded" and "buffers to be filled with video frames" in the video decoding system is not equal, the smaller one is taken as the number of "buffer pairs to be decoded".

3. The DVFS method of the hardware video decoder according to claim 2, wherein: In step S31, the workload of the hardware video decoder at a certain moment is the number of “buffer pairs to be decoded” in the video decoding system at that moment.

4. The DVFS method of the hardware video decoder according to claim 1, wherein: In step S31, the following three situations may occur: Case 1: The rate at which "buffer pairs to be decoded" are generated in the video decoding system is faster than the rate at which the hardware video decoder can process them. In this case, the decoding performance of the hardware video decoder cannot meet the requirements of the video decoding system, and the number of "buffer pairs to be decoded" gradually increases. Case 2: The rate at which "buffer pairs to be decoded" are generated in the video decoding system is slower than the rate at which the hardware video decoder can process them. In this case, the decoding performance of the hardware video decoder exceeds the requirements of the video decoding system, and the number of "buffer pairs to be decoded" gradually decreases. Case 3: The speed at which "buffer pairs to be decoded" are generated in the video decoding system is equal to the speed at which the hardware video decoder processes them. In this case, the decoding performance of the hardware video decoder just meets the requirements of the video decoding system, and the number of "buffer pairs to be decoded" remains essentially unchanged.

5. The DVFS method of the hardware video decoder according to claim 4, wherein: In step S31, a greater number of "buffer pairs to be decoded" indicates a greater workload on the hardware video decoder and a greater decoding delay; a smaller number of "buffer pairs to be decoded" indicates a smaller workload on the hardware video decoder and a smaller decoding delay.

6. The DVFS method of the hardware video decoder according to claim 1, wherein: In step S32, firstly, it is determined whether the current encoding performance of the hardware video decoder is surplus, insufficient, or suitable; and secondly, a new operating frequency of the hardware video decoder is calculated.

7. The DVFS method of the hardware video decoder according to claim 6, wherein: In step S32, M represents a threshold for determining whether the hardware video decoder has excessive decoding performance, and N represents a threshold for determining whether the hardware video decoder has insufficient decoding performance, where 0≤M<N. If the number of "buffer pairs to be decoded" in the current video decoding system is ≤ M, it is determined that the current decoding performance of the hardware video decoder is excessive; If the number of "buffer pairs to be decoded" in the current video decoding system is ≥ N, it is determined that the current decoding performance of the hardware video decoder is insufficient; If the number of “buffer pairs to be decoded” in the current video decoding system is greater than M and less than N, it is determined that the current decoding performance of the hardware video decoder is appropriate.

8. The DVFS method of the hardware video decoder according to claim 7, wherein: In step S32, the optional operating frequency values of the hardware video decoder are arranged in order from low to high, and a unique frequency number is set for each operating frequency value starting from 0 and incremented by 1, thereby obtaining a mapping table between the frequency number and the optional operating frequency value of the hardware video decoder; min_freq_idx is used to represent the lowest frequency number, max_freq_idx is used to represent the highest frequency number, cur_freq_idx is used to represent the frequency number corresponding to the current operating frequency value of the hardware video decoder, freq_up_step is used to represent the frequency increase step size, and freq_down_step is used to represent the frequency decrease step size; min_freq_idx≤cur_freq_idx≤max_freq_idx; When the decoding performance of the hardware video decoder is in excess, the difference between cur_freq_idx and freq_down_step is compared with the value of min_freq_idx, and the larger one is used as the new frequency number of the hardware video decoder. When the decoding performance of the hardware video decoder is insufficient, compare the sum of cur_freq_idx + freq_up_step with the value of max_freq_idx, and take the smaller one as the new frequency number of the hardware video decoder; When the decoding performance of the hardware video decoder is suitable, the current frequency number cur_freq_idx of the hardware video decoder is directly used as the new frequency number of the hardware video decoder; After the new frequency number of the hardware video decoder is calculated, a mapping table between the frequency number and the optional operating frequency value of the hardware video decoder is searched to obtain the new operating frequency of the hardware video decoder.

9. The DVFS method of the hardware video decoder according to claim 8, wherein: In step S32, the value of cur_freq_idx is updated using the calculated new frequency number of the hardware video decoder.

10. A DVFS device for a hardware video decoder, characterized in that: It includes a decoder workload calculation module, a decoder frequency calculation module, and a decoder frequency adjustment module; The decoder workload calculation module is used to count the number of to-be-decoded stream buffers and to-be-filled video frame buffers in the video decoding system. A "to-be-decoded stream buffer" and a "to-be-filled video frame buffer" are referred to as a "to-be-decoded buffer pair." The number of "to-be-decoded buffer pairs" at a certain moment represents the workload of the hardware video decoder at that moment. The decoder frequency calculation module is used to determine the relative relationship between the decoding performance and decoding requirements of the hardware video decoder at that moment according to the real-time workload of the hardware video decoder, and calculate the new operating frequency of the hardware video decoder accordingly; The decoder frequency adjustment module is used to adjust the hardware video decoder to a new operating frequency.

Images