Video processing method and electronic device

Abstract

The application discloses a video processing method and an electronic device. The method comprises the following steps: in response to a load evaluation trigger condition being met, obtaining execution state information of each video processing task in a video processing task set; wherein, the load evaluation trigger condition represents a time when the current load of the video processing task set needs to be evaluated; based on the execution state information, determining a target working parameter of a target processing unit executing the video processing task; wherein, the execution state information at least comprises information representing that the video processing task is in an active task state or a non-active task state; and controlling the target processing unit to enter a target power consumption state corresponding to the target working parameter.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and more specifically to a video processing method and electronic device. Background Technology

[0002] With the increasing demand for high-definition, high-frame-rate video playback from electronic devices, the power consumption generated by video encoding and decoding processing is accounting for a more significant proportion of the overall energy consumption of these devices, affecting their battery life. To balance video processing performance and power consumption, Dynamic Voltage and Frequency Scaling (DVFS) technology is widely used in the power management of Video Processing Units (VPUs). It primarily involves dynamically adjusting the operating voltage and frequency of the video processing unit based on the load of the video processing task.

[0003] However, in practical applications, the load of video processing tasks is not constant. For example, when a user pauses video playback, the corresponding video encoding and decoding tasks, although not finished, no longer actually occupy VPU processing resources, and the load has actually decreased; when the user resumes playback, the load increases again. Current dynamic voltage and frequency adjustment schemes typically only trigger load assessment and operating parameter adjustments when a task begins to be issued or ends to be recycled, failing to respond promptly to load change events such as task pauses or resumptions. This results in the VPU maintaining a high operating voltage and frequency during periods when the load has decreased, causing unnecessary power consumption waste; while when the load suddenly increases, the VPU may fail to improve performance in time, leading to decoding stuttering or frame rate drops, affecting the user experience. Summary of the Invention

[0004] In view of the above, this application provides the following technical solution:

[0005] A video processing method, comprising:

[0006] In response to the fulfillment of the load assessment trigger condition, the execution status information of each video processing task in the video processing task set is obtained; wherein, the load assessment trigger condition represents the moment when the current load of the video processing task set needs to be assessed.

[0007] Based on the execution status information, the target operating parameters of the target processing unit executing the video processing task are determined; wherein, the execution status information includes at least information indicating whether the video processing task is in an active task state or an inactive task state;

[0008] The target processing unit is controlled to enter the target power consumption state corresponding to the target operating parameters.

[0009] Optionally, the load assessment triggering condition includes a first condition, which represents the start time of the target processing unit processing video frames. The step of obtaining the execution status information of each video processing task in the video processing task set in response to satisfying the load assessment triggering condition includes:

[0010] In response to satisfying the first condition, a first time parameter of each video processing task in the video processing task set is obtained, wherein the first time parameter represents the time point at which the most recent frame of video data corresponding to the video processing task is processed.

[0011] Based on the first time parameter, the execution status information of the video processing task is determined.

[0012] Optionally, determining the execution status information of the video processing task based on the first time parameter includes:

[0013] Based on the first time parameter corresponding to each video processing task and the current time point, the idle time of the video processing task is determined.

[0014] Based on the idle time and target duration threshold corresponding to the video processing task, it is determined whether the video processing task is in the active task state or the inactive task state; wherein, the target duration threshold corresponding to each video processing task is determined based on the attribute feature information of the video data stream processed by the video processing task.

[0015] Optionally, the load assessment triggering condition includes a second condition, which indicates a change in the number of video processing tasks in the video processing task set that are in an active task state; determining the target operating parameters of the target processing unit executing the video processing task based on the execution status information includes:

[0016] Based on the execution status information, determine all first video processing tasks currently in the active task state in the video processing task set;

[0017] Based on the first video processing task, the target working parameters are determined.

[0018] Optionally, determining the target working parameters based on the first video processing task includes:

[0019] The current total load is calculated based on the video data processing requirements corresponding to each video processing task in the first video processing task.

[0020] Based on the preset mapping relationship between load and operating parameters, the target operating parameters corresponding to the current total load are determined.

[0021] Optionally, calculating the current total load based on the video data processing requirements corresponding to each video processing task in the first video processing task includes:

[0022] Obtain the video data resolution and frame rate corresponding to each video processing task in the first video processing task;

[0023] Based on the resolution and the frame rate, calculate the sub-load corresponding to each video processing task in the first video processing task;

[0024] The current total load is obtained by summing the sub-loads corresponding to all video processing tasks in the first video processing task.

[0025] Optionally, determining the target operating parameters of the target processing unit executing the video processing task based on the execution status information further includes:

[0026] In response to determining, based on the execution status information, that there is no video processing task in the set of video processing tasks that is in the active task state, the target operating parameters are determined as parameters for causing the target processing unit to enter a first power consumption state; wherein, the power consumption parameter of the first power consumption state is less than the target power consumption threshold.

[0027] Optionally, it also includes:

[0028] When the target operating parameters are parameters used to cause the target processing unit to enter the first power consumption state, the power supply to the target processing unit is cut off, or the operating clock is stopped from being supplied to the target processing unit, so that the target processing unit enters the first power consumption state.

[0029] Optionally, the method further includes:

[0030] In response to the second video processing task being in the inactive task state, it is detected that the second video processing task has been rescheduled;

[0031] Update the execution status information of the second video processing task to an active task status;

[0032] Based on the updated execution status information, the steps of determining the target operating parameters and controlling the entry into the target power consumption state are re-executed.

[0033] An electronic device, comprising:

[0034] The target processing unit is used to perform video processing tasks;

[0035] The control unit is configured to obtain the execution status information of each video processing task in the video processing task set in response to the fulfillment of the load assessment trigger condition; wherein the load assessment trigger condition represents the moment when the current load of the video processing task set needs to be assessed.

[0036] Based on the execution status information, the target operating parameters of the target processing unit executing the video processing task are determined; wherein, the execution status information includes at least information indicating whether the video processing task is in an active task state or an inactive task state;

[0037] The target processing unit is controlled to enter the target power consumption state corresponding to the target operating parameters. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0039] Figure 1 A flowchart illustrating a video processing method provided in an embodiment of this application;

[0040] Figure 2 A schematic diagram illustrating the distribution of load adjustment points in a video processing scenario, provided as an embodiment of this application;

[0041] Figure 3 A flowchart illustrating dynamic voltage and frequency adjustment of a video processing unit, provided as an embodiment of this application;

[0042] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0043] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] The terms "first" and "second," etc., used in this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units may include steps or units not listed, but may include steps or units not listed.

[0045] This application provides a video processing method applicable to electronic devices with video encoding and decoding capabilities, such as mobile terminals, wearable devices, and smart home devices. The method monitors the execution status of video processing tasks. When load assessment triggering conditions are met, it acquires the execution status information of each task in the current video processing task set. This execution status information at least indicates whether each video processing task is in an active or inactive state. Based on this status information, it determines the target operating parameters of the target processing unit executing the video processing task. According to these target operating parameters, it controls the target processing unit to enter the corresponding target power consumption state. Compared to traditional processing methods that only sense task start and end and cannot distinguish task pause states, leading to artificially high load, this application identifies the activity state of tasks, enabling power consumption control to accurately match actual load requirements, thereby reducing dynamic power consumption and eliminating static power consumption when there are no active tasks, thus improving the energy efficiency and battery life of electronic devices.

[0046] See Figure 1 The diagram illustrates a flowchart of a video processing method provided in an embodiment of this application, which may include the following steps:

[0047] S101. In response to the fulfillment of the load assessment triggering condition, obtain the execution status information of each video processing task in the video processing task set.

[0048] The video processing task set refers to the set of all video encoding and decoding tasks that are currently being processed by the target processing unit or have been issued but not yet retrieved by the electronic device. The target processing unit represents the hardware unit that performs the video encoding and decoding tasks, such as the VPU (Video Processing Unit).

[0049] The load assessment trigger condition represents the moment when the current load of the video processing task set is assessed. That is, the electronic device can start the load assessment process only when a specific moment or a specific event occurs, so as to avoid unnecessary computational overhead.

[0050] If the target processing unit is a VPU, the video processing method can be executed through the VPU's driver module, which accesses hardware resources and manages the VPU's power consumption, task scheduling, and voltage / frequency adjustment. First, it is determined whether the load assessment trigger condition is met. This trigger condition can be set based on various factors. For example, in one embodiment of this application, the trigger condition can be the moment when processing each frame of video data begins (i.e., before each frame is decoded or encoded). In another embodiment, the trigger condition can be the moment when a change in the number of active tasks is detected. The implementation methods of the above two trigger conditions will be described in detail in subsequent embodiments.

[0051] When the load assessment trigger condition is met, the VPU driver obtains the execution status information of all tasks in the video processing task set. In this embodiment, the execution status information indicates whether each video processing task is currently actually occupying the resources of the target processing unit to process video data. For example, the execution status information includes at least information indicating whether each video processing task is currently in an active task state or an inactive task state.

[0052] The active task status indicates that the video processing task is currently processing video data (such as decoding or encoding video frames); the inactive task status indicates that the video processing task is currently paused or idle (such as when the user has paused video playback).

[0053] Specifically, the VPU driver can traverse the currently maintained task queue and, for each video processing task in the queue, obtain its current activity status identifier. This status identifier can be a flag indicating whether the task is in an active or inactive state. The status identifier can be represented in various forms, such as a boolean flag or an integer value, where different values ​​correspond to different states. In one specific embodiment, "1" can represent an active task state and "0" can represent an inactive task state; in another embodiment, "true" and "false" can also be used for distinction.

[0054] For example, a user opens two video applications simultaneously on a tablet: one playing a high-definition video in the foreground, and the other paused in the background. At this time, the video processing task queue maintained by the VPU driver includes two video decoding tasks. When it detects that the next frame of video data is about to be processed (meeting the load assessment trigger condition), the VPU driver iterates through these two tasks and obtains their execution status information: the foreground video task, because it is continuously processing video frames, is marked as an "active task"; the background paused video task, because it has not processed new video frames for a long time, is marked as an "inactive task".

[0055] In this step, by acquiring execution status information in response to load assessment trigger conditions, load assessment can be performed only when necessary, avoiding the additional power consumption overhead of continuous monitoring. At the same time, by acquiring the active or inactive status of each task, a data foundation is provided for subsequent accurate load assessment and power consumption control, avoiding the problem of artificially inflated load caused by including all tasks (including inactive tasks) in the load calculation.

[0056] S102. Based on the execution status information, determine the target operating parameters of the target processing unit that performs the video processing task.

[0057] The target processing unit refers to the hardware module that actually performs video encoding and decoding processing. In different implementations, the target processing unit can be a video processing unit (VPU), a video encoding and decoding module in a graphics processing unit (GPU), or a processing core with integrated video encoding and decoding functions in a central processing unit (CPU). For ease of explanation, the target processing unit in this embodiment is a VPU.

[0058] Target operating parameters refer to configurable parameters used to control the operating state of a target processing unit, including configuration information indicating the operating voltage and / or operating frequency of the target processing unit. For example, target operating parameters may include pairing information for a voltage value of 0.65V and a frequency value of 300MHz, indicating that the target processing unit operates at that voltage and frequency. These types of parameters are typically used in Dynamic Voltage and Frequency Scaling (DVFS) scenarios.

[0059] The target operating parameters may also include control parameters used to instruct the target processing unit to enter a target power consumption state. The target power consumption state characterizes the power consumption level of the target processing unit in the current execution state of the video processing task. For example, the target power consumption state may include a normal operating state, a low-power state, or a power-off state. For instance, when the target power consumption state is a low-power state, the corresponding control parameters may be instructions to instruct the target processing unit to enter a power-off state or a clock-stop state; when the target power consumption state is a high-performance state, the corresponding control parameters may be specific operating voltage and / or operating frequency configuration information.

[0060] In one specific embodiment, if the target power consumption state is a power-off state, the target operating parameters may include a power-off command to instruct the power management unit to cut off the power supply to the target processing unit; if the target power consumption state is a clock-stop state, the target operating parameters may include a clock-stop command to instruct the clock controller to stop providing the operating clock to the target processing unit. It should be noted that the target operating parameters may individually indicate one of the aforementioned power consumption states, or may simultaneously include multiple configuration information depending on the actual application scenario and load requirements. Subsequent embodiments will describe in detail the methods for determining different types of target operating parameters.

[0061] Taking the VPU as the target processing unit as an example, the VPU driver first identifies the currently active tasks from the set of video processing tasks. Then, based on the processing requirements of these active tasks (e.g., the video resolution and frame rate to be decoded), it calculates the actual processing load required. Finally, based on the calculated actual load, it queries a predefined load-to-operating-parameter mapping relationship to determine the matching target operating parameters. It should be noted that when multiple active tasks exist, the VPU driver comprehensively evaluates the total load requirement of all active tasks. When no active tasks exist, the VPU driver can determine the target operating parameters as control instructions to instruct the target processing unit to enter a low-power state (e.g., a power-off state or a clock-stopped state).

[0062] This method of determining target operating parameters based on execution status information allows for configuration of target processing unit parameters solely based on the actual active task load, rather than on the artificially high load of all issued tasks (including paused tasks) as in traditional processing. This avoids unnecessary performance overkill and power waste, enabling precise control of power supply on demand. Furthermore, by defining target operating parameters as voltage and frequency configuration information or power-off control commands, both dynamic power optimization and static power elimination scenarios can be covered simultaneously.

[0063] S103, Control the target processing unit to enter the target power consumption state corresponding to the target operating parameters.

[0064] The target power consumption state refers to the power consumption level of the target processing unit under different operating parameters. For example, when the target operating parameters are a high operating voltage and frequency, the target processing unit is in a high-performance, high-power state; when the target operating parameters are a low operating voltage and frequency, the target processing unit is in a low-performance, low-power state; when the target operating parameters are a power-off or clock-stop control command, the target processing unit is in a zero-power or near-zero-power state.

[0065] For example, when the target processing unit is a VPU, the VPU driver controls the target processing unit to switch to the corresponding power consumption state according to the determined target operating parameters. When the target operating parameters are voltage and / or frequency configuration information (e.g., voltage 0.65V, frequency 300MHz), the VPU driver adjusts the operating state of the target processing unit according to a predetermined frequency and voltage regulation process through the clock controller and voltage regulator.

[0066] Typically, the frequency upscaling process involves first increasing the voltage and then increasing the frequency to ensure the voltage is sufficient for a stable frequency increase. The frequency downscaling process involves first decreasing the frequency and then decreasing the voltage to ensure the frequency is already within a safe range at a lower voltage. When the target operating parameter is a power-off or clock-stop control command, the VPU driver triggers a power gating mechanism. This cuts off the power supply to the target processing unit via the power management unit or stops supplying the operating clock to the target processing unit via the clock controller, thereby putting the target processing unit into an extremely low-power state. At this time, the leakage current power consumption of the target processing unit drops to near zero.

[0067] This application provides a video processing method that, in response to a load assessment trigger condition, obtains the execution status information of each video processing task in a video processing task set. This execution status information includes at least information indicating whether a video processing task is in an active or inactive task state, thereby accurately identifying the actual activity state of the video processing tasks. Based on this execution status information, the method determines the target operating parameters of the target processing unit executing the video processing task. This changes the basis for determining the target operating parameters from the traditional total load of all issued tasks to the actual load of active tasks, avoiding performance overkill caused by inactive tasks being included in the load calculation. By controlling the target processing unit to enter the target power consumption state corresponding to the target operating parameters, a closed loop from load assessment to power consumption control is achieved. Compared to traditional technologies that can only sense the start and end of tasks and cannot distinguish between task pause states, leading to inflated load and wasted power, this application identifies the activity state of tasks, enabling power consumption control to precisely match actual load requirements. This effectively reduces dynamic power consumption while ensuring video processing performance and eliminates static power consumption when there are no active tasks, thereby improving the energy efficiency and battery life of electronic devices.

[0068] In one embodiment of this application, the load assessment triggering condition includes a first condition, which represents the start time of the target processing unit processing video frames. In response to satisfying the load assessment triggering condition, obtaining the execution status information of each video processing task in the video processing task set includes:

[0069] S201. In response to the fulfillment of the first condition, obtain the first time parameters of each video processing task in the video processing task set.

[0070] The first condition is met whenever the target processing unit is about to begin processing a new frame of video data. This condition embodies a periodic or event-driven load assessment triggering mechanism. During video encoding and decoding, video frame processing is continuous and periodic (e.g., a 30fps video is processed approximately every 33 milliseconds). Using the start of each frame processing as the trigger point allows the system to check whether the current task state has changed before each actual processing (e.g., whether a task has been paused since the last processing), thereby ensuring the timeliness and accuracy of load assessment.

[0071] The first-time parameter represents the time point at which the most recent frame of video data corresponding to a video processing task is processed. For example, when a video decoding task completes the decoding of a frame, the current system time can be recorded as the task's first-time parameter. The first-time parameter is the fundamental data for determining whether a task is active. If a task frequently completes video frame processing recently, its first-time parameter will be constantly updated, indicating that the task is actively working; conversely, if a task does not update its first-time parameter for a long time, it indicates that the task may be paused or idle. It should be noted that there is a correspondence between the first-time parameter and the video processing tasks. Each task in the video processing task set has its own first-time parameter, and the first-time parameters of different tasks are recorded separately and do not interfere with each other. This correspondence ensures that the activity status of each task can be determined independently, thereby supporting accurate load statistics in multi-tasking scenarios.

[0072] Specifically, a timestamp variable can be maintained for each video processing task to record the system time when the task most recently completed video frame processing. Whenever a video processing task completes the encoding or decoding of a video frame, it reads the current system time and updates the corresponding timestamp variable. When the first condition is triggered (i.e., before each frame processing begins), the VPU driver traverses the currently maintained video processing task queue. For each task in the queue, it reads its maintained timestamp variable as the task's first time parameter.

[0073] S202. Based on the first time parameter, determine the execution status information of the video processing task.

[0074] Based on the first time parameter of each video processing task and the current system time, the elapsed time since the task last processed a video frame (i.e., idle time) can be calculated. This idle time is then compared to a preset or dynamically determined threshold. If the idle time is less than the threshold, it indicates that the task has recently been actively processing video frames, and its state is determined to be an active task state; if the idle time is greater than or equal to the threshold, it indicates that the task has not processed video frames for a long time, and its state is determined to be an inactive task state.

[0075] This embodiment defines the start time of video frame processing by the target processing unit as the first condition, and obtains the first time parameter for each task in response to this condition. Based on the first time parameter, the execution status information of the task is determined, thus realizing the function of actively checking the task activity status at the start of each frame processing. This allows the system to promptly detect and identify inactive tasks in a paused or idle state, providing a data basis for subsequently removing these tasks from load statistics. Compared with traditional technologies that only update load statistics at the start and end of tasks, this embodiment can track task status changes more precisely, thereby more accurately reflecting the current actual processing load, providing a reliable basis for subsequent power consumption adjustments, and ultimately achieving more effective power savings.

[0076] Furthermore, in this embodiment of the application, the process of determining the execution status information of the video processing task based on the first time parameter may include:

[0077] S301. Determine the idle time of each video processing task based on the first time parameter and the current time point.

[0078] Idle time refers to the length of time that has elapsed since a video processing task last processed a video frame. Idle time reflects how long the task has been in an idle state. If the idle time is short, it indicates that the task was still actively processing video frames recently and is likely still in a playback state; if the idle time is long, it indicates that the task has not processed new video frames for a long time and is likely in a paused or idle state.

[0079] In practical scenarios, the currently maintained video processing task queue can be traversed. For each video processing task in the queue, the following steps can be performed: read the first time parameter corresponding to the task (denoted as T1); read the current system time (denoted as T2); and calculate the idle duration Δt = T2 - T1. The unit of this idle duration Δt can be milliseconds or microseconds, depending on the precision requirements of the system clock. For example, in some operating systems, millisecond-level timestamps are usually sufficient for the needs of video encoding and decoding scenarios.

[0080] S302. Based on the idle time and target duration threshold corresponding to the video processing task, determine whether the video processing task is in an active or inactive state.

[0081] The target duration threshold for each video processing task is determined based on the attribute characteristics of the video data stream processed by the task and can be updated with the video stream attributes. That is, different video processing tasks in this embodiment can have different duration thresholds; for example, the duration threshold may be related to the frame rate of the video processed by the task. Frame rate (Frames Per Second, FPS) represents the number of video frames processed per second. The higher the frame rate, the shorter the time interval between adjacent frames. For example, a 25fps video has a single frame interval of 40 milliseconds; a 60fps video has a single frame interval of approximately 16.7 milliseconds.

[0082] In one embodiment of this application, the target duration threshold can be determined by a baseline threshold, which can be set to twice the frame interval, such as a baseline threshold T. base =2×(1000 / F), where F represents the frame rate and 1000 represents 1000ms. For example, for a 25fps video, the frame interval is 40 milliseconds, and the baseline threshold T... base The frame interval is 80 milliseconds; for 60fps video, the frame interval is approximately 16.7 milliseconds, and the baseline threshold T is... base It is approximately 33.3 milliseconds; for a 15fps video, the frame interval is approximately 66.7 milliseconds, and the target duration threshold is approximately 133.3 milliseconds.

[0083] The method of dynamically determining the target duration threshold based on frame rate in this embodiment has the advantage of strong business-specific compatibility. The target duration threshold can be dynamically adjusted according to the video frame rate, making the judgment rules closely tied to the actual characteristics of different video services. For high frame rate videos (e.g., 60fps), the threshold is shorter (approximately 33 milliseconds), enabling the system to respond to task pauses more quickly and promptly remove inactive tasks from load statistics. For low frame rate videos (e.g., 15fps), the threshold is longer (approximately 133 milliseconds), avoiding misjudgment of task pauses due to large normal frame intervals. This adaptive target duration threshold mechanism ensures accurate determination of the task execution status for video tasks at different frame rates.

[0084] The method of dynamically determining the target duration threshold based on frame rate is also more in line with the characteristics of human visual perception. Taking 25fps video as an example, the threshold setting of 80 milliseconds is close to the human eye's perception threshold for visual stuttering. When the video frame processing interval exceeds 80 milliseconds, users can usually perceive stuttering or pauses in the picture. At this time, the task is judged as inactive and the VPU's voltage frequency configuration is reduced, which will not negatively affect the user's viewing experience. Conversely, during brief idle periods within 80 milliseconds (such as normal frame processing interval fluctuations), the task remains active, ensuring that the VPU can continuously and smoothly process video frames. This threshold setting achieves a good balance between power consumption optimization and user experience.

[0085] It should be noted that the above example of dynamically determining the target duration threshold based on frame rate is only one implementation method in this application embodiment. In practical applications, other frame rate-related calculation methods can also be used to determine the target duration threshold, such as using three times the frame interval, or introducing other video attributes (such as bitrate, resolution, etc.) for comprehensive calculation. Those skilled in the art can flexibly adjust the threshold calculation method according to the actual application scenario and power consumption optimization requirements.

[0086] The calculated idle time is compared with the target duration threshold for the task. Based on the comparison result, it is determined whether the task is in an active or inactive state. If the idle time is less than the target duration threshold, the video processing task is determined to be in an active state; if the idle time is greater than or equal to the target duration threshold, the video processing task is determined to be in an inactive state.

[0087] For example, the calculated idle time for task A is 40 milliseconds, and the idle time for task B is 5000 milliseconds. Determine their respective target duration thresholds. For instance, if task A processes video frames at 25fps, its target duration threshold TA is 80 milliseconds. If task B processes video frames at 30fps, its target duration threshold TB is 66.7 milliseconds. Then, for task A: the idle time of 40 milliseconds is less than the threshold of 80 milliseconds, so task A is considered an active task. For task B: the idle time of 5000 milliseconds is greater than the threshold of 66.7 milliseconds, so task B is considered an inactive task. If the user resumes playback of task B, task B begins processing new video frames, and its first time parameter will be updated to the current time. The idle time will be reset to a value close to 0 (e.g., a few milliseconds). At this point, the idle time is less than the threshold, and task B will be reclassified as an active task.

[0088] In this embodiment, the duration threshold can adaptively adapt to video streams with different frame rates. For high frame rate videos, the threshold is shorter, enabling faster response to task pauses and timely removal of inactive tasks from load statistics. For low frame rate videos, the threshold is longer, avoiding misjudgment of pauses due to large normal frame intervals. This dynamic threshold mechanism ensures both timely power consumption optimization and avoids misjudgment of normal playback tasks, thus achieving a good balance between power saving and user experience.

[0089] In another embodiment of this application, the load assessment triggering condition includes a second condition, which characterizes a change in the number of video processing tasks in the video processing task set that are in an active task state. The changes in the number of active tasks include, but are not limited to, the following two scenarios: A decrease in active tasks, i.e., when a task that was originally active is determined to be inactive because its idle time exceeds a target duration threshold, the number of active tasks decreases. An increase in active tasks, i.e., when a task that was originally inactive is rescheduled (e.g., the user clicks the play button to resume video playback), and its state switches to an active state, the number of active tasks increases. When either of the above situations occurs, the second condition is satisfied, and the system should trigger a load assessment and a re-determination of the target operating parameters.

[0090] In this embodiment of the application, the target operating parameters of the target processing unit that performs the video processing task are determined based on the execution status information, including:

[0091] S401. Based on the execution status information, determine all first video processing tasks that are currently in the active task state in the video processing task set.

[0092] S402. Based on the first video processing task, determine the target working parameters.

[0093] In step S401, the first video processing task refers to all video processing tasks currently in an active task state. Specifically, when the second condition is triggered, based on the execution status information, all tasks in the video processing task set currently marked as active tasks are determined; these tasks are the first video processing tasks.

[0094] The electronic device can detect changes in the number of active tasks through proactive detection. This involves comparing the current set of active tasks with the previously recorded set; if they differ, it indicates a change in the number of active tasks, triggering the second condition. Alternatively, it can detect changes in the number of active tasks through an event-driven approach. When the state of a task changes (e.g., a task is deemed inactive due to idle timeout, or an inactive task is rescheduled), the VPU driver can directly trigger the second condition and execute this step.

[0095] Upon detecting the triggering of the second condition, the VPU driver reads the currently maintained execution status information. This execution status information can be maintained in various ways, such as maintaining an active task list that records all currently active tasks. A flag is set in the status identifier of each task, and this flag is read by traversing the task queue. Based on the execution status information, the VPU driver filters out all tasks with an active status, forming the first video processing task. It should be noted that this does not necessarily require creating a new data structure or list; it only requires determining which tasks are active, for example, by traversing the task queue and checking the status flag.

[0096] For example, initially, both video processing tasks A and B are active (assuming task B was recently paused and hasn't exceeded the threshold). The VPU driver maintains a set of active tasks including tasks A and B. After a period of time, task B is marked as inactive because its idle time exceeds the target duration threshold. The VPU driver detects that the number of active tasks has changed from 2 to 1, satisfying the second condition. In response to the second condition, the VPU driver determines all currently active tasks based on the execution status information, concluding that the first video processing task includes only task A.

[0097] For example, initially, task A is active and task B is inactive. When the user clicks the play button to resume playback of task B, the application begins sending new video frame data to the VPU. The VPU driver detects that task B has been rescheduled and updates its status to active, changing the number of active tasks from 1 to 2, thus satisfying the second condition. In response to the second condition, the VPU driver determines all currently active tasks, thus identifying the first video processing task as including both task A and task B.

[0098] By triggering load assessment in response to changes in the number of active tasks, the system can respond immediately to any actual changes in load and adjust the VPU's operating parameters in a timely manner.

[0099] In step S402, target operating parameters are determined based on all first video processing tasks in active task states. Attribute information (such as resolution, frame rate, etc.) of the video data stream processed by each active task can be obtained, and the total processing demand of all active tasks can be comprehensively evaluated. Then, target operating parameters matching this total processing demand are determined. For example, when the total processing demand is low, a lower operating voltage and frequency configuration is selected; when the total processing demand is high, a higher operating voltage and frequency configuration is selected. In this way, inactive tasks in paused or idle states are excluded from load evaluation, making the evaluated load value closer to the actual workload of the VPU, thereby avoiding performance overkill and power waste caused by inflated load, and achieving precise power management with on-demand power supply.

[0100] In one embodiment of this application, the process of determining target working parameters based on a first video processing task includes:

[0101] S501. Calculate the current total load based on the video data processing requirements corresponding to each video processing task in the first video processing task.

[0102] The current total load refers to a comprehensive measure of the processing capacity requirements of all currently active tasks on the target processing unit. When the target processing unit is a VPU, this load value reflects the total amount of video data that the VPU needs to process at the current moment and is the main basis for determining the working parameters. In one embodiment of this application, the process of calculating the current total load based on the video data processing requirements corresponding to each video processing task in the first video processing task may include:

[0103] S5011. Obtain the video data resolution and frame rate corresponding to each video processing task in the first video processing task.

[0104] S5012. Calculate the sub-load corresponding to each video processing task in the first video processing task based on the resolution and frame rate.

[0105] S5013. Accumulate the sub-loads corresponding to all video processing tasks in the first video processing task to obtain the current total load.

[0106] In step S5011, each task in the first video processing task acquires the resolution and frame rate information of the video data stream it processes. This information is typically sent by the application to the VPU driver module or program during the task configuration phase, and the VPU driver can store it in association with the task. Resolution includes width (number of pixels) and height (number of pixels), for example, 1920×1080. Frame rate represents the number of video frames processed per second, for example, 30 frames per second (fps).

[0107] In step S5012, for each video processing task in the first video processing task, the sub-load corresponding to that task is calculated based on its resolution and frame rate. In one embodiment of this application, the sub-load is calculated as the sub-load equal to the product of the resolution width, resolution height, and frame rate. This product represents the number of pixels that the task needs to process per second, which can better reflect the processing capacity requirements of the target processing unit (such as VPU) for the task.

[0108] In step S5013, all sub-loads are summed to obtain the current total load, which represents the total processing capacity requirement of all active tasks that the target processing unit needs to process simultaneously.

[0109] For example, video processing task A has a resolution of 1920×1080 and a frame rate of 30; video processing task B has a resolution of 1280×720 and a frame rate of 30. Obtain the resolution and frame rate for each task: Task A: width 1920, height 1080, frame rate 30; Task B: width 1280, height 720, frame rate 30. Calculate the sub-load for each task: Sub-load for Task A = 1920×1080×30 = 62208000 (pixels / second); Sub-load for Task B = 1280×720×30 = 27648000 (pixels / second). Add the two sub-loads together: Current total load = 62208000 + 27648000 = 89856000 (pixels / second). If the first video processing task only contains Task A, then the current total load is 62208000 pixels / second. If the first video processing task is empty (no active task), then the current total load is 0.

[0110] It should be noted that the calculation of sub-loads by multiplying resolution and frame rate and summing them to obtain the current total load described in this application embodiment is only an exemplary implementation. In specific applications, the calculation of the current total load can also be carried out in many other ways. For example, different weight coefficients can be assigned to different tasks according to the encoding type (such as H.264, H.265, AV1, etc.) used by each video processing task, and tasks with higher encoding complexity can be given larger weights; parameters such as bitrate, color depth, and chroma sampling format of video data can also be introduced as weighting factors for load calculation; or the number of pixels that each task needs to process per unit time can be directly counted as sub-loads, etc. This application embodiment does not limit these methods.

[0111] S502. Based on the preset mapping relationship between load and operating parameters, determine the target operating parameters corresponding to the current total load.

[0112] The mapping relationship between load and operating parameters refers to the defined rules used to convert load values ​​into corresponding operating parameters of the target processing unit. This mapping relationship is usually stored in the form of an OPP (Operating Performance Point) table. The OPP table stores the pre-verified correspondence between load ranges and voltage and frequency configurations. For example, the decoded OPP table shown in Table 1 defines the frequency and voltage values ​​corresponding to different load ranges.

[0113] Table 1

[0114] Total task load Frequency (MHz) Voltage (V) No higher than 4K 30fps 300 0.65 No higher than 4K 60fps 600 0.75 Higher than 4K 60fps 800 0.75

[0115] As shown in Table 1, target operating parameters typically include configuration information for operating voltage and operating frequency. For example, when the current total load is within the range of no more than 4K 30fps, the corresponding target operating parameters are a voltage of 0.65 volts and a frequency of 300 MHz.

[0116] For example, upon obtaining the current total load value, the load range to which the current total load belongs can be found in the OPP table through iteration or binary search. The voltage and frequency values ​​corresponding to this load range are read from the OPP table as target operating parameters. The determined voltage and frequency values ​​are then output as target operating parameters for subsequent power consumption control operations. It should be noted that different chips and different application scenarios (encoding or decoding) may use different OPP tables. The VPU driver can maintain separate OPP tables for encoding and decoding to adapt to different processing requirements.

[0117] In this embodiment, the target operating parameters are determined based on a preset mapping relationship between load and operating parameters, enabling the system to quickly convert load requirements into hardware configuration. This mapping relationship is pre-verified, ensuring that within each load range, the corresponding voltage and frequency configurations meet performance requirements while keeping power consumption within a reasonable range. This allows for rapid conversion from load to operating parameters, satisfying real-time requirements.

[0118] In some embodiments of this application, the process of determining the target operating parameters of the target processing unit performing the video processing task based on the execution state information further includes: in response to determining, based on the execution state information, that there is no video processing task in the video processing task set that is in an active task state, determining the target operating parameters as parameters for causing the target processing unit to enter a first power consumption state.

[0119] In this context, "no video processing task is in an active state" means that all tasks in the video processing task set are identified as inactive. That is, no video processing task is currently processing video data. This situation typically occurs in the following scenarios: the user has paused all playing videos, or all video playback tasks have ended but the `release` interface (a system call interface used to release video encoding / decoding task resources and terminate the current task execution) has not yet been called to reclaim resources. In traditional processing, even if all tasks are paused, the VPU driver still counts these tasks as load, causing the VPU to maintain a high voltage and frequency configuration, resulting in unnecessary static power consumption waste. The embodiments of this application can solve this problem.

[0120] The first power consumption state refers to a low-power operating state of the target processing unit. Compared with the high-power state during normal operation, the power consumption parameters (such as operating voltage, operating frequency, current consumption, etc.) of the first power consumption state are less than a preset target power consumption threshold. This target power consumption threshold can be a reference value pre-configured in the target processing unit driver or stored in the electronic device system configuration file. Its specific value can be determined comprehensively based on factors such as the hardware specifications of the target processing unit, chip manufacturing process, power consumption requirements of the application scenario, and energy efficiency strategy of the electronic device system.

[0121] For example, the target power consumption threshold can be preset by the device manufacturer or system developer before shipment based on the chip's hardware characteristics (such as static power consumption curves and leakage current parameters) and product positioning, and remain unchanged during system operation. This target power consumption threshold can also be dynamically adjusted by the system based on factors such as current battery level, temperature, and user usage scenarios to achieve a more flexible power management strategy. Those skilled in the art can reasonably set the value and update method of this target power consumption threshold according to actual application requirements.

[0122] In one embodiment of this application, the first power consumption state can be a power-off state or a clock-stopped state. In these states, the static power consumption (such as leakage current power consumption) of the target processing unit can be reduced to near zero. It should be noted that the specific implementation of the first power consumption state can be selected based on the hardware characteristics of the target processing unit and system requirements. For example, for a VPU that supports power gating, a power-off state can be selected to achieve maximum power saving; for scenarios requiring rapid wake-up, a clock-stopped state can be selected to shorten recovery time.

[0123] The parameter used to cause the target processing unit to enter a first power consumption state is a specific type of target operating parameter. This parameter indicates that the target processing unit should enter a low-power state. The parameter can take various forms, such as: a control command (e.g., a "power-down command" or "clock-stop command") that directly instructs the hardware to perform the corresponding low-power operation; an extremely low voltage and / or frequency value (e.g., 0 volts, 0 Hz), which the target processing unit will attempt to switch to upon receiving, effectively entering a power-down or clock-stop state; or a specific flag or status code, which triggers a low-power mode switch when detected by the hardware. Regardless of the form it takes, its main function is to trigger the target processing unit to enter a first power consumption state through the target operating parameter. This application does not impose any limitations on this.

[0124] In this embodiment, by determining the target operating parameters as the parameters for entering the first power consumption state when there are no active tasks, it is possible to identify scenarios where no video tasks need to be processed, and trigger the switching to a low-power mode accordingly. This solves the problem in some technologies where the VPU continues to be powered even when all tasks are paused, resulting in wasted static power consumption, thereby extending the battery life of mobile devices.

[0125] In one embodiment of this application, when the target operating parameter is a parameter used to cause the target processing unit to enter a first power consumption state, the power supply to the target processing unit is switched, or the operating clock is stopped from being supplied to the target processing unit, so that the target processing unit enters the first power consumption state.

[0126] Disconnecting the power supply refers to breaking the power supply circuit of the target processing unit through the Power Management Unit (PMU), causing the target processing unit to completely lose its power supply. At this time, the target processing unit consumes no electrical energy, and its static power consumption (including leakage current power consumption) drops to zero. Stopping the clock supply refers to stopping the supply of the clock signal to the target processing unit through the clock controller (CLK framework). At this time, the logic circuits of the target processing unit stop working, but because the power supply still exists, there will still be a small amount of static power consumption (such as leakage current). Compared with disconnecting the power supply, stopping the clock usually results in a faster wake-up speed and is suitable for scenarios requiring rapid recovery.

[0127] Specifically, the target operating parameters can be parsed to determine their type. If the target operating parameter is a voltage and frequency configuration parameter, the voltage and frequency are adjusted according to the DVFS (Dynamic Voltage and Frequency Scaling) process. If the target operating parameter is a control command for entering the first power consumption state, the low-power operation of this step is executed. When the target operating parameter is a power-off command, a power-off request is sent to the power management unit. Upon receiving the request, the power management unit cuts off the power supply circuit to the target processing unit. At this time, the target processing unit is completely powered off and consumes no power.

[0128] When the target operating parameter is a stop clock command, a stop clock request is sent to the clock controller. Upon receiving the request, the clock controller stops providing the operating clock signal to the target processing unit. At this time, the logic circuit of the target processing unit stops working, but since the power supply is still available, a very small amount of leakage current still exists.

[0129] It should be noted that either cutting off the power supply or stopping the clock can be performed, or both can be performed simultaneously depending on the system configuration. For example, in scenarios where maximum power saving is desired, cutting off the power supply can be selected; in scenarios where rapid wake-up is desired, only stopping the clock can be selected. Regardless of the method used, the target processing unit enters the first power consumption state, and its power consumption parameters are all less than the preset target power consumption threshold. Optionally, the status feedback of the target processing unit can also be read to confirm that it has successfully entered the first power consumption state, thereby ensuring the correct execution of the power control operation.

[0130] In this embodiment, when a previously inactive video processing task is rescheduled to process new video data, the system updates the task's status to an active task status and, based on the updated status information, re-executes the determination of target working parameters and power consumption control operations, thereby ensuring that the target processing unit (such as VPU) can restore performance supply in a timely manner and guarantee the smoothness of video playback.

[0131] In one implementation, the process may include:

[0132] S601, In response to the second video processing task being in an inactive task state, it is detected that the second video processing task has been rescheduled.

[0133] It can monitor the status changes of each video processing task. When a video processing task that is currently inactive (such as the second video processing task) is rescheduled,

[0134] For example, when a user clicks the pause button on a video, the application begins submitting new video frame data to the underlying framework that implements video and audio data encoding and decoding. This framework then forwards the processing request to the VPU driver. The VPU driver can then determine that the task has been reactivated by receiving the processing request.

[0135] S602. Update the execution status information of the second video processing task to the active task status.

[0136] The VPU driver can update the status flags corresponding to the task, changing them from inactive to active. Simultaneously, the VPU driver can also update the task's initial time parameter, setting it to the current time point as the basis for subsequent idle time calculations.

[0137] If the VPU has previously entered a power-off state or a clock-stopped state, this step should also include restoring power to the VPU and / or the operating clock. Specifically, the VPU driver sends a power-on request to the power management unit to restore power to the VPU; and / or sends a clock recovery request to the clock controller to resume supplying the operating clock to the VPU.

[0138] S603. Based on the updated execution status information, re-execute the steps of determining the target operating parameters and controlling the entry into the target power consumption state.

[0139] After the status update is complete, the VPU driver will re-execute. Figure 1 The steps of the video processing method shown are as follows: obtaining the execution status information of each task, determining the target operating parameters based on the information, and controlling the target processing unit to enter the corresponding target power consumption state.

[0140] Since the status of the second video processing task has been updated from inactive to active, this task has been added to the active task set. The VPU driver will re-determine the target operating parameters based on the total load of all active tasks, including this task. Typically, adding an active task will increase the total load, and the VPU driver will accordingly increase the VPU's operating voltage and frequency (if it was previously downclocked), or exit the power-down / clock-stop state and restore the normal operating voltage and frequency configuration.

[0141] When an inactive task is rescheduled, this embodiment can promptly update the task's status to active and re-execute the load assessment and power adjustment process, ensuring that the target processing unit quickly recovers from a low-power state to a working state that matches the current load. In this way, it not only ensures the smoothness of video playback when the user resumes playback (without noticeable delays or stuttering), but also achieves a complete power management closed loop from entering to exiting low-power mode.

[0142] See Figure 2This illustration shows a schematic diagram of the load adjustment point distribution in a video processing scenario provided by an embodiment of this application. The load adjustment point represents the moment when the load assessment triggering condition is met, i.e., the moment when the current load of the video processing task set needs to be reassessed.

[0143] like Figure 2 As shown, in traditional video processing modes, load adjustment points are typically set only at the moments when video tasks are assigned and released. For example, when task 1 is assigned, load adjustment point 1 is triggered; when task 2 is assigned, load adjustment point 2 is triggered; when task 1 is released, load adjustment point 3 is triggered; when task 2 is released, load adjustment point 4 is triggered; when task 3 is assigned, load adjustment point 5 is triggered; and when task 3 is released, load adjustment point 6 is triggered. However, during the time period from when task 1 is paused (i.e., in an inactive task state) to when task 1 resumes (i.e., switches back to an active task state), and during the time period from when task 2 is paused to when task 2 resumes, the traditional solution cannot detect the paused state of the tasks and therefore fails to set the corresponding load adjustment points. This results in the VPU driver still including paused tasks in the load during the aforementioned time periods, causing the load statistics to be artificially high. The VPU maintains an unnecessary high voltage and high frequency configuration, resulting in wasted power consumption.

[0144] To address the aforementioned issues, the video processing method provided in this application adds an additional load adjustment point to the traditional load adjustment point. For example... Figure 2 As shown in load adjustment point 1.5, when task 1 is paused (i.e., the idle time of task 1 exceeds the target duration threshold and is judged as an inactive task), the system triggers additional load adjustment, removing task 1 from the active task set and re-determining the VPU's target operating parameters based on the actual load of the remaining active tasks. Similarly, as... Figure 2 As shown in load adjustment point 3.5, when task 2 is paused, the system triggers additional load adjustment to further optimize the VPU's operating parameter configuration.

[0145] In addition, when a paused task is rescheduled (e.g.) Figure 2 When Task 1 and Task 2 resume, the system also triggers load adjustment, re-adding the task to the active task set. Based on the updated total load of active tasks, the system redetermines the VPU's target operating parameters to ensure the VPU can provide sufficient processing performance to guarantee smooth video playback. Through these added load adjustment points, this embodiment achieves real-time tracking and precise response to the activity status of video processing tasks, enabling load assessment and power consumption control to remain synchronized with changes in actual active tasks. This minimizes both dynamic and static power consumption of the VPU while ensuring video processing performance.

[0146] See Figure 3 The illustration shows a flowchart of a video processing unit dynamic voltage frequency adjustment according to an embodiment of this application, wherein the video processing unit is represented by VPU and the dynamic voltage frequency adjustment is represented by DVFS.

[0147] exist Figure 3 In the illustrated embodiment, the dynamic voltage frequency adjustment process of the video processing unit includes the following steps:

[0148] S31, VPU receives the decoding request for the current video frame.

[0149] The VPU driver receives a current video frame decoding request from the multimedia framework, which corresponds to a specific video processing task.

[0150] S32. Determine whether the video processing task corresponding to the current decoding request is in the target video processing task queue that is in an active task state.

[0151] The VPU driver checks whether the video processing task has been recorded in the currently maintained active task queue. If yes, it jumps directly to step S36; otherwise, it executes step S33.

[0152] S33. Add the video processing task to the target video processing task queue.

[0153] If the video processing task corresponding to the current decoding request is not in the active task queue (for example, the task was previously removed from the queue due to being paused and is now being reactivated), the VPU driver adds the task to the active task queue.

[0154] S34. Determine if the VPU is powered on.

[0155] After a new task is added, the VPU driver checks the current power supply status of the VPU. If the VPU is not yet powered on (e.g., it was previously powered off due to no active tasks), then step S35 is executed; if the VPU is already powered on, then the process jumps directly to step S36.

[0156] S35 and VPU are powered on.

[0157] If the VPU is in a power-off state, the VPU driver sends a power-on request to the power management unit to restore power to the VPU, enabling it to process video decoding tasks normally.

[0158] S36. Update the timestamp of this video processing task.

[0159] The VPU driver updates the first time parameter of the current video processing task (i.e., the time point of the most recent video data processing) and sets it to the current system time for subsequent idle time calculation.

[0160] S37. Traverse the video processing task queue that is in an active task state, and check whether the difference between the last activity time and the current time of each video processing task is greater than the threshold.

[0161] The VPU driver iterates through each task in the currently active task queue, calculates the difference between the first time parameter of each task and the current system time (i.e., idle time), and compares this idle time with the target duration threshold corresponding to the task. This threshold is dynamically determined based on the frame rate of the video data stream processed by the task, and can be set to twice the frame interval, for example.

[0162] S38. Remove inactive video processing tasks.

[0163] For tasks with an idle duration greater than or equal to the target duration threshold, the VPU driver classifies them as inactive tasks and removes them from the active task queue. Tasks with an idle duration less than the threshold are retained, i.e., step S39 is executed.

[0164] S39, Retain the task.

[0165] S40. Determine if the number of active tasks is greater than 0.

[0166] After completing the traversal and removal operations, the VPU driver checks whether there is still at least one active task in the current active task queue. If the number of active tasks is greater than 0, step S41 is executed; if the number of active tasks is equal to 0, step S42 is executed.

[0167] S41. Calculate the total load of the VPU.

[0168] If at least one active task exists, the VPU driver calculates the current total load based on the processing requirements of all active tasks. Specifically, for each active task, the resolution and frame rate of the video data it processes are obtained, the product of the resolution and frame rate is calculated as the sub-load of that task, and then all sub-loads are summed to obtain the current total load.

[0169] S42. Turn off the VPU power.

[0170] If there are no active tasks, the VPU driver triggers power gating, sends a power-down request to the power management unit, cuts off the power supply to the VPU, and puts the VPU into the first power consumption state (power-down state), thereby eliminating static power consumption.

[0171] S43. Adjust the DVFS configuration according to the load.

[0172] If there are active tasks, the VPU driver queries the preset load-operating parameter mapping relationship (such as the OPP table) based on the current total load calculated in step S41, determines the target operating parameters (including operating voltage and operating frequency) corresponding to the current total load, and adjusts the voltage and frequency configuration of the VPU through the clock controller and voltage regulator.

[0173] S44. Execute the decoding of the current frame.

[0174] After completing the above load assessment and power consumption adjustment, the VPU executes the decoding operation of the current video frame in the adjusted working state.

[0175] pass Figure 3 The process described in this embodiment of the application implements proactive checking and dynamic adjustment of the video processing task activity status before decoding each frame. This process integrates two load assessment triggering mechanisms: on the one hand, steps S32 to S35 implement a response to task reactivation (meeting the second condition); on the other hand, steps S37 to S39 implement periodic (before decoding each frame) active status checks and maintenance. Compared to some technologies that only adjust the load at the start and end of a task, this embodiment can perceive the pause and resume status of tasks in real time, dynamically maintain the active task queue, and accurately configure the VPU's voltage and frequency according to the actual load of active tasks, thereby minimizing the dynamic and static power consumption of the VPU while ensuring video decoding performance.

[0176] In the video processing method provided in this application embodiment, the driving module of the video processing unit (VPU) can dynamically adjust the dynamic voltage-frequency adjustment (DVFS) configuration of the VPU according to the updated active task load (i.e., the total load of all video processing tasks currently determined to be active tasks). For example, it can be based on the dynamic power consumption formula:

[0177] P dynamic ∝V DD 2 ×f.

[0178] Among them, P dynamic This represents dynamic power consumption, where f is the operating frequency, and V is the voltage. DD The operating voltage is used as the reference point, and the dynamic power consumption is proportional to the square of the operating voltage and the operating frequency. Therefore, when the operating voltage decreases, the power consumption will decrease at a quadratic rate, resulting in significant power savings.

[0179] For example, in a certain application scenario, the VPU's operating voltage decreases from 0.75V to 0.65V, and its operating frequency decreases from 800MHz to 300MHz. Based on the dynamic power consumption formula above, the percentage change in power consumption can be calculated:

[0180] Pafter / P before = (0.65) 2 (×300) / (0.75) 2 (×800)≈0.28

[0181] Among them, P after This indicates the adjusted dynamic power consumption (e.g., power consumption after reducing the VPU operating voltage and frequency); P before This indicates the dynamic power consumption before adjustment (e.g., the power consumption value before adjusting the VPU operating parameters); P after / P before This represents the ratio of the adjusted dynamic power consumption to the dynamic power consumption before adjustment, i.e., the percentage change in power consumption. Using this example, the adjusted dynamic power consumption is approximately 28% of the original, meaning the dynamic power consumption can be reduced by a maximum of about 72%. Therefore, through the active task identification and load assessment mechanism of this application embodiment, the VPU driver can promptly adjust the VPU's voltage and frequency configuration when the load decreases, thereby significantly reducing dynamic power consumption.

[0182] Furthermore, the video processing method provided in this application also shows significant optimization in terms of static power consumption. When the VPU driver detects that there are no active video processing tasks, i.e., all video tasks have been paused or completed, the VPU driver triggers a power gating mechanism to control the disconnection of the VPU's power supply circuit. At this time, the VPU is completely powered off, and its leakage current power consumption drops to almost zero. Compared with some technologies that keep the VPU powered even when there are no tasks, resulting in continuous static power consumption waste, the embodiments of this application can eliminate static power consumption and further extend the battery life of electronic devices.

[0183] In another embodiment of this application, an electronic device is also provided, see [link to relevant documentation]. Figure 4 The electronic device includes:

[0184] Target processing unit 11 is used to perform video processing tasks;

[0185] The control unit 12 is configured to obtain the execution status information of each video processing task in the video processing task set in response to the fulfillment of the load assessment trigger condition; wherein the load assessment trigger condition represents the moment when the current load of the video processing task set needs to be assessed.

[0186] Based on the execution status information, the target operating parameters of the target processing unit executing the video processing task are determined; wherein, the execution status information includes at least information indicating whether the video processing task is in an active task state or an inactive task state;

[0187] The target processing unit is controlled to enter the target power consumption state corresponding to the target operating parameters.

[0188] It should be noted that the specific implementation of the control unit in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0189] In another embodiment of this application, a readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the video processing method as described in any of the preceding claims.

[0190] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0191] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0192] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0193] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A video processing method, comprising: In response to the fulfillment of the load assessment trigger condition, the execution status information of each video processing task in the video processing task set is obtained; wherein, the load assessment trigger condition represents the moment when the current load of the video processing task set needs to be assessed. Based on the execution status information, the target operating parameters of the target processing unit executing the video processing task are determined; wherein, the execution status information includes at least information indicating whether the video processing task is in an active task state or an inactive task state; The target processing unit is controlled to enter the target power consumption state corresponding to the target operating parameters.

2. The method according to claim 1, wherein the load assessment triggering condition includes a first condition, the first condition representing the start time of the target processing unit processing video frames, and the step of obtaining the execution status information of each video processing task in the video processing task set in response to satisfying the load assessment triggering condition includes: In response to satisfying the first condition, a first time parameter of each video processing task in the video processing task set is obtained, wherein the first time parameter represents the time point at which the most recent frame of video data corresponding to the video processing task is processed. Based on the first time parameter, the execution status information of the video processing task is determined.

3. The method according to claim 2, wherein determining the execution status information of the video processing task based on the first time parameter includes: Based on the first time parameter corresponding to each video processing task and the current time point, the idle time of the video processing task is determined. Based on the idle time and target duration threshold corresponding to the video processing task, it is determined whether the video processing task is in the active task state or the inactive task state; wherein, the target duration threshold corresponding to each video processing task is determined based on the attribute feature information of the video data stream processed by the video processing task.

4. The method according to claim 1 or 2, wherein the load assessment triggering condition includes a second condition, the second condition indicating that the number of video processing tasks in the set of video processing tasks that are in an active task state changes; The determination of the target operating parameters of the target processing unit executing the video processing task based on the execution status information includes: Based on the execution status information, determine all first video processing tasks currently in the active task state in the video processing task set; Based on the first video processing task, the target working parameters are determined.

5. The method according to claim 4, wherein determining the target working parameters based on the first video processing task includes: The current total load is calculated based on the video data processing requirements corresponding to each video processing task in the first video processing task. Based on the preset mapping relationship between load and operating parameters, the target operating parameters corresponding to the current total load are determined.

6. The method according to claim 5, wherein calculating the current total load based on the video data processing requirements corresponding to each video processing task in the first video processing task includes: Obtain the video data resolution and frame rate corresponding to each video processing task in the first video processing task; Based on the resolution and the frame rate, calculate the sub-load corresponding to each video processing task in the first video processing task; The current total load is obtained by summing the sub-loads corresponding to all video processing tasks in the first video processing task.

7. The method according to claim 4, wherein determining the target operating parameters of the target processing unit executing the video processing task based on the execution status information further includes: In response to determining, based on the execution status information, that there is no video processing task in the set of video processing tasks that is in the active task state, the target operating parameters are determined as parameters for causing the target processing unit to enter a first power consumption state; wherein, the power consumption parameter of the first power consumption state is less than the target power consumption threshold.

8. The method according to claim 7, further comprising: When the target operating parameters are parameters used to cause the target processing unit to enter the first power consumption state, the power supply to the target processing unit is cut off, or the operating clock is stopped from being supplied to the target processing unit, so that the target processing unit enters the first power consumption state.

9. The method according to claim 1, further comprising: In response to the second video processing task being in the inactive task state, it is detected that the second video processing task has been rescheduled; Update the execution status information of the second video processing task to an active task status; Based on the updated execution status information, the steps of determining the target operating parameters and controlling the entry into the target power consumption state are re-executed.

10. An electronic device, comprising: The target processing unit is used to perform video processing tasks; The control unit is configured to obtain the execution status information of each video processing task in the video processing task set in response to the fulfillment of the load assessment trigger condition; wherein the load assessment trigger condition represents the moment when the current load of the video processing task set needs to be assessed. Based on the execution status information, the target operating parameters of the target processing unit executing the video processing task are determined; wherein, the execution status information includes at least information indicating whether the video processing task is in an active task state or an inactive task state; The target processing unit is controlled to enter the target power consumption state corresponding to the target operating parameters.

Images