A mobile terminal power consumption optimization method based on reinforcement learning

Abstract

This invention discloses a mobile terminal power consumption optimization method based on reinforcement learning, comprising: collecting terminal operating state data, processing and fusing multi-source data to form a continuous state sequence; merging multi-source events within a time window, identifying candidate time windows, and constructing a power consumption opportunity graph and interference chain; generating candidate control actions, inputting them into an evaluator to predict three execution results, and calculating the net benefit of the actions; determining the budget based on the opportunity graph, combining the net benefit with the reinforcement learning model, and outputting the optimal control action; executing the target control action, occupying the target window and suppressing the interference event chain, and deferring the release of related tasks; obtaining execution feedback data, performing delay settlement based on the window fulfillment results, and updating the reinforcement learning model. This invention achieves collaborative power consumption optimization control of multi-source resources in mobile terminals by constructing a power consumption opportunity graph and combining counterfactual evaluation and reinforcement learning decision-making mechanisms.

Description

Technical Field

[0001] This invention relates to the field of reinforcement learning technology, and in particular to a method for optimizing power consumption of mobile terminals based on reinforcement learning. Background Technology

[0002] As mobile terminals increasingly enhance their capabilities in communication, display, graphics processing, sensing, and multitasking, power consumption during operation becomes a prominent issue. Existing mobile terminal power optimization methods typically revolve around processor downclocking, screen brightness adjustment, background task restrictions, and network strategy switching. These methods adjust terminal operating parameters through preset rules, threshold judgments, or scene recognition to reduce overall power consumption and extend battery life. Some technical solutions also incorporate machine learning or reinforcement learning methods to dynamically select power control strategies based on current load, battery status, network status, and other operational information.

[0003] However, most existing technologies rely on instantaneous load, static rules, or single decision results as control criteria, primarily focusing on surface-level states such as processor utilization, screen status, or network activity. They struggle to perform correlation analysis on fine-grained operational events such as background wake-ups, periodic network packets, rendering drivers, and sensor sampling, and also find it difficult to identify the interference paths these events cause to the terminal's entry into a continuous energy-saving operating state. For continuous disturbance chains formed by multiple sequential operational events, existing technologies typically lack unified modeling and timing reconstruction capabilities. This means that while control strategies may suppress local modules, they still cannot effectively guarantee stable energy-saving operation within the target time window.

[0004] Existing technologies for power consumption control often employ immediate feedback or short-cycle reward updates, making it difficult to assess the impact of delays caused by control actions in subsequent operational phases, such as compensatory wake-ups, task recovery, network reconnection, and fluctuations in interactive responses. This can easily lead to situations where short-term power consumption decreases without a significant improvement in overall energy efficiency. Particularly in reinforcement learning applications, updating strategies solely based on immediate gains can disrupt candidate energy-saving time windows, impacting long-term optimization results.

[0005] Therefore, how to provide a method for optimizing the power consumption of mobile terminals based on reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a mobile terminal power consumption optimization method based on reinforcement learning. This invention comprehensively utilizes power opportunity graph construction, counterfactual three-branch evaluator, and reinforcement learning decision-making techniques to perform correlation analysis on multi-source operating events during the operation of the mobile terminal, identify candidate time windows and their interference event chains that affect the continuous maintenance of the target power consumption state, evaluate, decide, and execute candidate control actions, and achieve coordinated power consumption control of objects such as processor, graphics processor, display, network communication, background tasks, and sensors. It has the advantages of being able to identify fine-grained sources of power consumption interference, reduce compensatory wake-up and bounce power consumption, improve the dwell effect of the target power consumption state, and take into account both user experience and overall battery life performance.

[0007] A mobile terminal power consumption optimization method based on reinforcement learning according to an embodiment of the present invention includes:

[0008] Collect the operating status data of the mobile terminal, process the operating status data, and form a status sequence;

[0009] Within a preset time window, multi-source running events in the state sequence are correlated and merged to identify candidate time windows that can enter the target power consumption state, a power consumption opportunity map is constructed, and the correlation between each candidate time window and the corresponding interference event chain are marked.

[0010] Candidate control actions are generated based on the power opportunity diagram. The running state data, power opportunity diagram and candidate control actions are input into the counterfactual three-branch evaluator to predict the state evolution results of the candidate control actions under the following conditions: immediate execution, execution after the end of the target candidate time window, and not execution while maintaining the current scheduling state. The net benefit of the candidate control actions is then determined.

[0011] The power opportunity budget for the future time range is determined based on the power opportunity map. The power opportunity budget and net revenue are used as inputs, and a reinforcement learning model is used to make decisions on the set of candidate control actions to obtain the target control action.

[0012] Based on the target control action, opportunity occupancy control is performed on the target candidate time window in the power consumption opportunity diagram. Hierarchical suppression and deferred release control are performed on the interference event chain that disrupts the target candidate time window, so as to maintain the target power consumption state within the target candidate time window and release the deferred events after the window ends.

[0013] Within the observation time window after the execution of the target control action, actual operation feedback is obtained. Based on the realization results of the corresponding candidate time windows in the power opportunity map, the target control action is delayed and the reinforcement learning model is updated.

[0014] Optionally, the operating status data includes processor operating status data, graphics processing status data, display operating status data, wireless communication status data, system wake-up status data, task scheduling status data, and power status data.

[0015] Optionally, the process of processing the running status data to form a status sequence includes timestamp alignment, data cleaning, outlier removal and standardization of the running status data, and chronological arrangement according to a preset sampling period to generate a status sequence corresponding to a continuous time window.

[0016] Optionally, the construction of the power consumption opportunity graph and the marking of the correlation between each candidate time window and the corresponding interference event chain include:

[0017] Within a preset time window, the state sequence of the mobile terminal is scanned in segments, and the running events in each time period are extracted. The events are then uniformly identified according to their occurrence time, duration, triggering source, belonging task object, and resource object, forming an event set.

[0018] The event set is continuously merged and causal relationship analyzed. Events that occur continuously in time, are mutually transmitted in triggering relationship, and are coupled in resource occupation are grouped into the same event segment. Based on the occupation relationship of each event segment on the processor, graphics processor, display module, wireless communication module and sensor module, candidate time windows for characterizing the target power consumption state can be entered.

[0019] A power consumption opportunity graph is constructed using candidate time windows as graph nodes and the connection, splicing, and blocking relationships between adjacent candidate time windows as graph edges.

[0020] For each candidate time window, the running events associated with the candidate time window are traced back along the time sequence and the trigger transmission sequence to determine the source event, transmission event and maintenance event that caused the candidate time window to be shortened, interrupted or failed. The source event, transmission event and maintenance event are then linked together according to their sequential relationship to obtain the corresponding interference event chain.

[0021] The candidate time windows, their connections, splicing relationships, and blocking relationships, as well as the interference event chains corresponding to each candidate time window, are associated and registered to generate a power consumption opportunity map.

[0022] Optionally, determining the net benefit of the candidate control action includes:

[0023] Candidate time windows and corresponding interference event chains are extracted based on power consumption opportunity diagrams, and candidate control actions are generated by combining operating status data. Each candidate control action is bound to the corresponding target candidate time window and interference event chain to form an evaluation object for candidate time window protection.

[0024] For each evaluation object, a counterfactual three-branch evaluator is constructed. The counterfactual three-branch evaluator includes a state mirroring unit, a window constraint injection unit, and a branch generation unit. The state mirroring unit copies the current running state data and the current power opportunity map to form a unified baseline evaluation state. The window constraint injection unit writes the start and end boundaries of the target candidate time window, the positional relationship of the corresponding interference event chain, and the action time of the candidate control action into the baseline evaluation state. The branch generation unit generates an immediate execution branch, a window deferred branch, and a hold branch under the same baseline evaluation state.

[0025] The state evolution process of candidate control actions after execution at the current moment is deduced by using the immediate execution branch, the state evolution process of candidate control actions after execution at the end of the target candidate time window is deduced by using the window deferral branch, and the state evolution process of candidate control actions when they are not executed is deduced by using the hold branch, so as to obtain the power consumption state change, candidate time window continuity change, interference event chain propagation change and terminal operation response change corresponding to each branch.

[0026] The deduction results of each branch are compared to determine the impact of candidate control actions on the target candidate time window when executed immediately, the impact on the target candidate time window when executed deferred, and the impact on the natural propagation of the interference event chain when not executed. Based on the impact of occupation, protection and retention, the corresponding branch evaluation results are generated.

[0027] Based on the branch evaluation results of each branch, determine the net benefit corresponding to each candidate control action.

[0028] Optionally, the step of using a reinforcement learning model to make a decision on the set of candidate control actions to obtain the target control action includes:

[0029] Based on the power opportunity graph, target candidate time windows, the correlation between candidate time windows, and the interference event chain corresponding to each candidate time window are extracted. Combined with the net benefit of candidate control actions, decision input data for reinforcement learning model is generated.

[0030] A proximal policy optimization reinforcement learning model is used to make decisions on a set of candidate control actions. A policy network and a value network corresponding to the proximal policy optimization reinforcement learning model are constructed. The policy network outputs the selection result of each candidate control action based on the decision input data, and the value network outputs the state value corresponding to the current decision input data.

[0031] The target candidate time window in the power opportunity graph is written into the policy network to form a window-constrained decision state. The candidate control actions are classified and encoded according to immediate execution actions, window-deferred actions, and hold actions, and then input into the policy network. The policy network outputs the selection result of the candidate control action based on the boundary of the target candidate time window, the net benefit corresponding to the candidate control action, and the event chain disturbance evaluation result.

[0032] The selection results output by the policy network are screened for window consistency. Candidate control actions that conflict with the start and end boundaries of the target candidate time window, conflict with the propagation direction of the interference event chain, or overlap with the already occupied candidate time window are eliminated. The screened candidate control actions are then input into the value network for state value evaluation to obtain the target control action.

[0033] Record the target candidate time window, action execution time, action category, and window occupancy result corresponding to the target control action.

[0034] Optionally, the step of performing hierarchical suppression and delayed release control on the interference event chain that disrupts the target candidate time window, so as to maintain the target power consumption state within the target candidate time window and release the delayed events after the window ends, includes:

[0035] Based on the target control action, the corresponding target candidate time window and associated interference event chain are determined from the power opportunity diagram. The start and end boundaries of the target candidate time window, the triggering order of the interference event chain, and the execution time of the target control action are written into the execution control sequence.

[0036] The running events in the interference event chain are segmented into links to identify source triggering events, transmission events, and maintenance events. Based on the positional relationship between each running event and the target candidate time window, the type of resources occupied, and the impact on the continuity of the target candidate time window, corresponding suppression control flags and release control flags are generated for the running events.

[0037] Hierarchical suppression control is performed on the interference event chain based on suppression control flags:

[0038] Entry blocking control is performed on source-triggered events located before the boundary of the target candidate time window; propagation truncation control is performed on transit events located inside the target candidate time window; duration compression control is performed on maintenance events that maintain the existence of the interference event chain; and the running events after hierarchical suppression are written into the deferred event queue.

[0039] Deferred release control is executed after the window in the deferred event queue based on the release control flag:

[0040] The release sequence is reconstructed according to the original triggering order and resource occupation order of the interference event chain. After the target candidate time window ends, the deferred running events are released in sequence. During the release process, adjacent running events of the same type are merged and released to form a concentrated release sequence after the window.

[0041] Within the target candidate time window, the integrity of the execution window of the running event after hierarchical suppression and deferred release control is verified to confirm that the target candidate time window has not been reconnected by the interference event chain during the execution period, and the opportunity occupancy result, interference event chain suppression result and deferred release result corresponding to the target candidate time window are recorded.

[0042] Optionally, the step of delaying the calculation of the target control action and updating the reinforcement learning model based on the realization results of the corresponding candidate time windows in the power opportunity graph includes:

[0043] Within the observation time window after the target control action is executed, the actual operation feedback data of the mobile terminal is collected. The actual operation feedback data includes actual energy consumption data, target power consumption state dwell time, system wake-up times, task scheduling execution status, network communication behavior, and user interaction response data. The actual operation feedback data is time-aligned with the target candidate time window.

[0044] Based on the power opportunity diagram, the fulfillment status of the target candidate time window is determined. The fulfillment status includes whether the target candidate time window is fully occupied, whether the interrupted event chain is reconnected, and the actual duration of the target candidate time window, and the corresponding window fulfillment result is generated.

[0045] Based on the window fulfillment results and actual operation feedback data, a delayed settlement evaluation result is constructed. Based on the delayed settlement evaluation result, the execution result corresponding to the target control action is matched with the prediction result of the candidate control action in the counterfactual three-branch evaluator to determine the actual benefit deviation of the target control action and generate feedback information for updating the reinforcement learning model.

[0046] Feedback information is input into the reinforcement learning model to update the policy network and value network in the model. This allows the reinforcement learning model to adjust the selection strategy of candidate control actions based on the delayed settlement evaluation results, thereby completing the delayed settlement update based on the candidate time window realization results.

[0047] The beneficial effects of this invention are:

[0048] This invention addresses the common problems in existing mobile terminal power management, such as coarse-grained control, insufficient understanding of the correlation between multiple operating events, and difficulty in maintaining a stable low-power state. By introducing a power opportunity graph and interference event chain modeling mechanism, it achieves accurate identification and representation of potential energy-saving opportunities during system operation. Compared to traditional schemes that control based solely on instantaneous load or a single threshold, this invention can analyze the impact of various events on power consumption from a temporal and causal perspective, effectively avoiding frequent interruptions to energy-saving opportunities caused by scattered interference, and improving the continuous persistence of the target power consumption state.

[0049] This invention constructs a counterfactual three-branch evaluator and combines it with a reinforcement learning decision framework to perform multi-path prediction and long-term benefit evaluation of candidate control actions. This enables the system to make better choices among immediate execution, deferred execution, and no execution, effectively reducing the frequent adjustments and power consumption bounce problems caused by short-sighted decision-making in traditional strategies. This makes the power consumption control process more stable and coherent, while also taking into account the performance requirements of foreground applications and avoiding significant interference with the user experience.

[0050] This invention achieves system-level overall energy efficiency optimization through the coordinated control of multiple resources, including the processor, graphics processor, display module, network communication, and background tasks. In practical applications, it not only reduces the average power consumption of the entire machine and extends the device's battery life, but also reduces resource waste caused by invalid wake-ups and redundant calculations, improving the smoothness and consistency of system operation. It has good engineering feasibility and application value. Attached Figure Description

[0051] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0052] Figure 1 This is a flowchart of a mobile terminal power consumption optimization method based on reinforcement learning proposed in this invention;

[0053] Figure 2 This is a schematic diagram of the counterfactual three-branch evaluator for a mobile terminal power consumption optimization method based on reinforcement learning proposed in this invention. Detailed Implementation

[0054] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0055] refer to Figure 1 and Figure 2 A mobile terminal power consumption optimization method based on reinforcement learning, comprising:

[0056] Collect the operating status data of the mobile terminal, process the operating status data, and form a status sequence;

[0057] Within a preset time window, multi-source running events in the state sequence are correlated and merged to identify candidate time windows that can enter the target power consumption state, a power consumption opportunity map is constructed, and the correlation between each candidate time window and the corresponding interference event chain are marked.

[0058] Candidate control actions are generated based on the power opportunity diagram. The running state data, power opportunity diagram and candidate control actions are input into the counterfactual three-branch evaluator to predict the state evolution results of the candidate control actions under the following conditions: immediate execution, execution after the end of the target candidate time window, and not execution while maintaining the current scheduling state. The net benefit of the candidate control actions is then determined.

[0059] The power opportunity budget for the future time range is determined based on the power opportunity map. The power opportunity budget and net revenue are used as inputs, and a reinforcement learning model is used to make decisions on the set of candidate control actions to obtain the target control action.

[0060] Based on the target control action, opportunity occupancy control is performed on the target candidate time window in the power consumption opportunity diagram. Hierarchical suppression and deferred release control are performed on the interference event chain that disrupts the target candidate time window, so as to maintain the target power consumption state within the target candidate time window and release the deferred events after the window ends.

[0061] Within the observation time window after the execution of the target control action, actual operation feedback is obtained. Based on the realization results of the corresponding candidate time windows in the power opportunity map, the target control action is delayed and the reinforcement learning model is updated.

[0062] In this embodiment, the operating status data includes processor operating status data, graphics processing status data, display operating status data, wireless communication status data, system wake-up status data, task scheduling status data, and power status data.

[0063] In this embodiment, the process of processing the running status data to form a status sequence includes timestamp alignment, data cleaning, outlier removal and standardization of the running status data, and chronological arrangement according to a preset sampling period to generate a status sequence corresponding to a continuous time window.

[0064] In this embodiment, the construction of the power consumption opportunity graph and the marking of the correlation between each candidate time window and the corresponding interference event chain include:

[0065] Within a preset time window, the state sequence of the mobile terminal is scanned in segments, and the running events in each time period are extracted. The events are then uniformly identified according to their occurrence time, duration, triggering source, belonging task object, and resource object, forming an event set.

[0066] The event set is continuously merged and causal relationship analyzed. Events that occur continuously in time, are mutually transmitted in triggering relationship, and are coupled in resource occupation are grouped into the same event segment. Based on the occupation relationship of each event segment on the processor, graphics processor, display module, wireless communication module and sensor module, candidate time windows for characterizing the target power consumption state can be entered.

[0067] Using candidate time windows as graph nodes and the connection, splicing, and blocking relationships between adjacent candidate time windows as graph edges, a power opportunity graph is constructed. The connection relationship is used to characterize the continuous connection relationship between adjacent candidate time windows in time order. The splicing relationship is used to characterize the continuous target power state residence interval that can be formed after multiple discrete candidate time windows are scheduled and rearranged. The blocking relationship is used to characterize the disruptive effect of event segments on the continuity of candidate time windows.

[0068] For each candidate time window, the running events associated with the candidate time window are traced back along the time sequence and the trigger transmission sequence to determine the source event, transmission event and maintenance event that caused the candidate time window to be shortened, interrupted or failed. The source event, transmission event and maintenance event are then linked together according to their sequential relationship to obtain the corresponding interference event chain.

[0069] The candidate time windows, their connections, splicing relationships, and blocking relationships, as well as the interference event chains corresponding to each candidate time window, are associated and registered to generate a power consumption opportunity map.

[0070] In this embodiment, determining the net benefit of the candidate control action includes:

[0071] Candidate time windows and corresponding interference event chains are extracted based on power consumption opportunity diagrams, and candidate control actions are generated by combining operating status data. Each candidate control action is bound to the corresponding target candidate time window and interference event chain to form an evaluation object for candidate time window protection.

[0072] A counterfactual three-branch evaluator is constructed for each evaluation object. The counterfactual three-branch evaluator includes a state mirroring unit, a window constraint injection unit, and a branch generation unit. The state mirroring unit copies the current running state data and the current power opportunity graph to form a unified baseline evaluation state. The window constraint injection unit writes the start and end boundaries of the target candidate time window, the positional relationship of the corresponding interference event chain, and the action time of the candidate control action into the baseline evaluation state. The branch generation unit generates an immediate execution branch, a window deferred branch, and a hold branch under the same baseline evaluation state.

[0073] A unified benchmark assessment status is established, specifically as follows:

[0074] The current running status data is completely copied, retaining the processor status, graphics processing status, display status, wireless communication status, system wake-up status, task scheduling status, and power status. The candidate time windows, the relationships between candidate time windows, and the corresponding interference event chains in the current power opportunity diagram are copied synchronously, so that each branch can perform state evolution deduction under the same initial state, the same candidate time window distribution, and the same interference event chain.

[0075] The branch generation unit generates immediate execution branches, window-deferred branches, and hold branches under the same baseline evaluation state, specifically as follows:

[0076] The baseline evaluation state is copied into three identical initial evaluation copies. In the first initial evaluation copy, the execution time of the candidate control action is written into the current time, generating an immediate execution branch.

[0077] In the second initial evaluation copy, the execution time of the candidate control action is written into the end time of the target candidate time window, and a window deferred branch is generated;

[0078] In the third initial evaluation copy, the execution flag of the candidate control action is masked and the current scheduling state is kept unchanged. A hold branch is generated. The immediate execution branch, the window deferred branch and the hold branch share the same initial value of the running state data, the same initial value of the power opportunity graph and the same initial value of the disturbance event chain.

[0079] The state evolution process of candidate control actions after execution at the current moment is deduced using the immediate execution branch; the state evolution process of candidate control actions after execution at the end of the target candidate time window is deduced using the window deferral branch; and the state evolution process of candidate control actions when they are not executed is deduced using the hold branch. This yields the power consumption state changes, candidate time window continuity changes, interference event chain propagation changes, and terminal operation response changes corresponding to each branch.

[0080] The state evolution process after the candidate control action is executed at the current moment is deduced using the immediate execution branch, specifically as follows:

[0081] At the current moment, the candidate control actions corresponding to the target candidate time window are executed and activated. Based on the running status data, power opportunity diagram and interference event chain in the immediate execution branch, the changes in the processor state, graphics processing state, display state, wireless communication state, task scheduling state and system wake-up state after the candidate control action is executed are continuously deduced. Whether the target candidate time window is shortened, interrupted, extended or maintained, and whether the corresponding interference event chain is truncated, weakened, transferred or reconnected.

[0082] The state evolution process when candidate control actions are deferred until the end of the target candidate time window is simulated using window deferred branches. Specifically:

[0083] Within the target candidate time window, the current scheduling state remains unchanged, and the interference event chain corresponding to the target candidate time window is continuously simulated until the end of the target candidate time window. Then, the execution flag of the candidate control action is activated. Based on the running state data, power opportunity diagram and interference event chain in the window deferred branch, the changes in processor state, graphics processing state, display state, wireless communication state, task scheduling state and system wake-up state caused by the deferred execution of the candidate control action are continuously simulated. The maintenance status of the target candidate time window before the deferred execution and the changes in event release and propagation after the deferred execution are recorded.

[0084] The state evolution process when candidate control actions are not executed is deduced using the hold branch method, specifically as follows:

[0085] Maintaining the current scheduling state and the current interference event chain propagation state unchanged, without activating the execution flag of candidate control actions, based on the running state data, power opportunity map and interference event chain in the maintained branch, continuously deduce the continuous change process of the target candidate time window under natural operating conditions, record the natural propagation, superposition, duration and decay of the corresponding interference event chain, as well as the changes in processor state, graphics processing state, display state, wireless communication state, task scheduling state and system wake-up state caused by it;

[0086] The deduction results of each branch are compared to determine the impact of candidate control actions on the target candidate time window when executed immediately, on the protection of the target candidate time window when executed deferred, and on the preservation of the natural propagation of the interference event chain when not executed. Based on the occupancy, protection, and preservation effects, corresponding branch evaluation results are generated. Specifically, the generation of corresponding branch evaluation results based on the occupancy, protection, and preservation effects is as follows:

[0087] Based on the continuous changes of the target candidate time window, the maintenance of the target power consumption state, the truncation of the interference event chain, and the changes in the terminal operation response in the immediate execution branch, the immediate execution branch evaluation results corresponding to the candidate control action are generated. Based on the complete maintenance of the target candidate time window, the continuation of the target power consumption state, the delayed release of the interference event chain, and the changes in the terminal operation response in the window deferred branch, the window deferred branch evaluation results corresponding to the candidate control action are generated. Based on the natural changes of the target candidate time window, the natural propagation of the interference event chain, the degradation of the target power consumption state, and the changes in the terminal operation response in the hold branch, the hold branch evaluation results corresponding to the candidate control action are generated. The immediate execution branch evaluation results, the window deferred branch evaluation results, and the hold branch evaluation results are uniformly correlated to form a branch evaluation result set used to characterize the differences in the execution effect of the candidate control action.

[0088] Occupancy impact refers to the actual occupancy result of the candidate control action on the target candidate time window when the candidate control action is executed in real time. Specifically, it reflects the changes caused by the candidate control action after intervention at the current moment to the continuity of the target candidate time window, the maintenance of the target power consumption state, and the propagation state of the corresponding interference event chain.

[0089] The protection effect refers to the protection result of the target candidate time window when the candidate control action is delayed until after the target candidate time window ends. Specifically, it reflects the impact of the candidate control action not being executed within the target candidate time window on the integrity of the target candidate time window, the continuation of the target power consumption state, and the delayed release of the interference event chain.

[0090] The preservation effect refers to the preservation results of the natural propagation of the interference event chain and the natural evolution of the target candidate time window when the candidate control action is not executed. Specifically, it reflects the effects of the continuous change of the target candidate time window, the degradation of the target power consumption state, and the natural propagation state of the interference event chain while maintaining the current scheduling state unchanged.

[0091] Based on the branch evaluation results of each branch, the net profit corresponding to each candidate control action is determined, specifically as follows:

[0092] The evaluation results of the immediate execution branch, the window-deferred branch, and the hold branch corresponding to the same candidate control action are compared. The evaluation values ​​corresponding to the continuous change of the target candidate time window, the change of the target power consumption state, the change of the disturbance of the interference event chain, and the change of the terminal operation response are extracted. With the hold branch evaluation result as the benchmark, the benefit difference between the immediate execution branch and the hold branch and the window-deferred branch and the hold branch are determined respectively. Then, the net benefit of the candidate control action under the immediate execution mode and the deferred execution mode is determined according to the benefit difference. The net benefit under the immediate execution mode and the net benefit under the deferred execution mode are output accordingly.

[0093] In this embodiment, the step of using a reinforcement learning model to make decisions on the set of candidate control actions to obtain the target control action includes:

[0094] Based on the power opportunity graph, target candidate time windows, the correlation between candidate time windows, and the interference event chain corresponding to each candidate time window are extracted. Combined with the net benefit of candidate control actions, decision input data for reinforcement learning model is generated. The decision input data includes the current running state, the current power opportunity graph, the net benefit of candidate control actions, and the boundary information of the target candidate time window.

[0095] A proximal policy optimization reinforcement learning model is used to make decisions on a set of candidate control actions. A policy network and a value network corresponding to the proximal policy optimization reinforcement learning model are constructed. The policy network outputs the selection result of each candidate control action based on the decision input data, and the value network outputs the state value corresponding to the current decision input data.

[0096] The target candidate time window in the power opportunity graph is written into the policy network to form a window-constrained decision state. Candidate control actions are categorized and encoded as immediate execution actions, window-deferred actions, and hold actions, and then input into the policy network. The policy network outputs the selection result of the candidate control action based on the boundary of the target candidate time window, the net benefit corresponding to the candidate control action, and the event chain perturbation evaluation result. Specifically, the window-constrained decision state is formed as follows:

[0097] Extract the start boundary, end boundary, window duration interval, correlation with adjacent candidate time windows, and positional relationship of the corresponding interference event chain of the target candidate time window from the power opportunity map. Then, uniformly associate and encode the start boundary, end boundary, window duration interval, correlation, and positional relationship with the current running state data and write them into the input state of the policy network to form a window constraint decision state for constraining the timing, scope, and target of candidate control actions.

[0098] The selection results output by the policy network are screened for window consistency. Candidate control actions that conflict with the start and end boundaries of the target candidate time window, conflict with the propagation direction of the interference event chain, or overlap with the already occupied candidate time window are eliminated. The screened candidate control actions are then input into the value network for state value evaluation to obtain the target control action.

[0099] Record the target candidate time window, action execution time, action category, and window occupancy result corresponding to the target control action.

[0100] In this embodiment, the step of performing hierarchical suppression and delayed release control on the interference event chain that disrupts the target candidate time window, so as to maintain the target power consumption state within the target candidate time window and release the delayed events after the window ends, includes:

[0101] Based on the target control action, the corresponding target candidate time window and associated interference event chain are determined from the power opportunity diagram. The start and end boundaries of the target candidate time window, the triggering order of the interference event chain, and the execution time of the target control action are written into the execution control sequence.

[0102] The running events in the interference event chain are segmented into links to identify source triggering events, transmission events, and maintenance events. Based on the positional relationship between each running event and the target candidate time window, the type of resources occupied, and the impact on the continuity of the target candidate time window, corresponding suppression control flags and release control flags are generated for the running events.

[0103] Hierarchical suppression control is performed on the interference event chain based on suppression control flags:

[0104] Entry blocking control is performed on source-triggered events located before the boundary of the target candidate time window; propagation truncation control is performed on transit events located within the target candidate time window; duration compression control is performed on sustaining events that maintain the persistence of the interference event chain; and the run events after hierarchical suppression are written into the deferred event queue, wherein:

[0105] Entry blocking control is performed on source-triggered events located before the boundary of the target candidate time window, specifically as follows:

[0106] Identify source triggering events that are triggered before the start boundary of the target candidate time window and can cause the interference event chain to enter the target candidate time window. Apply a blocking flag to the triggering entry corresponding to the source triggering event to prevent the source triggering event from continuing to trigger the corresponding task scheduling, network communication, sensor sampling or rendering drive events within the target candidate time window. Write the blocked source triggering event into the deferred event queue.

[0107] Perform propagation truncation control on transit events located within the target candidate time window, specifically as follows:

[0108] Identify the propagating events that continue to propagate along the interference event chain within the target candidate time window, apply a truncation mark to the subsequent triggering relationship corresponding to the propagating events, terminate the continued propagation of the propagating events to downstream running events, and retain the state record of the propagating events at the current node to prevent the interference event chain from continuing to extend within the target candidate time window.

[0109] Duration compression control is applied to sustaining events that maintain the persistence of the disturbance event chain, specifically as follows:

[0110] Identify maintenance events used to keep the interference event chain active, apply compression control to the resource occupation duration, maintenance duration or active duration range corresponding to the maintenance event, limit the continuous action time of the maintenance event to within the preset compression duration range, and release its continuous occupation of the processor, graphics processor, wireless communication module, display module or sensor module after the preset compression duration is reached.

[0111] Deferred release control is executed after the window in the deferred event queue based on the release control flag:

[0112] The release sequence is reconstructed according to the original triggering order and resource occupancy order of the interference event chain. After the target candidate time window ends, the deferred running events are released sequentially. During the release process, adjacent running events of the same type are subject to merged release control to form a centralized release sequence after the window. Specifically, the merged release control for adjacent running events of the same type during the release process is as follows:

[0113] The system identifies running events in the release sequence that are identical in event type, resource-occupying object, and adjacent within a preset release time interval. These adjacent running events of the same type are merged into the same release batch and triggered all at once at a unified release time. Specifically, background task running events are merged and released according to task scheduling objects, network communication running events are merged and released according to communication request objects, sensor sampling running events are merged and released according to sampling source and sampling period, and rendering driver running events are merged and released according to display refresh boundaries. This reduces repeated wake-ups and resource switching caused by discrete releases after the window ends.

[0114] Within the target candidate time window, the integrity of the execution window of the running event after hierarchical suppression and deferred release control is verified to confirm that the target candidate time window has not been reconnected by the interference event chain during the execution period, and the opportunity occupancy result, interference event chain suppression result and deferred release result corresponding to the target candidate time window are recorded.

[0115] In this embodiment, the step of delaying the calculation of the target control action and updating the reinforcement learning model based on the realization results of the corresponding candidate time windows in the power opportunity graph includes:

[0116] Within the observation time window after the target control action is executed, the actual operation feedback data of the mobile terminal is collected. The actual operation feedback data includes actual energy consumption data, target power consumption state dwell time, system wake-up times, task scheduling execution status, network communication behavior, and user interaction response data. The actual operation feedback data is time-aligned with the target candidate time window.

[0117] Based on the power opportunity diagram, the fulfillment status of the target candidate time window is determined. The fulfillment status includes whether the target candidate time window is fully occupied, whether the interrupted event chain is reconnected, and the actual duration of the target candidate time window, and the corresponding window fulfillment result is generated.

[0118] Based on the window fulfillment results and actual operational feedback data, a delayed settlement evaluation result is constructed. Based on this evaluation result, the execution result corresponding to the target control action is matched with the prediction result of the candidate control action in the counterfactual three-branch evaluator to determine the actual benefit deviation of the target control action and generate feedback information for updating the reinforcement learning model. Specifically, the construction of the delayed settlement evaluation result involves:

[0119] The actual occupancy, duration, and suppression of interference event chains of the target candidate time window in the window fulfillment results are extracted. Correlation analysis is performed on the actual energy consumption changes, target power consumption status maintenance, system wake-up changes, task scheduling changes, network communication changes, and terminal operation response changes in the actual operation feedback data to form the actual execution evaluation results corresponding to the target control action. The actual execution evaluation results are then uniformly correlated with the target candidate time window, the corresponding interference event chain, and the execution time of the target control action to generate a delay settlement evaluation result that characterizes the impact of the target control action on the fulfillment effect of the target candidate time window, the control effect of the interference event chain, and the terminal operation status after actual execution.

[0120] Feedback information is input into the reinforcement learning model to update the policy network and value network in the model. This allows the reinforcement learning model to adjust the selection strategy of candidate control actions based on the delayed settlement evaluation results, thereby completing the delayed settlement update based on the candidate time window realization results.

[0121] Example 1: To verify the feasibility of this invention in practice, it was applied to a smartphone. The test terminal was equipped with an octa-core processor (maximum clock speed of 2.8GHz), an integrated graphics processor, a 120Hz adaptive refresh rate display, and a battery capacity of approximately 5000mAh. It supported 5G network communication and Wi-Fi connectivity. The test was conducted daily from 19:00 to 21:00. This time period represents a typical high-frequency usage period for users and can realistically reflect the power consumption performance of the mobile terminal under complex application loads.

[0122] In this scenario, the terminal simultaneously runs a short video app, an instant messaging app, an e-commerce app, and background music playback, with Bluetooth device connection, location services, and message synchronization enabled. An automated script simulates user behaviors such as continuously swiping short videos, intermittently replying to messages, frequently switching between apps, and updating background data. During this process, it is clearly observed that even when the processor load is at a low to medium level, the system frequently experiences events such as network packet transmissions, background task wake-ups, and sensor triggers. These discrete events continuously interrupt the continuous time window for the system to enter a low-power state, resulting in unsatisfactory power optimization. Traditional threshold-based energy-saving strategies typically only reduce frequency or limit power consumption when the load increases, lacking the ability to identify and handle these fine-grained interference behaviors, thus easily leading to repeated power consumption fluctuations.

[0123] In this embodiment, when applying the method of the present invention, firstly, multi-source state data during the operation of the terminal is collected, including CPU and GPU load, display refresh status, network communication behavior, system wake-up times, and background task scheduling information, and a continuous state sequence is constructed. Multi-source events are correlated and analyzed through a sliding time window to construct a power opportunity map, thereby identifying multiple potential target power state candidate time windows. Interference events that disrupt these time windows are chain-tracked to form an interference event chain.

[0124] Subsequently, the runtime status data, power opportunity map, and candidate control actions are input into a counterfactual three-branch evaluator to predict the effects of different control strategies (immediate execution, delayed execution, and no execution). The reinforcement learning model combines the evaluation results to select the optimal control action from a long-term benefit perspective. During the execution phase, opportunity occupancy control is applied to candidate time windows, while non-critical tasks in the interference event chain are deferred. For example, background synchronization tasks are postponed, network requests are merged, and system wake-ups are time-aligned to reduce interference with low-power states. After the window ends, related tasks are gradually released to avoid performance fluctuations caused by concentrated execution.

[0125] Through the above methods, this embodiment achieves more stable power consumption optimization while ensuring that the user experience is largely unaffected. Compared with traditional strategies, this invention can more effectively extend the duration of low-power states and reduce energy waste caused by invalid wake-ups and repeated scheduling. Due to the introduction of counterfactual evaluation and reinforcement learning mechanisms, the system can gradually optimize the control strategy after multiple rounds of operation, demonstrating good adaptive capabilities. To verify the actual effect of this invention, traditional power-saving strategies and the method of this invention are compared under the same test conditions.

[0126] Table 1. Comparison of power consumption optimization effects in typical mobile terminal usage scenarios.

[0127] Indicator Items Test conditions Traditional strategy Method of the present invention Differences in performance illustrate Data source Average power consumption of the whole machine 2 hours of continuous use 3.72 W 3.35 W A decrease of approximately 9.9%. This is within the normal optimization range. Laboratory measurements Power consumption 2-hour power consumption ratio 14.8% 13.2% Down 1.6% In line with daily experience Battery statistics System wake-up count Hourly average 365 times 298 times A decrease of approximately 18% Reduce fragmented wake-up System Log Number of background task executions Hourly average 11.2 times 8.7 times A decrease of approximately 22% More focused on tasks Scheduling records Low power dwell time Cumulative duration 51 minutes 63 minutes An increase of approximately 23% Make fuller use of windows Power consumption analysis tools Page response delay average value 108 ms 113 ms Slight increase Still within acceptable range UI Testing Video playback frame rate Average frame rate 59.2 fps 58.7 fps Basically stable The user has no obvious perception. Rendering statistics

[0128] As shown in Table 1, the method of this invention achieves a relatively stable and reasonable improvement in energy efficiency under typical high-frequency usage scenarios in terms of overall power consumption. The average power consumption of the entire device decreased from 3.72W to 3.35W, a reduction of nearly 10%, while the corresponding two-hour power consumption decreased from 14.8% to 13.2%. This reduction is within a relatively ideal optimization range without sacrificing performance, indicating that this invention, through power opportunity graphs and reinforcement learning decision-making mechanisms, can effectively reduce ineffective power consumption, enabling the terminal to maintain a higher level of energy efficiency in complex usage environments, rather than relying on traditional methods such as simple frequency reduction.

[0129] From the perspective of system operation behavior, this invention is particularly effective in reducing invalid system activity. The number of system wake-ups decreased from 365 per hour to 298, and the number of background task executions decreased from 11.2 to 8.7, representing reductions of approximately 18% and 22%, respectively. This indicates that by identifying and delaying and aggregating interfering event chains, this invention effectively reduces fragmented wake-ups and scattered execution, making system operation more focused and orderly. The low-power state dwell time increased from 51 minutes to 63 minutes, an increase of approximately 23%, further verifying that this invention can successfully identify and maintain a stable power consumption window, extending the duration of the terminal in power-saving mode.

[0130] In terms of user experience, the impact of this invention on performance is limited and within an acceptable range. Page response latency only slightly increases from 108ms to 113ms, a minor change that will not be noticeable to the user. Video playback frame rate decreases slightly from 59.2fps to 58.7fps, but remains generally smooth. This demonstrates that while optimizing power consumption, this invention does not sacrifice foreground performance. Instead, it achieves a balance between power consumption and performance through a more refined scheduling strategy, showcasing good engineering practicality and application value.

[0131] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for optimizing power consumption in mobile terminals based on reinforcement learning, characterized in that, include: Collect the operating status data of the mobile terminal, process the operating status data, and form a status sequence; Within a preset time window, multi-source running events in the state sequence are correlated and merged to identify candidate time windows that can enter the target power consumption state, a power consumption opportunity map is constructed, and the correlation between each candidate time window and the corresponding interference event chain are marked. Candidate control actions are generated based on the power opportunity diagram. The running state data, power opportunity diagram and candidate control actions are input into the counterfactual three-branch evaluator to predict the state evolution results of the candidate control actions under the following conditions: immediate execution, execution after the end of the target candidate time window, and not execution while maintaining the current scheduling state. The net benefit of the candidate control actions is then determined. The power opportunity budget for the future time range is determined based on the power opportunity map. The power opportunity budget and net revenue are used as inputs, and a reinforcement learning model is used to make decisions on the set of candidate control actions to obtain the target control action. Based on the target control action, opportunity occupancy control is performed on the target candidate time window in the power consumption opportunity diagram. Hierarchical suppression and deferred release control are performed on the interference event chain that disrupts the target candidate time window, so as to maintain the target power consumption state within the target candidate time window and release the deferred events after the window ends. Within the observation time window after the execution of the target control action, actual operation feedback is obtained. Based on the realization results of the corresponding candidate time windows in the power opportunity map, the target control action is delayed and the reinforcement learning model is updated.

2. The mobile terminal power consumption optimization method based on reinforcement learning according to claim 1, characterized in that, The operational status data includes processor operational status data, graphics processing status data, display operational status data, wireless communication status data, system wake-up status data, task scheduling status data, and power status data.

3. The mobile terminal power consumption optimization method based on reinforcement learning according to claim 1, characterized in that, The process of processing the operational status data to form a status sequence includes timestamp alignment, data cleaning, outlier removal, and standardization of the operational status data, arranging the data in a temporal sequence according to a preset sampling period, and generating a status sequence corresponding to a continuous time window.

4. The mobile terminal power consumption optimization method based on reinforcement learning according to claim 1, characterized in that, The construction of the power consumption opportunity graph, and the marking of the correlation between each candidate time window and the corresponding interference event chain, includes: Within a preset time window, the state sequence of the mobile terminal is scanned in segments, and the running events in each time period are extracted. The events are then uniformly identified according to their occurrence time, duration, triggering source, belonging task object, and resource object, forming an event set. The event set is continuously merged and causal relationship analyzed. Events that occur continuously in time, are mutually transmitted in triggering relationship, and are coupled in resource occupation are grouped into the same event segment. Based on the occupation relationship of each event segment on the processor, graphics processor, display module, wireless communication module and sensor module, candidate time windows for characterizing the target power consumption state can be entered. A power consumption opportunity graph is constructed using candidate time windows as graph nodes and the connection, splicing, and blocking relationships between adjacent candidate time windows as graph edges. For each candidate time window, the running events associated with the candidate time window are traced back along the time sequence and the trigger transmission sequence to determine the source event, transmission event and maintenance event that caused the candidate time window to be shortened, interrupted or failed. The source event, transmission event and maintenance event are then linked together according to their sequential relationship to obtain the corresponding interference event chain. The candidate time windows, their connections, splicing relationships, and blocking relationships, as well as the interference event chains corresponding to each candidate time window, are associated and registered to generate a power consumption opportunity map.

5. The mobile terminal power consumption optimization method based on reinforcement learning according to claim 1, characterized in that, The net benefit of determining the candidate control action includes: Candidate time windows and corresponding interference event chains are extracted based on power consumption opportunity diagrams, and candidate control actions are generated by combining operating status data. Each candidate control action is bound to the corresponding target candidate time window and interference event chain to form an evaluation object for candidate time window protection. For each evaluation object, a counterfactual three-branch evaluator is constructed. The counterfactual three-branch evaluator includes a state mirroring unit, a window constraint injection unit, and a branch generation unit. The state mirroring unit copies the current running state data and the current power opportunity map to form a unified baseline evaluation state. The window constraint injection unit writes the start and end boundaries of the target candidate time window, the positional relationship of the corresponding interference event chain, and the action time of the candidate control action into the baseline evaluation state. The branch generation unit generates an immediate execution branch, a window deferred branch, and a hold branch under the same baseline evaluation state. The state evolution process of candidate control actions after execution at the current moment is deduced by using the immediate execution branch, the state evolution process of candidate control actions after execution at the end of the target candidate time window is deduced by using the window deferral branch, and the state evolution process of candidate control actions when they are not executed is deduced by using the hold branch, so as to obtain the power consumption state change, candidate time window continuity change, interference event chain propagation change and terminal operation response change corresponding to each branch. The deduction results of each branch are compared to determine the impact of candidate control actions on the target candidate time window when executed immediately, the impact on the target candidate time window when executed deferred, and the impact on the natural propagation of the interference event chain when not executed. Based on the impact of occupation, protection and retention, the corresponding branch evaluation results are generated. Based on the branch evaluation results of each branch, determine the net benefit corresponding to each candidate control action.

6. The mobile terminal power consumption optimization method based on reinforcement learning according to claim 1, characterized in that, The process of using a reinforcement learning model to make decisions on a set of candidate control actions to obtain the target control action includes: Based on the power opportunity graph, target candidate time windows, the correlation between candidate time windows, and the interference event chain corresponding to each candidate time window are extracted. Combined with the net benefit of candidate control actions, decision input data for reinforcement learning model is generated. A proximal policy optimization reinforcement learning model is used to make decisions on a set of candidate control actions. A policy network and a value network corresponding to the proximal policy optimization reinforcement learning model are constructed. The policy network outputs the selection result of each candidate control action based on the decision input data, and the value network outputs the state value corresponding to the current decision input data. The target candidate time window in the power opportunity graph is written into the policy network to form a window-constrained decision state. The candidate control actions are classified and encoded according to immediate execution actions, window-deferred actions, and hold actions, and then input into the policy network. The policy network outputs the selection result of the candidate control action based on the boundary of the target candidate time window, the net benefit corresponding to the candidate control action, and the event chain disturbance evaluation result. The selection results output by the policy network are screened for window consistency. Candidate control actions that conflict with the start and end boundaries of the target candidate time window, conflict with the propagation direction of the interference event chain, or overlap with the already occupied candidate time window are eliminated. The screened candidate control actions are then input into the value network for state value evaluation to obtain the target control action. Record the target candidate time window, action execution time, action category, and window occupancy result corresponding to the target control action.

7. The mobile terminal power consumption optimization method based on reinforcement learning according to claim 1, characterized in that, The hierarchical suppression and delayed release control of the interference event chain that disrupts the target candidate time window, in order to maintain the target power consumption state within the target candidate time window and release the delayed events after the window ends, includes: Based on the target control action, the corresponding target candidate time window and associated interference event chain are determined from the power opportunity diagram. The start and end boundaries of the target candidate time window, the triggering order of the interference event chain, and the execution time of the target control action are written into the execution control sequence. The running events in the interference event chain are segmented into links to identify source triggering events, transmission events, and maintenance events. Based on the positional relationship between each running event and the target candidate time window, the type of resources occupied, and the impact on the continuity of the target candidate time window, corresponding suppression control flags and release control flags are generated for the running events. Hierarchical suppression control is performed on the interference event chain based on suppression control flags: Entry blocking control is performed on source-triggered events located before the boundary of the target candidate time window; propagation truncation control is performed on transit events located inside the target candidate time window; duration compression control is performed on maintenance events that maintain the existence of the interference event chain; and the running events after hierarchical suppression are written into the deferred event queue. Deferred release control is executed after the window in the deferred event queue based on the release control flag: The release sequence is reconstructed according to the original triggering order and resource occupation order of the interference event chain. After the target candidate time window ends, the deferred running events are released in sequence. During the release process, adjacent running events of the same type are merged and released to form a concentrated release sequence after the window. Within the target candidate time window, the integrity of the execution window of the running event after hierarchical suppression and deferred release control is verified to confirm that the target candidate time window has not been reconnected by the interference event chain during the execution period, and the opportunity occupancy result, interference event chain suppression result and deferred release result corresponding to the target candidate time window are recorded.

8. The mobile terminal power consumption optimization method based on reinforcement learning according to claim 1, characterized in that, The step of delaying and updating the reinforcement learning model based on the fulfillment results of the corresponding candidate time windows in the power opportunity graph includes: Within the observation time window after the target control action is executed, the actual operation feedback data of the mobile terminal is collected. The actual operation feedback data includes actual energy consumption data, target power consumption state dwell time, system wake-up times, task scheduling execution status, network communication behavior, and user interaction response data. The actual operation feedback data is time-aligned with the target candidate time window. Based on the power opportunity diagram, the fulfillment status of the target candidate time window is determined. The fulfillment status includes whether the target candidate time window is fully occupied, whether the interrupted event chain is reconnected, and the actual duration of the target candidate time window, and the corresponding window fulfillment result is generated. Based on the window fulfillment results and actual operation feedback data, a delayed settlement evaluation result is constructed. Based on the delayed settlement evaluation result, the execution result corresponding to the target control action is matched with the prediction result of the candidate control action in the counterfactual three-branch evaluator to determine the actual benefit deviation of the target control action and generate feedback information for updating the reinforcement learning model. Feedback information is input into the reinforcement learning model to update the policy network and value network in the model. This allows the reinforcement learning model to adjust the selection strategy of candidate control actions based on the delayed settlement evaluation results, thereby completing the delayed settlement update based on the candidate time window realization results.

Images