Power task processing method and device, computer device and readable storage medium

Abstract

The application relates to a power task processing method, which comprises the following steps: obtaining a task request, wherein the task request comprises task information of at least one power task; for each decision moment, constructing a task state space at the decision moment according to the task information of each power task at the decision moment and the task processing information of each power task at a previous decision moment; calling a target deep Q network to analyze the task state space to obtain a task action space at the decision moment; the task action space is represented by the working frequency of each processor core and the processor core allocated to the power task; calling the processor core to process the corresponding power task according to the working frequency of each processor core to obtain a task processing result at the decision moment; and counting the task processing result corresponding to each decision moment to obtain a processing result for the task request. The method can improve the edge computing efficiency for the power task.

Description

Technical Field

[0001] This application relates to the field of power Internet of Things (IoT) technology, and in particular to a power task processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of power Internet of Things (IoT) technology, a low-power, multi-power task parallel edge computing solution, Mobile Edge Computing (MEC), has emerged. This solution can reduce energy consumption by dynamically adjusting the processor's voltage and clock frequency through Dynamic Voltage and Frequency Scaling (DVFS) technology.

[0003] However, current power task processing methods using DVFS technology are mainly implemented in the operating system kernel, which cannot be aware of power applications at the upper layer of the operating system. Therefore, it is impossible to adaptively adjust the clock according to the power task requirements, resulting in uneven load among multiple cores and mismatch between clock frequency adjustment and power task requirements. As a result, the problems of low efficiency and high energy consumption of edge computing still exist. Summary of the Invention

[0004] Therefore, it is necessary to provide a power task processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the edge computing efficiency for power tasks and reduce processor power consumption in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for processing power tasks, including:

[0006] Obtain a task request, the task request including task information for at least one power task;

[0007] For each decision moment, a task state space is constructed based on the task information of each power task at the decision moment and the task processing information of each power task at the previous decision moment.

[0008] The task state space is analyzed by calling the target depth Q network to obtain the task action space at the decision moment; the task action space is represented by the operating frequency of each processor core and the processor cores allocated for the power task.

[0009] According to the operating frequency of each processor core, the processor core is called to process the corresponding power task to obtain the task processing result at the decision time.

[0010] The processing results of the tasks at each decision point are statistically analyzed to obtain the processing result for the task request.

[0011] In one embodiment, the task information includes the maximum task delay and the task size, and the task processing information includes the task delay, the remaining task capacity, the scheduling indication parameters, and the processor core operating frequency.

[0012] The step of constructing the task state space at the decision time based on the task information of each power task at the decision time and the task processing information of each power task at the previous decision time includes:

[0013] For each power task at the decision time, the current task delay of the power task at the decision time is determined based on the historical task delay, historical task remaining amount, historical scheduling indication parameters and processor core historical operating frequency of the power task at the previous decision time.

[0014] Based on the task size of the power task, the historical scheduling indication parameters, the historical operating frequency of the processor core, the historical task delay, and the current task delay, the remaining amount of the current task of the power task at the decision time is determined;

[0015] Based on the relationship between the current task delay and the maximum task delay, and the remaining amount of the current task, the delay guarantee status of the power task at the decision time is determined;

[0016] Based on the task status, current task delay, delay guarantee status, current task remaining amount, and energy consumption of each processor core, the task state space at the decision moment is constructed.

[0017] In one embodiment, the training method of the target depth Q-network includes:

[0018] Obtain an initial depth Q-network; the initial depth Q-network includes an initial action network and an initial target network; the network parameters of the initial action network and the initial target network are the same;

[0019] For each historical decision moment, when the first historical state space at the historical decision moment is input into the initial action network, a first historical action space corresponding to the first historical state space is determined; the first historical state space is constructed based on the historical task information of the historical power task corresponding to the historical decision moment and the historical processing information of the previous historical decision moment.

[0020] Determine the historical reward at the historical decision moment and the second historical state space at the next historical decision moment;

[0021] By combining the first historical state space, the first historical action space, the historical reward, and the second historical state space, an experience sample is obtained.

[0022] For at least one experience sample, with the second historical state space input into the initial target network, the sample target value corresponding to the experience sample is obtained based on the historical reward.

[0023] Based on the training loss determined for each of the target values ​​of the samples, the initial action network is updated to obtain the updated action network;

[0024] The updated target network is obtained by setting the network parameters of the initial target network to be consistent with the updated action network;

[0025] Based on the updated action network and the updated target network, the target depth Q network is obtained.

[0026] In one embodiment, determining the historical reward at the historical decision moment includes:

[0027] The historical task delay guarantee rate at the historical decision-making time is determined based on the historical delay guarantee status of each historical power task at the historical decision-making time and the total number of historical power tasks.

[0028] The historical processor energy consumption at the historical decision-making time is determined based on the historical energy consumption of each processor core at the historical decision-making time.

[0029] The energy-saving delay guarantee index at the historical decision moment is obtained by weighting the difference between the historical task delay guarantee rate and the historical processor energy consumption.

[0030] Based on the energy-saving delay guarantee index, the historical reward at the historical decision-making moment is determined.

[0031] In one embodiment, determining the first historical action space corresponding to the first historical state space when the first historical state space at the historical decision moment is input into the initial action network includes:

[0032] When the first historical state space at the historical decision moment is input into the initial action network, the first target value corresponding to each of the multiple candidate action spaces is obtained;

[0033] Based on each of the first target values ​​and the first constraints, a first historical action space corresponding to the first historical state space is determined from the plurality of candidate action spaces; the first constraint satisfies that after the processor core completes processing a historical power task, the processor core begins to process other historical power tasks.

[0034] In one embodiment, determining the first historical action space corresponding to the first historical state space from the plurality of candidate action spaces based on each of the first target values ​​and the first constraints includes:

[0035] Obtain the second constraint condition; the second constraint condition satisfies: the operating frequency of the processor core is less than or equal to the maximum operating frequency;

[0036] Based on each of the first target values, the first constraint, and the second constraint, a first historical action space corresponding to the first historical state space is determined from the plurality of candidate action spaces.

[0037] Secondly, this application also provides a power task processing device, the device comprising:

[0038] The acquisition module is used to acquire task requests, wherein the task requests include task information for at least one power task.

[0039] The construction module is used to construct the task state space at each decision time based on the task information of each power task at the decision time and the task processing information of each power task at the previous decision time.

[0040] The first processing module is used to call the target depth Q network to analyze the task state space and obtain the task action space at the decision moment; the task action space is represented by the operating frequency of each processor core and the processor cores allocated for the power task.

[0041] The second processing module is used to call the processor core to process the corresponding power task according to the operating frequency of each processor core, and obtain the task processing result at the decision time.

[0042] The third processing module is used to statistically analyze the task processing results corresponding to each of the decision times, and obtain the processing results for the task request.

[0043] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0044] Obtain a task request, the task request including task information for at least one power task;

[0045] For each decision moment, a task state space is constructed based on the task information of each power task at the decision moment and the task processing information of each power task at the previous decision moment.

[0046] The task state space is analyzed by calling the target depth Q network to obtain the task action space at the decision moment; the task action space is represented by the operating frequency of each processor core and the processor cores allocated for the power task.

[0047] According to the operating frequency of each processor core, the processor core is called to process the corresponding power task to obtain the task processing result at the decision time.

[0048] The processing results of the tasks at each decision point are statistically analyzed to obtain the processing result for the task request.

[0049] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0050] Obtain a task request, the task request including task information for at least one power task;

[0051] For each decision moment, a task state space is constructed based on the task information of each power task at the decision moment and the task processing information of each power task at the previous decision moment.

[0052] The task state space is analyzed by calling the target depth Q network to obtain the task action space at the decision moment; the task action space is represented by the operating frequency of each processor core and the processor cores allocated for the power task.

[0053] According to the operating frequency of each processor core, the processor core is called to process the corresponding power task to obtain the task processing result at the decision time.

[0054] The processing results of the tasks at each decision point are statistically analyzed to obtain the processing result for the task request.

[0055] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0056] Obtain a task request, the task request including task information for at least one power task;

[0057] For each decision moment, a task state space is constructed based on the task information of each power task at the decision moment and the task processing information of each power task at the previous decision moment.

[0058] The task state space is analyzed by calling the target depth Q network to obtain the task action space at the decision moment; the task action space is represented by the operating frequency of each processor core and the processor cores allocated for the power task.

[0059] According to the operating frequency of each processor core, the processor core is called to process the corresponding power task to obtain the task processing result at the decision time.

[0060] The processing results of the tasks at each decision point are statistically analyzed to obtain the processing result for the task request.

[0061] The aforementioned power task processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product, by acquiring task requests containing task information for at least one power task, can proactively perceive the specific task requirements of the upper-layer power scenario. This provides a precise data foundation for subsequent adaptive task scheduling and processor resource allocation, overcoming the shortcomings of existing DVFS solutions that cannot perceive the needs of the power application layer, leading to blind frequency adjustments and improving the accuracy of task scheduling. For each decision moment, a task state space is constructed based on the current task information and the task processing information from the previous decision moment, enabling real-time tracking of task processing progress. This provides accurate state input for task scheduling and processor resource allocation, avoiding scheduling deviations caused by missing information and further improving task scheduling accuracy. By calling a target depth Q-network to analyze the task state space, the output is a task action space jointly represented by the operating frequency of each processor core and task allocation. This ensures that frequency adjustments are precisely matched with the actual needs of the power task, effectively improving load balancing among processor cores and reducing processor power consumption. By calling the corresponding processor core to process the allocated power tasks according to the working frequency of each processor core determined by the decision, and statistically analyzing the task processing results at each decision time, a complete processing result for the task request is formed. In this way, while meeting the processing requirements of each power task, task computing efficiency can be improved, multi-core load balancing can be achieved, and processor power consumption can be reduced. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a flowchart illustrating a power task processing method in one embodiment;

[0064] Figure 2 This is a flowchart illustrating how the task state space at a decision time is constructed based on the task information of each power task at the decision time and the task processing information of each power task at the previous decision time, in one embodiment.

[0065] Figure 3 This is a flowchart illustrating the process of determining historical rewards at a historical decision point in one embodiment.

[0066] Figure 4 This is a structural block diagram of a 5G MEC device in one embodiment;

[0067] Figure 5 This is a flowchart illustrating the power task processing method in another embodiment;

[0068] Figure 6 This is a structural block diagram of a power task processing device in one embodiment;

[0069] Figure 7 This is an internal structural diagram of a computer device in one embodiment;

[0070] Figure 8 This is a diagram of the internal structure of a computer device in another embodiment. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0072] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various parameters, but these parameters are not limited by these terms. These terms are only used to distinguish the first parameter from the second parameter. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the solutions, or any combination of multiple solutions.

[0073] The power task processing method provided in this application can be applied to fifth-generation (5G) mobile edge computing devices (hereinafter referred to as power 5G MEC devices) in the power industry. Power 5G MEC devices include multi-core processors, memory, storage, and 5G communication units. The power 5G MEC device can be a terminal or a server. Terminals can be, but are not limited to, various personal computers, laptops, smartphones, tablets, etc. Servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud computing services.

[0074] In one exemplary embodiment, such as Figure 1 As shown, a power task processing method is provided. Taking the application of this method to a power 5G MEC device as an example, the method includes the following steps:

[0075] S102, Obtain a task request, the task request including task information for at least one power task.

[0076] The task information includes, but is not limited to, task size and maximum task delay. Task size refers to the total amount of data that the power task needs to process, and maximum task delay refers to the maximum time interval allowed from the initiation of the power task to its completion, i.e., the deadline for the task to be completed normally.

[0077] For example, multiple power tasks form a task set. Represents a set of power tasks. This indicates the size of the tasks corresponding to the task set. This represents the task size of the m-th power task. This indicates the maximum task latency corresponding to the task set. This represents the maximum task delay for the m-th power task, where M represents the total number of power tasks.

[0078] In an optional embodiment, the power 5G MEC device is configured with a 5G MEC platform, which includes an edge computing scheduling management unit and a power application management unit. Specifically, when the power application management unit receives multiple power tasks from its assigned power 5G private network via a 5G communication unit, the power application management unit sends task requests to the scheduling decision unit in the edge computing scheduling management unit. The task requests include task information for each of the multiple power tasks.

[0079] In an optional embodiment, the scheduling decision unit is configured to extract task information of multiple power tasks from the task request upon receiving a task request, in order to form a task set.

[0080] S104. For each decision moment, construct the task state space at the decision moment based on the task information of each power task at the decision moment and the task processing information of each power task at the previous decision moment.

[0081] Here, the previous decision moment refers to the moment when the task was last processed relative to the current decision moment. Task processing information refers to the progress of the power task at the previous decision moment and related dynamic status parameters. For example, task processing information includes, but is not limited to: historical task delays, historical task remaining capacity, historical scheduling indication parameters, and the historical operating frequency of the processor core.

[0082] S106, call the target depth Q network to analyze the task state space and obtain the task action space at the decision moment; the task action space is represented by the operating frequency of each processor core and the processor cores allocated for power tasks.

[0083] The target deep Q-network is a pre-trained deep Q-network that can be trained based on multiple historical power tasks.

[0084] For example, This represents the task action space at the t-th decision time. , T represents the current decision time, and T represents the maximum decision time. This represents the scheduling instruction parameter for the m-th task on the k-th processor core at the t-th decision time. When, it means that at the t-th decision time, the m-th power task is assigned to the k-th processor core; otherwise... . Let K represent the operating frequency of each processor core at decision time t, and K represent the total number of processor cores. In other words, the operating frequency of each processor core at decision time t, as well as the processor cores allocated for power tasks, can be determined through the task action space.

[0085] In an optional embodiment, by discretizing the processor core frequency into a finite number of frequency values, a target depth Q-network is invoked to analyze the task state space to obtain the task action space at the decision moment. In other words, the target operating frequency of the processor core in the task action space is essentially a frequency value selected from a finite number of frequency values. This reduces training complexity and improves training efficiency.

[0086] S108 calls the processor core to process the corresponding power task according to the operating frequency of each processor core, and obtains the task processing result at the decision moment.

[0087] In an optional embodiment, the edge computing scheduling management unit includes a scheduling decision unit, which is used to call the processor core to process the corresponding power task according to the operating frequency of each processor core, and obtain the task processing result at the decision time.

[0088] S110, Statistically analyze the task processing results corresponding to each decision time to obtain the processing results for the task request.

[0089] The method described in the above embodiments, by acquiring task requests containing task information for at least one power task, can proactively perceive the specific task requirements of the upper-layer power scenario. This provides a precise data foundation for subsequent adaptive task scheduling and processor resource allocation, overcoming the shortcomings of existing DVFS solutions that cannot perceive the needs of the power application layer, leading to blind frequency adjustments and improving the accuracy of task scheduling. For each decision moment, a task state space is constructed based on the current task information and the task processing information from the previous decision moment, enabling real-time tracking of task processing progress. This provides accurate state input for task scheduling and processor resource allocation, avoiding scheduling deviations caused by missing information and further improving task scheduling accuracy. By calling a target depth Q-network to analyze the task state space, the output is a task action space jointly represented by the operating frequency of each processor core and task allocation. This ensures that frequency adjustments are precisely matched with the actual needs of power tasks, effectively improving load balancing among processor cores and reducing processor power consumption. By calling the corresponding processor core to process the allocated power tasks according to the working frequency of each processor core determined by the decision, and statistically analyzing the task processing results at each decision time, a complete processing result for the task request is formed. In this way, while meeting the processing requirements of each power task, task computing efficiency can be improved, multi-core load balancing can be achieved, and processor power consumption can be reduced.

[0090] In one embodiment, the method further includes: periodically updating the target depth Q-network and periodically monitoring the processor's resource status and power task performance; when the processor's real-time power consumption exceeds a preset power consumption tolerance threshold, or the actual latency guarantee rate of the power task is lower than a preset latency guarantee rate tolerance threshold, returning to the step of constructing a task state space at the decision time based on the task information of each power task at the decision time and the task processing information of each power task at the previous decision time, so as to update the joint optimization strategy of power task allocation, task scheduling, and processor core operating frequency adjustment. Thus, by realizing adaptive response to abnormal states and dynamic strategy adjustment, the robustness and quality of service of power task processing can be improved.

[0091] In one embodiment, task information includes maximum task latency and task size, and task processing information includes historical task latency, historical task remaining capacity, historical scheduling indication parameters, and historical operating frequency of the processor core. For example... Figure 2 As shown, a flowchart illustrates a method for constructing a task state space at a decision-making time based on the task information of each power task at that decision-making time and the task processing information of each power task at the previous decision-making time. The method is illustrated using a power 5G MEC device as an example, and includes the following steps:

[0092] S1042, for each power task at the decision time, determine the current task delay of the power task at the decision time based on the historical task delay, historical task remaining amount, historical scheduling indication parameters and historical processor core operating frequency of the power task at the previous decision time.

[0093] For example, the current task delay of the m-th power task at the t-th decision time. satisfy:

[0094]

[0095] in, This indicates the historical task delay of the m-th power task at decision time t-1. This represents the remaining historical task quantity of the m-th power task at decision time t-1. This represents the historical scheduling indication parameters for the m-th power task at decision time t-1. This represents the historical operating frequency of the k-th processor core at decision time t-1. This indicates the time interval between adjacent decision moments. Describes the minimum value function. This indicates the delay increment.

[0096] S1044, based on the task size of the power task, historical scheduling indication parameters, historical operating frequency of the processor core, historical task delay and current task delay, determines the current task remaining amount of the power task at the decision time.

[0097] For example, the remaining amount of the m-th power task at the t-th decision time. satisfy:

[0098]

[0099] in, Represents the maximum value function. This represents the task size of the m-th power task.

[0100] S1046, Based on the relationship between the current task delay and the maximum task delay, and the remaining amount of the current task, determine the delay guarantee status of the power task at the decision time.

[0101] In an optional embodiment, if the remaining amount of the power task at the decision time is greater than 0, the delay guarantee status of the power task at the decision time is set to a first preset value, which indicates that the power task has not yet started or is still running. If the remaining amount of the power task at the decision time is 0, and the power task delay is less than or equal to the maximum task delay, the delay guarantee status of the power task at the decision time is set to a second preset value, which indicates that the power task has been completed and has not timed out. If the remaining amount of the power task at the decision time is 0, and the power task delay is greater than the maximum task delay, the delay guarantee status of the power task at the decision time is set to a third preset value, which indicates that the power task has been completed and has timed out. For example, the first preset value can be represented by 0, the second preset value can be represented by 1, and the third preset value can be represented by 2, or other representation methods are also possible.

[0102] S1048 constructs the task state space at the decision moment based on the current task status, current task delay, delay guarantee status, current task remaining amount, and the energy consumption of each processor core for each power task.

[0103] The current task status of the power task can be determined based on the current remaining amount of the power task.

[0104] In an optional embodiment, if the remaining amount of the current power task is not 0 and equal to the size of the power task, the task status of the power task can be set to a first preset value, indicating that the power task has not yet started execution; if the remaining amount of the current power task is not 0 and less than the size of the power task, the task status of the power task can be set to a second preset value, indicating that the power task is being executed; if the remaining amount of the current power task is 0, the task status of the power task can be set to a third preset value, indicating that the power task has been completed. For example, the first preset value can be represented by 0, the second preset value can be represented by 1, the third preset value can be represented by 2, or other representation methods are also possible.

[0105] For example, at each decision time t, the environmental state of the multi-task scheduling deep reinforcement learning agent can include the task running state, performance state, and processor core energy consumption state of the power task corresponding to decision time t, and the state space is used as follows: express.

[0106] in, This indicates the task status corresponding to the task set. This represents the task status of the m-th power task, where 0 indicates that the task has not yet started, 1 indicates that the task is running, and 2 indicates that the task has ended. This indicates the task delay for the task set. This represents the delay guarantee status of the task set corresponding to the t-th decision time, where, This indicates the delay guarantee status of the m-th power task. 1 indicates that the task has not yet started or is still running (not timed out), 1 indicates that the task has been completed and has not timed out, and 2 indicates that the task has been completed but has timed out. This represents the remaining amount of tasks in the task set at decision time t (i.e., the remaining computational load of the tasks). This indicates the individual power consumption of each processor core.

[0107] In the above embodiments, by integrating historical and current multi-dimensional task information and processor power consumption, a refined state space is constructed, which includes latency, remaining capacity, latency guarantee status, and processor core power consumption. This provides a complete and accurate state basis for subsequent scheduling decisions, thereby improving the matching degree between task scheduling and frequency adjustment. While improving the accuracy of task scheduling, it can also reduce processor power consumption.

[0108] In one embodiment, the training of a target depth Q-network includes the following steps:

[0109] S10, Obtain the initial depth Q-network; the initial depth Q-network includes the initial action network and the initial target network; the network parameters of the initial action network and the initial target network are the same.

[0110] For example, the network parameters of the initial action network in the initial Deep Q-Network (DQN) are: Then set the network parameters of the initial target network. .

[0111] S20, for each historical decision moment, with the first historical state space at the historical decision moment input into the initial action network, the first historical action space corresponding to the first historical state space is determined; the first historical state space is constructed based on the historical task information of the historical power task at the historical decision moment and the historical processing information of the previous historical decision moment.

[0112] In an optional embodiment, when the first historical state space at the historical decision-making moment is input into the initial action network, determining the first historical action space corresponding to the first historical state space includes: obtaining a first target value corresponding to each of multiple candidate action spaces when the first historical state space at the historical decision-making moment is input into the initial action network; and determining the candidate action space corresponding to the largest first target value among the multiple candidate action spaces as the first historical action space corresponding to the first historical state space based on the first target values ​​corresponding to the multiple candidate action spaces. Thus, at each historical decision-making moment, the initial action network evaluates the value of all candidate action spaces and selects the action space corresponding to the largest target value, achieving optimal action decision based on the current state and improving the real-time performance and rationality of task scheduling and processor core frequency allocation.

[0113] In an optional embodiment, when the first historical state space at the historical decision-making moment is input into the initial action network, determining the first historical action space corresponding to the first historical state space includes: when the first historical state space at the historical decision-making moment is input into the initial action network, obtaining the first target value corresponding to each of the multiple candidate action spaces; and using... Based on the first target value corresponding to each of the multiple candidate action spaces, the first historical action space corresponding to the first historical state space is determined. This enhances the adaptability of deep Q-networks to dynamic power task environments and helps improve decision-making accuracy.

[0114] S30 determines the historical reward at the historical decision moment and the second historical state space at the next historical decision moment.

[0115] The construction methods of the first historical action space and the second historical state space can refer to the relevant content of this application for the construction of the task action space.

[0116] In an optional embodiment, the edge computing scheduling management unit includes a scheduling decision unit, a resource monitoring unit, and a performance monitoring unit. The scheduling decision unit can obtain the second historical state space corresponding to the next historical decision moment through the resource monitoring unit, and the scheduling decision unit can obtain the historical reward at the historical decision moment through the performance monitoring unit.

[0117] S50 combines the first historical state space, the first historical action space, the historical reward space, and the second historical state space to obtain experience samples.

[0118] In an optional embodiment, the scheduling decision unit stores experience samples in an experience revisit pool. For example, an experience sample can be represented as... . Let j represent the first historical state space at the j-th historical decision moment. This represents the space of the first historical action at the j-th historical decision moment. This represents the historical reward at the j-th historical decision point. Let represent the second historical state space at the (j+1)th historical decision moment.

[0119] S50: For at least one experience sample, with the second historical state space input into the initial target network, the sample target value corresponding to the experience sample is obtained based on the historical reward.

[0120] In an optional embodiment, when the second historical state space is input into the initial target network, the sample target value corresponding to the experience sample is obtained based on the historical reward, including: when the second historical state space is input into the initial target network, obtaining the second target value corresponding to each of the multiple reference actions corresponding to the second historical action space; and obtaining the sample target value corresponding to the experience sample based on the maximum value among the multiple second target values ​​and the historical reward.

[0121] For example, the sample target value corresponding to the empirical sample. satisfy:

[0122]

[0123] in, Indicates the discount factor. Indicates a reference action.

[0124] S60, based on the training loss determined by the target value of each sample, updates the initial action network to obtain the updated action network.

[0125] The training loss can be determined using the mean squared error loss function, for example, the training loss. satisfy:

[0126]

[0127] in, Represents the historical state space in the training samples. Below, the current mobile network has the following network parameters. Under the mapping, for the historical action space executed The output is the expected value of the predicted future cumulative reward, which is used to evaluate the quality of the historical action space in the current state. This represents the total number of training samples.

[0128] S70: The updated target network is obtained by setting the network parameters of the initial target network to be consistent with those of the updated action network.

[0129] S80, based on updating the action network and updating the target network, yields the target depth Q-network.

[0130] In the above embodiments, by collecting experience samples based on the initial action network and calculating stable sample target values ​​using the initial target network, and updating the action network in combination with training loss, the action network parameters are periodically copied to the target network. Thus, this method can effectively improve the stability and convergence speed of the training process, enabling the deep Q network to learn better power task scheduling strategies, thereby reducing edge computing energy consumption and ensuring task processing efficiency.

[0131] When a target depth Q-network is trained using the methods described in the above embodiments or other methods, the task action space at the decision time can be obtained by calling the target depth Q-network to analyze the task state space. For example, when calling the target depth Q-network to analyze the task state space, ... Probabilistic selection of task action space .

[0132] In one embodiment, such as Figure 3 As shown, the historical rewards at the historical decision-making moment are determined, including:

[0133] S302, determine the historical task delay guarantee rate at the historical decision-making time based on the historical delay guarantee status of each historical power task at the historical decision-making time and the total number of historical power tasks.

[0134] Specifically, the historical task delay guarantee rate is determined based on the ratio of the number of tasks whose delays are guaranteed after executing the current decision to the total number of tasks. For example, the historical task delay guarantee rate at the j-th historical decision moment satisfies:

[0135]

[0136] in, For a binary function, when If true, Y=1; otherwise, Y=0. Let N represent the historical delay guarantee status of the i-th historical power task at the j-th historical decision time, and let N represent the total number of historical power tasks.

[0137] S304 determines the historical processor power consumption at the historical decision-making time based on the historical power consumption of each processor core at the historical decision-making time.

[0138] For example, the historical processor energy consumption at the j-th historical decision moment. satisfy:

[0139]

[0140] in, The historical energy consumption of the k-th processor core is represented by the following condition:

[0141]

[0142] in, Indicates the energy consumption factor. This represents the operating frequency of the k-th processor core at the j-th historical decision point. This indicates the time interval between adjacent historical decision moments.

[0143] It should be noted that the number of processor cores used when training the network can be the same as or different from the number of processor cores used to process the current task request.

[0144] S306 derives the energy-saving latency guarantee index at the historical decision point based on the weighted difference between the historical task latency guarantee rate and the historical processor power consumption.

[0145] For example, the energy-saving delay guarantee index at the j-th historical decision moment. satisfy:

[0146]

[0147] in, This represents the first weight corresponding to the historical mission delay guarantee rate. The second weight represents the historical processor power consumption, while both the first and second weights are greater than 0.

[0148] S308 determines the historical rewards at the historical decision-making moment based on the energy-saving delay guarantee index.

[0149] For example, the negative of the energy-saving delay guarantee index is determined as the historical reward at the historical decision point. It can be understood that the long-term optimization objective of the multi-task, multi-core collaborative scheduling deep reinforcement learning model can be defined as the sum of the instantaneous energy-saving delay guarantee indices, i.e., satisfying:

[0150]

[0151] Therefore, with maximizing the energy-saving delay guarantee index as the joint optimization objective, the reward function is set as the negative of the energy-saving delay guarantee index, i.e., the historical reward at the j-th historical decision time. satisfy:

[0152]

[0153] In the above embodiments, by calculating the historical task latency guarantee rate and historical processor power consumption respectively, and constructing an energy-saving latency guarantee index based on the weighted difference between the two, the historical reward is determined. Thus, processor power consumption is reduced while ensuring that power tasks are completed on time. This overcomes the shortcomings of traditional DVFS technology, which only adjusts the frequency at the kernel level and cannot take into account both task latency and power consumption optimization, thereby improving the overall performance of edge computing.

[0154] In one embodiment, when the first historical state space at the historical decision moment is input into the initial action network, determining the first historical action space corresponding to the first historical state space includes: when the first historical state space at the historical decision moment is input into the initial action network, obtaining a first target value corresponding to each of the multiple candidate action spaces; and determining the first historical action space corresponding to the first historical state space from the multiple candidate action spaces based on each first target value and a first constraint.

[0155] The first constraint condition is satisfied: after the processor core finishes processing a historical power task, the processor core begins processing other historical power tasks. That is, if power task m is at time... If it starts running on processor core k, it will continue until the task is completed (e.g., at time k). This task runs on this processor core, and this processor core is... Only this task will run during the period.

[0156] In an optional embodiment, based on each first target value and a first constraint condition, determining the first historical action space corresponding to the first historical state space from multiple candidate action spaces includes: if the candidate action space corresponding to the largest first target value satisfies the first constraint condition, determining the candidate action space corresponding to the largest first target value as the first historical action space corresponding to the first historical state space.

[0157] In the above embodiments, by introducing processor core occupancy constraints during the action selection process, it is ensured that each processor core will not receive a new task before completing the currently allocated historical power task. This avoids resource conflicts and frequent context switching during concurrent execution of multiple tasks, reduces the probability of task blocking and additional overhead, improves the stability of the task processing process, further guarantees the real-time requirements of power tasks, and reduces processor energy waste caused by task switching.

[0158] In one embodiment, determining the first historical action space corresponding to the first historical state space from multiple candidate action spaces based on each first target value and the first constraint includes: obtaining a second constraint; the second constraint satisfies that the processor core's operating frequency is less than or equal to the maximum operating frequency; and determining the first historical action space corresponding to the first historical state space from multiple candidate action spaces based on each first target value, the first constraint, and the second constraint.

[0159] In an optional embodiment, the scheduling decision unit can obtain the maximum operating frequency of each processor core through the resource monitoring unit. For example, This represents the maximum operating frequency of each processor core. Therefore, the second constraint condition is satisfied:

[0160]

[0161] in, This represents the operating frequency of the k-th processor core at the j-th historical decision point. This represents the maximum operating frequency of the k-th processor core.

[0162] In conjunction with the above, in one embodiment, such as Figure 4 The diagram illustrates the structure of a power 5G MEC device. The power 5G MEC device 40 includes a multi-core processor 401, memory 402, storage 403, and a 5G communication unit 404. The power 5G MEC device also includes a power 5G MEC platform 405, which comprises a power application management unit 4051 and an edge computing scheduling management unit 4052. The edge computing scheduling management unit 4052 includes a scheduling decision unit, a resource monitoring unit, and a performance monitoring unit.

[0163] exist Figure 4 Based on what is shown, as Figure 5 A power task processing method is provided, and the method is illustrated by taking its application to a power 5G MEC device as an example. The method includes the following steps:

[0164] S501, Obtain a task request, the task request including task information for at least one power task.

[0165] S502, for each power task at the decision time, determines the current task delay of the power task at the decision time based on the historical task delay, historical task remaining amount, historical scheduling indication parameters and processor core historical operating frequency of the power task at the previous decision time.

[0166] S503 determines the remaining amount of the current task at the decision time based on the task size of the power task, historical scheduling indication parameters, historical operating frequency of the processor core, historical task delay, and current task delay.

[0167] S504, based on the relationship between the current task delay and the maximum task delay, and the remaining amount of the current task, determine the delay guarantee status of the power task at the decision time.

[0168] S505 constructs the task state space at the decision moment based on the task status of each power task, the current task delay, the delay guarantee status, the current task remaining amount, and the power consumption of each processor core.

[0169] S506 calls the target depth Q network to analyze the task state space and obtain the task action space at the decision moment; the task action space is represented by the operating frequency of each processor core and the processor cores allocated for power tasks.

[0170] S507 calls upon each processor core to process its corresponding power task according to the core's operating frequency, thus obtaining the task processing result at the decision moment.

[0171] S508: Statistically analyze the task processing results at each decision point to obtain the processing results for the task request.

[0172] The specific content of S501 to S508 can be found in the aforementioned description and will not be repeated here.

[0173] In summary, current DVFS implementations in the operating system kernel primarily rely on dynamically adjusting the frequency based on processor load. This fails to adaptively adjust the frequency according to the demands of power applications, thus failing to meet latency guarantee requirements and reducing processor efficiency. Therefore, the method provided in this application adjusts the processor core operating frequency based on awareness of power application needs, improving the service awareness capability of frequency adjustment. By distributing power computing tasks across multiple cores for scheduling, it balances the load on processor cores, improves the latency guarantee rate for power computing tasks, and reduces processor energy consumption. This provides technical support for energy saving and efficient utilization of limited resources in lightweight power MEC devices.

[0174] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0175] Based on the same inventive concept, this application also provides a power task processing apparatus for implementing the power task processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more power task processing apparatus embodiments provided below can be found in the limitations of the power task processing method described above, and will not be repeated here.

[0176] In one exemplary embodiment, such as Figure 6 As shown, a power task processing device is provided, comprising: an acquisition module 601, a construction module 602, a first processing module 603, a second processing module 604, and a third processing module 605, wherein:

[0177] The module 601 is used to acquire a task request, the task request including task information of at least one power task; the module 602 is used to construct a task state space for each decision time based on the task information of each power task at the decision time and the task processing information of each power task at the previous decision time; the first processing module 603 is used to call a target depth Q network to analyze the task state space to obtain the task action space at the decision time; the task action space is represented by the operating frequency of each processor core and the processor cores allocated for the power task; the second processing module 604 is used to call the processor cores to process the corresponding power task according to the operating frequency of each processor core to obtain the task processing result at the decision time; the third processing module 605 is used to statistically analyze the task processing results corresponding to each decision time to obtain the processing result for the task request.

[0178] In one embodiment, the task information includes the maximum task delay and the task size, and the task processing information includes the task delay, the remaining task capacity, the scheduling indication parameters, and the processor core operating frequency. The construction module 602 is further configured to: for each power task at the decision time, determine the current task delay of the power task at the decision time based on the historical task delay, historical remaining task capacity, historical scheduling indication parameters, and historical processor core operating frequency of the power task at the previous decision time; determine the current remaining task capacity of the power task at the decision time based on the task size, the historical scheduling indication parameters, the historical processor core operating frequency, the historical task delay, and the current task delay; determine the delay guarantee status of the power task at the decision time based on the relationship between the current task delay and the maximum task delay, and the current remaining task capacity; and construct the task state space at the decision time based on the task state, current task delay, delay guarantee status, current remaining task capacity, and energy consumption of each processor core for each power task.

[0179] In one embodiment, the first processing module 603 is further configured to: acquire an initial depth Q-network; the initial depth Q-network includes an initial action network and an initial target network; the initial action network and the initial target network have the same network parameters; for each historical decision moment, when the first historical state space at the historical decision moment is input to the initial action network, determine a first historical action space corresponding to the first historical state space; the first historical state space is constructed based on the historical task information of the historical power task corresponding to the historical decision moment and the historical processing information of the previous historical decision moment; determine the historical reward at the historical decision moment and the next historical... The second historical state space at the decision moment; combining the first historical state space, the first historical action space, the historical reward, and the second historical state space to obtain an experience sample; for at least one experience sample, with the second historical state space input into the initial target network, obtaining the sample target value corresponding to the experience sample based on the historical reward; updating the initial action network based on the training loss determined by each sample target value to obtain an updated action network; obtaining the updated target network by setting the network parameters of the initial target network to be consistent with the updated action network; and obtaining a target depth Q network based on the updated action network and the updated target network.

[0180] In one embodiment, the first processing module 603 is further configured to: determine the historical task delay guarantee rate at the historical decision time based on the historical delay guarantee status of each historical power task at the historical decision time and the total number of historical power tasks; determine the historical processor energy consumption at the historical decision time based on the historical energy consumption of each processor core at the historical decision time; obtain the energy-saving delay guarantee index at the historical decision time based on the weighted difference between the historical task delay guarantee rate and the historical processor energy consumption; and determine the historical reward at the historical decision time based on the energy-saving delay guarantee index.

[0181] In one embodiment, the first processing module 603 is further configured to: when the first historical state space at the historical decision moment is input into the initial action network, obtain a first target value corresponding to each of the multiple candidate action spaces; based on each first target value and a first constraint, determine a first historical action space corresponding to the first historical state space from the multiple candidate action spaces; the first constraint satisfies that after the processor core completes processing a historical power task, the processor core begins to process other historical power tasks.

[0182] In one embodiment, the first processing module 603 is further configured to: obtain a second constraint condition; the second constraint condition satisfies: the operating frequency of the processor core is less than or equal to the maximum operating frequency;

[0183] Based on each of the first target values, the first constraint, and the second constraint, a first historical action space corresponding to the first historical state space is determined from the plurality of candidate action spaces.

[0184] Each module in the aforementioned power task processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the 5G MEC device in hardware form or independent of it, or stored in the memory of the 5G MEC device in software form, so that the processor can call and execute the operations corresponding to each module.

[0185] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data used in the power task processing. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a power task processing method.

[0186] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a power task processing method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0187] Those skilled in the art will understand that Figure 7 or Figure 8The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0188] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0189] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0190] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0191] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0192] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0193] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0194] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for processing power tasks, characterized in that, The method includes: Obtain a task request, the task request including task information for at least one power task; For each decision moment, a task state space is constructed based on the task information of each power task at the decision moment and the task processing information of each power task at the previous decision moment. The task state space is analyzed by calling the target depth Q network to obtain the task action space at the decision moment; the task action space is represented by the operating frequency of each processor core and the processor cores allocated for the power task. According to the operating frequency of each processor core, the processor core is called to process the corresponding power task to obtain the task processing result at the decision time. The processing results of the tasks at each decision point are statistically analyzed to obtain the processing result for the task request.

2. The method according to claim 1, characterized in that, The task information includes the maximum task delay and the task size; the task processing information includes historical task delay, historical task remaining amount, historical scheduling indication parameters, and the historical operating frequency of the processor core. The step of constructing the task state space at the decision time based on the task information of each power task at the decision time and the task processing information of each power task at the previous decision time includes: For each power task at the decision time, the current task delay of the power task at the decision time is determined based on the historical task delay, historical task remaining amount, historical scheduling indication parameters and processor core historical operating frequency of the power task at the previous decision time. Based on the task size of the power task, the historical scheduling indication parameters, the historical operating frequency of the processor core, the historical task delay, and the current task delay, the remaining amount of the current task of the power task at the decision time is determined; Based on the relationship between the current task delay and the maximum task delay, and the remaining amount of the current task, the delay guarantee status of the power task at the decision time is determined; Based on the current task status, current task delay, delay guarantee status, current task remaining amount, and the energy consumption of each processor core, the task state space at the decision moment is constructed.

3. The method according to claim 1 or 2, characterized in that, The training methods for the target depth Q-network include: Obtain an initial depth Q-network; the initial depth Q-network includes an initial action network and an initial target network; the network parameters of the initial action network and the initial target network are the same; For each historical decision moment, when the first historical state space at the historical decision moment is input into the initial action network, a first historical action space corresponding to the first historical state space is determined; the first historical state space is constructed based on the historical task information of the historical power task corresponding to the historical decision moment and the historical processing information of the previous historical decision moment. Determine the historical reward at the historical decision moment and the second historical state space at the next historical decision moment; By combining the first historical state space, the first historical action space, the historical reward, and the second historical state space, an experience sample is obtained. For at least one experience sample, with the second historical state space input into the initial target network, the sample target value corresponding to the experience sample is obtained based on the historical reward. Based on the training loss determined for each of the target values ​​of the samples, the initial action network is updated to obtain the updated action network; The updated target network is obtained by setting the network parameters of the initial target network to be consistent with the updated action network; Based on the updated action network and the updated target network, the target depth Q network is obtained.

4. The method according to claim 3, characterized in that, Determining the historical reward at the historical decision moment includes: The historical task delay guarantee rate at the historical decision-making time is determined based on the historical delay guarantee status of each historical power task at the historical decision-making time and the total number of historical power tasks. The historical processor energy consumption at the historical decision-making time is determined based on the historical energy consumption of each processor core at the historical decision-making time. The energy-saving delay guarantee index at the historical decision moment is obtained by weighting the difference between the historical task delay guarantee rate and the historical processor energy consumption. Based on the energy-saving delay guarantee index, the historical reward at the historical decision-making moment is determined.

5. The method according to claim 3, characterized in that, When the first historical state space at the historical decision moment is input into the initial action network, determining the first historical action space corresponding to the first historical state space includes: When the first historical state space at the historical decision moment is input into the initial action network, the first target value corresponding to each of the multiple candidate action spaces is obtained; Based on each of the first target values ​​and the first constraints, a first historical action space corresponding to the first historical state space is determined from the plurality of candidate action spaces; the first constraint satisfies that after the processor core completes processing a historical power task, the processor core begins to process other historical power tasks.

6. The method according to claim 5, characterized in that, The step of determining the first historical action space corresponding to the first historical state space from the plurality of candidate action spaces based on each of the first target values ​​and the first constraints includes: Obtain the second constraint condition; the second constraint condition satisfies: the operating frequency of the processor core is less than or equal to the maximum operating frequency; Based on each of the first target values, the first constraint, and the second constraint, a first historical action space corresponding to the first historical state space is determined from the plurality of candidate action spaces.

7. A power task processing device, characterized in that, The device includes: The acquisition module is used to acquire task requests, wherein the task requests include task information for at least one power task. The construction module is used to construct the task state space at each decision time based on the task information of each power task at the decision time and the task processing information of each power task at the previous decision time. The first processing module is used to call the target depth Q network to analyze the task state space and obtain the task action space at the decision moment; the task action space is represented by the operating frequency of each processor core and the processor cores allocated for the power task. The second processing module is used to call the processor core to process the corresponding power task according to the operating frequency of each processor core, and obtain the task processing result at the decision time. The third processing module is used to statistically analyze the task processing results corresponding to each of the decision times, and obtain the processing results for the task request.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images