A cloud edge inference scheduling method based on candidate action consequence perception

Abstract

The present application belongs to the field of mobile edge computing and edge intelligent inference technology, and discloses a cloud edge terminal inference scheduling method based on candidate action consequence perception. It includes: obtaining terminal device task demand and cloud edge terminal resource state; establishing a candidate execution action set; estimating the consequences of each candidate execution action before action selection to generate an original consequence estimation vector; generating constraint perception features, action mask matrix and minimum transmission power matrix according to the consequence estimation results; inputting the constraint perception features into the multi-agent strategy network and outputting the joint offloading decision under the constraint of the action mask matrix; determining the uplink transmission power of each terminal device according to the minimum transmission power matrix and performing batch scheduling on the offloaded tasks; and updating the strategy network and the dynamic constraint penalty coefficient according to the task execution result. The present application can reduce the unactionable action exploration in the early stage of training, reduce the constraint violation rate, and improve the service quality satisfaction rate of the inference task.

Description

Technical Field

[0001] This invention relates to the fields of mobile edge computing and edge intelligent reasoning technology, and in particular to a cloud-edge-device reasoning scheduling method based on candidate action consequence perception. Background Technology

[0002] As large language model inference services expand to mobile terminals, in-vehicle devices, unmanned devices, and edge sensing terminals, individual inference tasks typically combine high model accuracy requirements, strict response latency constraints, and large computational resource demands. Limited by local model size, computing power, and energy resources, terminal devices relying solely on local computing often struggle to balance accuracy, latency, and privacy requirements. Therefore, collaborative offloading and joint optimization of wireless transmission power and batch processing resources are necessary among local terminal devices, edge servers, and cloud servers.

[0003] MEC-LLM inference offloading scenarios are characterized by multi-objective coupling and dynamic resource competition. Local execution offers good privacy and low communication overhead, but is limited by local model accuracy and computing power. Edge servers have low transmission latency and moderate model capabilities, but are affected by concurrent offloading within the region, batch processing capacity, and cross-regional backhaul overhead. Cloud servers offer higher model accuracy and stronger computing power, but typically result in longer communication links, higher privacy losses, and greater transmission costs.

[0004] Currently, existing deep reinforcement learning and multi-agent reinforcement learning methods typically output the probability of unloading actions directly based on the current state, or jointly entrust discrete unloading actions and continuous power control to the policy network for learning. Some methods reduce the proportion of infeasible actions through action masks, heuristic action scoring, or resource constraint penalties; however, they usually do not explicitly characterize the accuracy margin, latency risk, power requirements, privacy loss, and capacity pressure corresponding to each candidate action before action sampling. Specifically, existing methods have at least the following problems: the policy network needs to rely on lag rewards for repeated trial and error in the early stages of training, which easily leads to the sampling of obviously infeasible unloading actions; when unloading actions and continuous power control are learned by the policy network at the same time, low-power ineffective exploration and policy update fluctuations are likely to occur near the power boundary; heuristic scoring often compresses the consequences of multi-dimensional candidate actions into a single scalar, making it difficult to distinguish whether the degradation of action quality is caused by insufficient accuracy, excessive latency, power infeasibility, privacy loss, or batch processing congestion; when multiple terminal devices unload concurrently, the batch processing capacity of edge servers and cloud servers will form a dynamic competition, and simply outputting actions based on the current state cannot reflect the capacity pressure in a timely manner. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing a cloud-edge-device inference scheduling method based on candidate action consequence perception. This solves the problem that existing policy networks are unable to utilize the accuracy, latency, power, and capacity consequence information of candidate actions before action sampling, which leads to slow training convergence, high constraint violation rate, and unstable continuous power control.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a cloud-edge-device inference scheduling method based on candidate action consequence perception, comprising the following steps: S1. Obtain task requirement information, wireless communication status information, and cloud-edge-device resource status information from multiple terminal devices; S2. Based on the task requirement information and cloud-edge-device resource status information, construct a candidate set of execution actions for each terminal device; the candidate set of execution actions includes local execution actions, edge server execution actions, and cloud execution actions. S3. Perform consequence perception estimation on each candidate execution action in the candidate execution action set to obtain the execution accuracy, end-to-end latency, computation latency, communication latency, capacity pressure, estimated batch size, and uplink transmit power required to meet the latency constraints for each candidate execution action. S4. Generate constraint-aware features based on the consequence-aware estimation results, and construct an action mask matrix and a minimum transmission power matrix; S5. Input the constraint-aware features into the multi-agent policy network, and under the constraint of the action mask matrix, output a joint unloading action composed of the unloading actions of each terminal device through a sampling mechanism. S6. Determine the uplink transmission power of each terminal device according to the minimum transmission power matrix, and perform batch processing scheduling for tasks offloaded to the edge server or cloud server. S7. Update the multi-agent policy network parameters based on the feedback results of task execution, and adaptively adjust the penalty coefficient based on the constraint violation situation.

[0007] The consequence-aware estimation is used to generate the original consequence estimation vector corresponding to each candidate execution action before action sampling; the original consequence estimation vector includes the inference accuracy provided by the candidate execution location, the estimated total latency, the computation latency, the communication latency, the original uplink transmit power required to meet the latency constraint, the latency violation degree, the accuracy violation degree, the capacity pressure, and the estimated batch size.

[0008] The constraint-aware features are used to jointly characterize the feasibility and execution quality of candidate actions, including action mask flag, latency margin, accuracy margin, privacy utility, execution cost, normalized minimum transmit power, capacity pressure, location matching flag, execution level, and bottleneck ratio.

[0009] The action mask matrix is ​​used to constrain and filter candidate actions and to sample them. When a candidate action meets the basic action feasibility, accuracy, delay, and uplink transmit power constraints, the corresponding action is retained as a sampleable state; otherwise, the corresponding action is masked as an unsampleable state.

[0010] The minimum transmit power matrix is ​​used to record the minimum transmit power required to complete uplink data transmission under the condition of satisfying the delay constraint, and is obtained by back-calculation based on wireless channel state information, data transmission requirements and remaining available uplink transmission time.

[0011] For actions performed on edge servers or in the cloud, based on one or more of the following: computation latency, batch processing waiting latency, backhaul latency, and downlink transmission latency, non-uplink transmission latency is deducted from the task latency constraint to obtain the remaining available uplink transmission time. Then, based on the remaining available uplink transmission time, the uplink transmit power required to satisfy the latency constraint is calculated, and upper and lower bound constraints are applied to the uplink transmit power.

[0012] The multi-agent policy network includes a task feature encoding module, a candidate action consequence encoding module, a global resource encoding module, a region embedding module, a multi-device interaction encoding module, and an action output module. Features are extracted from data from each terminal through the task feature encoding module, the candidate action consequence encoding module, and the region embedding module. Features of the global resource state are extracted through the global resource encoding module. These features are then fused by the feature fusion module and passed through the multi-device interaction encoding module to obtain interaction features. These interaction features are constrained by an action mask matrix in the action output module to obtain various output actions. The multi-device interaction encoding module is used to model the resource competition relationship between multiple terminal devices using one of the following: attention network, graph neural network, or Transformer network.

[0013] The joint offloading action is decoupled from the uplink transmit power control process. The discrete offloading action is output by the multi-agent policy network, and the continuous uplink transmit power is determined by the minimum transmit power matrix combined with the power projection strategy, and task scheduling is performed based on the cloud-edge-device batch processing resource status.

[0014] The multi-agent policy network is updated based on reinforcement learning, and the reward function is dynamically penalized and corrected in combination with delay violations, accuracy violations, and batch processing capacity violations during task execution, so as to reduce the constraint violation rate and improve the policy convergence stability.

[0015] An electronic device includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the electronic device performs a cloud-edge-device inference scheduling method based on candidate action consequence awareness.

[0016] The beneficial effects of adopting the above technical solution are as follows: The cloud-edge-device inference scheduling method based on candidate action consequence perception provided by this invention adopts a candidate action consequence perception estimation mechanism to estimate the execution accuracy, end-to-end latency, capacity pressure, and required uplink transmission power of each candidate action before action sampling in the multi-agent policy network. This allows the multi-agent policy network to obtain the constraint state of each candidate action before action sampling, reducing the exploration of unacceptable actions that rely solely on delayed rewards and accelerating training convergence. By using the action mask matrix and minimum transmission power matrix to pre-position service quality constraints and wireless power constraints, the number of unacceptable action samplings is reduced, and the latency constraint violation rate is lowered. Discrete offloading actions and continuous power control are decoupled, reducing policy update fluctuations near the power boundary. The resource competition relationship under concurrent offloading of multiple terminal devices is modeled through a multi-device interactive coding structure, effectively representing the edge server load, cloud server load, and batch processing capacity pressure. A dynamic constraint penalty mechanism is adopted to adaptively adjust the penalty coefficient according to the constraint violation rate, reducing the constraint violation rate during the convergence phase. Attached Figure Description

[0017] Figure 1 This is a framework diagram of candidate action consequence perception unloading action and power control provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of the cloud-edge-device inference scheduling method based on candidate action consequence awareness provided in Embodiment 1 of the present invention. Figure 3 This is a flowchart of the SCA-MAPPO model training process provided in Embodiment 1 of the present invention; Figure 4 This is a structural diagram of the multi-agent policy network module provided in Embodiment 1 of the present invention; Figure 5 This is a training reward comparison chart provided in Embodiment 1 of the present invention; Figure 6 This is a comparison chart of comprehensive performance indicators provided in Embodiment 1 of the present invention; Figure 7 This is a batch size impact diagram provided in Embodiment 1 of the present invention. Detailed Implementation

[0018] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0019] Example 1 This embodiment provides a cloud-edge-device inference scheduling method based on candidate action consequence awareness, such as... Figure 1 As shown, this invention constructs a cloud-edge-device collaborative inference environment. This inference environment includes multiple terminal devices, multiple edge servers, and a cloud server, deploying large language models of varying scales at different execution levels. The training phase initializes the training environment, SCA candidate action consequence perception controller, multi-agent policy network, centralized value network, trajectory cache, and training hyperparameters; wherein the multi-agent policy network adopts an SCA-MAPPO joint policy network. Training hyperparameters include maximum training epochs, maximum number of time slots per epoch, learning rate, discount factor, PPO pruning factor, initial values ​​of Lagrange multipliers, maximum transmit power, minimum transmit power, and power safety margin coefficient; as shown... Figure 2 The diagram shown is a flowchart of the cloud-edge-device inference scheduling method based on candidate action consequence awareness provided by the present invention. The specific steps are as follows: Step S1: Obtain the status; Within each scheduling time slot, the task requirements, wireless communication status, and cloud-edge-device resource status of multiple terminal devices are acquired. Task requirements include the number of input tokens, the number of output tokens, minimum inference accuracy, maximum end-to-end latency, accuracy preference, latency preference, and privacy preference. Global resource status includes edge server load, edge server batch processing capacity, potential cloud load, cloud batch processing capacity, wireless link status, and current time slot information. Each LLM inference task can be represented as: In the formula, This represents the number of input tokens for the i-th terminal device; This represents the number of output tokens for the i-th terminal device; Indicates the minimum inference precision requirement; Indicates the maximum end-to-end latency requirement; Indicates the precision preference weight; Indicates the delay preference weight; This represents the privacy preference weight; and the sum of the accuracy preference weight, latency preference weight, and privacy preference weight is 1.

[0020] Step S2: Establish a set of candidate actions to be executed; Establish a set of candidate actions for each terminal device: In the formula, ED indicates execution by the local terminal device; ES indicates execution by the edge server and its subscript is used to distinguish different edge servers; Cloud indicates execution offloaded to the cloud server.

[0021] Step S3: Estimate the consequences of candidate actions; Before action selection, the SCA candidate action consequence awareness controller estimates the consequences of each candidate action. For locally executed actions, it estimates local computation latency, local inference accuracy, and the degree of violation of local execution constraints based on the terminal device's local computing power, task input length, task output length, and local model accuracy, and sets the corresponding minimum feasible uplink transmit power to a preset minimum power. For edge server executed actions, it estimates edge computation latency, uplink communication latency, downlink communication latency, cross-regional backhaul latency, total latency, edge capacity pressure, and minimum feasible uplink transmit power based on the terminal device's region, target edge server batch processing capacity, edge model accuracy, wireless link rate, cross-regional backhaul relationship, and current edge load status. For cloud executed actions, it estimates cloud computation latency, uplink communication latency, cloud backhaul latency, downlink communication latency, total latency, cloud capacity pressure, and minimum feasible uplink transmit power based on cloud server batch processing capacity, cloud model accuracy, wireless access rate, cloud backhaul rate, and potential cloud load status.

[0022] For the i-th terminal device and the candidate action a, the SCA candidate action consequence perception controller first generates the original consequence estimation vector, denoted as: In the formula Indicates the inference accuracy that can be provided by the candidate execution location; This indicates an estimated total delay; Indicates computation delay; Indicates communication delay; This represents the original uplink transmit power required to satisfy the delay constraint; Indicates the degree of delay violation; Indicates the degree of precision violation; Indicates capacity pressure; This represents the estimated batch size; the above original consequence estimation vector is used to characterize the direct estimation result of the candidate execution action under the current scheduling slot.

[0023] Step S4: Generate constraint-aware features and action constraints; Constraint-aware features, an action mask matrix, and a minimum transmit power matrix are generated based on the candidate action consequence estimation results. When constructing the policy network input, constraint-aware feature vectors are generated based on the aforementioned original consequence estimation vectors, combined with the feasibility, privacy utility, execution cost, location matching relationship, and execution level of the candidate actions. Each candidate action corresponds to a ten-dimensional constraint-aware feature vector, represented as: In the formula, Indicates the action mask flag; Indicates latency margin; Indicates the precision margin; Indicates privacy utility; Indicates execution cost; This represents the normalized minimum transmit power; Indicates capacity pressure; Indicates a location matching flag; Indicates the execution level; This indicates the bottleneck ratio.

[0024] The bottleneck ratio is determined by the maximum value among the computation latency ratio, communication latency ratio, and estimated batch processing ratio. These features form constraint-aware feature tensors in the device and action dimensions, and are then concatenated or mapped with the task state of the corresponding terminal device to form an enhanced observation O', which is input into the multi-agent policy network.

[0025] The delay margin is expressed as follows: In the formula, i represents the terminal device number, and a represents the candidate action to be executed. Indicates delay constraints, This indicates an estimated total delay; The precision margin is expressed as: In the formula, This indicates the inference accuracy that the candidate execution location can provide. This represents the minimum inference accuracy required for the task. Indicates the maximum acceptable precision; Capacity pressure is expressed as: In the formula, Indicates the processable size. This indicates the maximum batch processing capacity.

[0026] The continuous values ​​in the above ten-dimensional features can be clipped or normalized according to preset upper and lower bounds.

[0027] The action mask matrix is ​​generated per device, per candidate action. Its matrix elements indicate whether the corresponding candidate action is sampleable. For candidate action 'a', the corresponding matrix element is set to 1 if it satisfies the following conditions: basic action feasibility, candidate execution node inference accuracy not lower than task accuracy constraint, estimated total latency not exceeding task latency constraint or preset latency tolerance upper bound, and original uplink transmit power not exceeding maximum uplink transmit power; otherwise, it is set to 0. Capacity pressure serves as a constraint-aware feature and the basis for subsequent dynamic penalties, but is not used as a hard mask condition.

[0028] The minimum transmit power matrix is ​​used to record the minimum transmit power after pruning by upper and lower bounds. For actions executed by the edge server and actions executed in the cloud, if there is still remaining uplink transmission time after deducting the estimated computation delay, fixed communication delay, return delay, downlink delay, and batch processing waiting delay, then the original uplink transmit power required to meet the delay constraint is inferred from the task input data volume, uplink bandwidth, channel gain, and noise power, and is expressed as: In both formulas, This represents the original uplink transmit power required for the i-th terminal device to satisfy the delay constraint under candidate action a; This represents the corresponding element in the minimum transmit power matrix; and These represent the preset minimum and maximum transmit power, respectively; the power feasibility of the candidate actions is still determined by... judge.

[0029] Step S5: Output the combined unloading action; like Figure 4 As shown, the multi-agent policy network includes a task feature encoding module, a candidate action consequence encoding module, a global resource encoding module, a region embedding module, a multi-device interaction encoding module, and an action output module. Among these, the task feature encoding module, candidate action consequence encoding module, region embedding module, and global resource encoding module constitute the feature encoding layer. Figure 4 The diagram also shows a feature fusion module used to fuse the results of each encoding step.

[0030] Suppose there are N terminal devices in the current scheduling time slot. For the i-th terminal device, the task feature xi is input to the task feature encoding module, the candidate action consequence feature Fi is input to the candidate action consequence encoding module, the region information qi is input to the region embedding module, and the global resource status Gt is input to the global resource encoding module. The above encoding results are fused by the feature fusion module to obtain the fused feature hi corresponding to the i-th terminal device. The fused features corresponding to each terminal device are then input to the multi-device interaction encoding module to model the resource competition relationship between multiple terminal devices.

[0031] The multi-device interaction encoding module outputs the interaction feature zi corresponding to the i-th terminal device. The action output module generates the action probability distribution πi corresponding to the i-th terminal device based on the interaction feature zi. The action mask matrix Mt is input into the action output module as independent constraint information to mask inactive actions and normalize the probability of the masked actions to finally obtain the joint unloading action dt.

[0032] Based on the aforementioned action probability distribution, the unloading action of the i-th terminal device in the t-th scheduling time slot can be expressed as: In the formula, Represented as a multi-agent policy network; This indicates enhanced observation; This represents the mask vector corresponding to the i-th terminal device in the action mask matrix.

[0033] The joint unloading action is obtained by sampling the action probability distribution constrained by the action mask matrix. If a terminal device has no action to sample, it will first fall back to the basic set of available actions; if it is still empty, a backup distribution will be constructed on all candidate actions to ensure that unloading actions can be output during both the training and deployment decision-making phases.

[0034] Step S6: Power control and batch scheduling; Sampling yields joint unloading action Then, the power projection module reads the corresponding element from the minimum transmit power matrix and generates the uplink transmit power according to the power safety margin coefficient, as follows: In the formula, Let be the uplink transmit power of the i-th terminal device in the t-th scheduling time slot; This represents the power safety margin coefficient; the multi-agent policy network is responsible for outputting discrete offloading actions. The continuous uplink transmit power is determined by the minimum transmit power matrix. The power projection module generates the power output of each terminal device after projection, which can be denoted as: For actions performed locally, the actual transmission power is the preset minimum power; for actions performed on edge servers or in the cloud, the actual transmission power is obtained by projecting the corresponding minimum transmission power.

[0035] After performing joint offloading and uplink transmit power control actions in the cloud-edge-device collaborative inference environment, batch scheduling is performed on tasks offloaded to edge servers or cloud servers. Batch scheduling includes edge server batch scheduling and cloud server batch scheduling. For edge server batch scheduling, tasks offloaded to edge servers are grouped according to the target edge server. Scheduling priorities are determined based on the remaining latency budget, task weight, communication completion time, batch processing wait latency, and the remaining batch processing capacity of the target edge server, forming edge batch processing task groups that do not exceed the batch processing capacity and meet the latency budget. For cloud server batch scheduling, tasks offloaded to cloud servers are sorted according to communication completion time, remaining latency budget, task weight, or batch processing wait latency, forming cloud batch processing task groups within the maximum batch processing capacity of the cloud server. When the number of candidate tasks exceeds the corresponding batch processing capacity, tasks that cannot enter the current batch processing task group are marked as capacity failure tasks.

[0036] Step S7: Policy update and constraint penalty adjustment; The joint reward and constraint violation information are calculated based on the actual task execution results, and the multi-agent policy network and dynamic constraint penalty coefficients are updated based on the joint reward and constraint violation information. Specifically, accuracy satisfaction, latency satisfaction, and privacy satisfaction are calculated based on the task execution results, and an effective weighted satisfaction reward is formed according to the task weight; the accuracy violation rate, latency violation rate, and capacity failure rate are calculated based on whether the task violates the accuracy constraint, latency constraint, and batch processing capacity constraint; the corresponding dynamic constraint penalty coefficient is updated based on the difference between the violation rate and the preset target violation rate, expressed as: In the formula, This represents the reward after a dynamic constraint penalty. Indicates the original reward; , , These are the penalty coefficients for delay constraints, accuracy constraints, and capacity constraints, respectively. , , These are the degrees of latency violation, precision violation, and capacity violation, respectively. The learning rate; The current violation rate; For the target violation rate; This is the upper bound of the penalty coefficient.

[0037] Will enhance observation Global resource status, joint uninstallation actions Uplink transmit power after projection The logarithmic probability of the joint action, the state of the next time slot, and the termination flag are stored in the trajectory buffer. This represents the joint execution action that actually acts on the environment in the t-th scheduling time slot. At least include joint unloading actions and projected uplink transmit power The trajectory cache can be recorded as Calculate the discount reward based on the trajectory cache. The state value is output by a centralized value network. Advantage function estimation It can be represented as Subsequently, the joint policy network is updated using PPO pruning objectives, the centralized value network is updated using value loss, and the Lagrange multipliers are updated based on the violation rate.

[0038] Finally, it is determined whether the current training round has reached the maximum number of training rounds or whether the strategy meets the convergence condition. If not, the next training round is entered to continue environmental interaction, candidate action consequence estimation, joint action sampling, and network update; if the conditions are met, the model, training checkpoints, and training results are saved, and the training phase is completed.

[0039] In this embodiment, the training phase and the deployment decision phase are as follows: Figure 3 As shown, the training phase includes: initializing the training environment, candidate action consequence-aware controller, multi-agent policy network, centralized value network, trajectory cache, and training hyperparameters; acquiring environmental observations, task requirements, and global resource status, and using the candidate action consequence-aware controller to evaluate candidate actions and their constraints; generating an action mask matrix, constraint-aware features, and a minimum transmit power matrix; selecting a joint offloading action under the constraints of the action mask matrix, and determining the uplink transmit power based on the minimum transmit power matrix; executing the offloading action and power control action, acquiring reward and constraint violation information, and storing it in the trajectory cache; updating the multi-agent policy network and centralized value network based on trajectory rewards and PPO pruning targets, and updating the dynamic constraint penalty coefficient based on the constraint violation rate; determining whether the maximum number of training rounds has been reached or the convergence condition has been met; if not, returning to the step of acquiring environmental observations, task requirements, and global resource status to continue training; if yes, saving the model and training results.

[0040] In one optional implementation, the training hyperparameters include the maximum number of training epochs, the maximum number of scheduling slots per epoch, the policy network learning rate, the value network learning rate, the discount factor, the PPO pruning factor, the number of PPO repetition updates, and the dynamic constraint penalty factor. The maximum number of training epochs can be 500 to 3000 epochs, typically 1000 epochs; the maximum number of scheduling slots per epoch can be 50 to 200, typically 100; the policy network learning rate and the value network learning rate can be... to Typical value The discount factor can range from 0.95 to 0.999, with a typical value of 0.99; the PPO pruning factor can range from 0.1 to 0.3, with a typical value of 0.2; the number of PPO repetition updates can range from 3 to 10, with a typical value of 8. The dynamic constraint penalty factor can be adaptively updated based on the latency violation rate, accuracy violation rate, and capacity failure rate. Its initial value, update step size, and upper bound can be set according to the task scale and resource constraint strength. The above values ​​are only used to illustrate an feasible training configuration and can be adjusted according to the task scale, number of devices, and resource configuration, and do not constitute a limitation on the scope of protection of this invention.

[0041] In one optional implementation, the convergence condition is determined jointly based on training rewards, effective weighted satisfaction, and constraint violations. Specifically, after the training epochs reach a preset minimum number of training epochs, if the average reward and effective weighted satisfaction changes in the most recent training epochs are both less than preset thresholds, and the average latency violation rate, average precision violation rate, and average capacity failure rate are all not higher than the corresponding target violation rate or its tolerance range, then the strategy is deemed to have met the convergence condition. In a specific embodiment, the preset minimum number of training epochs can be 200 epochs, the sliding statistical window can be 50 epochs, the number of consecutive satisfactions can be 3, the reward change threshold and the effective weighted satisfaction change threshold can be 0.01, and the constraint violation rate tolerance margin can be 0.005. If the training process does not meet the convergence condition in advance, training is terminated when the maximum number of training epochs is reached, and model parameters with higher average rewards or higher effective weighted satisfaction are saved for the deployment decision stage.

[0042] During the deployment decision-making phase, the control node receives the terminal device's task requirements, wireless communication status, and cloud-edge-device resource status in each scheduling time slot. It then calls the candidate action consequence perception controller to generate an action mask matrix, a minimum transmit power matrix, and constraint perception features. Finally, it calls the trained multi-agent policy network to output a joint offloading action. The uplink transmit power is determined by the power projection module. The batch processing scheduling module outputs batch processing task groups for edge servers or cloud servers.

[0043] In one optional implementation, the system includes multiple edge regions, each with an edge server. Terminal devices are assigned to their corresponding edge regions based on their access locations. When the load on an edge server in a certain region is high, the SCA candidate action consequence awareness controller increases the capacity pressure and estimated batch processing size for the corresponding candidate action on that edge server. Based on the cross-regional backhaul latency and the remaining batch processing capacity of the target edge server, it generates latency margins and power requirements for different candidate actions, enabling the joint policy network to achieve cross-regional traffic distribution under the constraints of the action mask matrix.

[0044] In one alternative implementation, the tasks can include hybrid tasks with different accuracy requirements and privacy preferences. For high-precision tasks, the accuracy margin in the constraint-aware features can characterize the accuracy differences between local, edge, and cloud models; for high-privacy tasks, the privacy utility dimension can enhance the relative attractiveness of performing actions locally or on edge servers; for low-latency tasks, latency margin and minimum transmit power can characterize the communication cost required for candidate actions to meet the latency budget. Thus, the joint policy network can learn differentiated offloading strategies based on task preferences and the consequences of candidate actions without setting independent rules for different task types.

[0045] In one alternative implementation, the system reacquires the available batch processing capacity of the edge server and cloud server at the beginning of each scheduling time slot and writes it into the global resource state. When the available batch processing capacity of the edge server decreases, the SCA candidate action consequence awareness controller increases the capacity pressure of the corresponding candidate execution action; when the batch processing capacity increases, the capacity pressure decreases, and the joint policy network can adjust the offloading probability of the edge server to reduce the capacity failure rate and improve the service quality satisfaction rate.

[0046] like Figure 5 As shown, Figure 5 This is a comparison of average training rewards under multiple random seeds. Under the same training rounds, the average training reward of the method in this invention remains at a high level overall, and shows a relatively stable convergence trend in subsequent training stages. Compared with the comparative methods, the method in this invention improves the stability and convergence efficiency of the training process by introducing candidate action consequence estimation before action selection and combining action mask matrix and minimum emission power matrix to reduce the exploration of unsuitable actions.

[0047] like Figure 6 As shown, Figure 6 A comparison chart of the comprehensive performance indicators of different algorithms is provided. Under the same task scale and resource configuration, the method of this invention maintains a high level in comprehensive performance indicators such as the joint satisfaction rate of Service Level Objectives (SLOs), effective weighted satisfaction, and average reward; wherein, the joint satisfaction rate of Service Level Objectives (SLOs) represents the proportion of tasks that simultaneously satisfy latency constraints and accuracy constraints. This result shows that the method of this invention, by inputting candidate action consequence information such as latency margin, accuracy margin, privacy utility, capacity pressure, and minimum transmit power into the multi-agent policy network, can improve the adaptability to service quality constraints and resource constraints during cloud-edge-device inference scheduling.

[0048] like Figure 7 As shown, Figure 7This diagram illustrates the impact of changes in edge server batch size on effective weighted satisfaction. As the edge server batch size increases from small to medium and large configurations, the effective weighted satisfaction of the proposed method remains consistently high, and it can adjust its offloading actions according to changes in available batch capacity at the edge. This result demonstrates that the proposed method, by introducing features such as capacity pressure and estimated batch size, enables the policy network to perceive changes in edge server batch resources, reducing the impact of batch congestion on task execution performance and improving scheduling adaptability under different edge resource conditions.

[0049] Example 2 This embodiment provides an electronic device, including one or more processors and a memory. The memory stores computer-executable instructions, which, when executed by the one or more processors, cause the one or more processors to perform the cloud-edge-device inference scheduling method based on candidate action consequence perception as described in Embodiment 1. The processor may be a central processing unit, a graphics processing unit, a neural network processing unit, or other general-purpose or special-purpose processors. By executing the above method, the electronic device can realize candidate action consequence perception, multi-agent collaborative decision-making, and dynamic constraint penalty adjustment during the inference task offloading action generation process.

[0050] Example 3 This embodiment provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are executed by a processor, all or part of the steps of the cloud-edge-device inference scheduling method based on candidate action consequence awareness as described in Embodiment 1 are implemented. The computer-readable storage medium includes various media capable of storing program code, such as USB flash drives, external hard drives, read-only memory, random access memory, magnetic disks, or optical disks. The computer-readable storage medium can be deployed in terminal devices, edge servers, cloud servers, or independent control nodes.

[0051] Example 4 This embodiment provides a computer program product, including a computer program or computer-executable instructions. When the computer program or computer-executable instructions are executed by a processor, all or part of the steps of the cloud-edge-device inference scheduling method based on candidate action consequence awareness as described in Embodiment 1 are implemented. The computer program product can be applied to mobile edge computing, vehicle-to-everything (V2X) computing, industrial IoT, and cloud-edge-device collaborative inference scheduling scenarios.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the present invention.

Claims

1. A cloud-edge-device inference scheduling method based on candidate action consequence perception, characterized in that, Includes the following steps: S1. Obtain task requirement information, wireless communication status information, and cloud-edge-device resource status information from multiple terminal devices; S2. Based on the task requirement information and cloud-edge-device resource status information, construct a candidate set of execution actions for each terminal device; the candidate set of execution actions includes local execution actions, edge server execution actions, and cloud execution actions. S3. Perform consequence perception estimation on each candidate execution action in the candidate execution action set to obtain the execution accuracy, end-to-end latency, computation latency, communication latency, capacity pressure, estimated batch size, and uplink transmit power required to meet the latency constraints for each candidate execution action. S4. Generate constraint-aware features based on the consequence-aware estimation results, and construct an action mask matrix and a minimum transmission power matrix; S5. Input the constraint-aware features into the multi-agent policy network, and under the constraint of the action mask matrix, output a joint unloading action composed of the unloading actions of each terminal device through a sampling mechanism. S6. Determine the uplink transmission power of each terminal device according to the minimum transmission power matrix, and perform batch processing scheduling for tasks offloaded to the edge server or cloud server. S7. Update the multi-agent policy network parameters based on the feedback results of task execution, and adaptively adjust the penalty coefficient based on the constraint violation situation.

2. The cloud-edge-device inference scheduling method based on candidate action consequence awareness according to claim 1, characterized in that, The consequence-aware estimation is used to generate the original consequence estimation vector corresponding to each candidate execution action before action sampling; the original consequence estimation vector includes the inference accuracy provided by the candidate execution location, the estimated total latency, the computation latency, the communication latency, the original uplink transmit power required to meet the latency constraint, the latency violation degree, the accuracy violation degree, the capacity pressure, and the estimated batch size.

3. The cloud-edge-device inference scheduling method based on candidate action consequence awareness according to claim 1, characterized in that, The constraint-aware features are used to jointly characterize the feasibility and execution quality of candidate actions, including action mask flag, latency margin, accuracy margin, privacy utility, execution cost, normalized minimum transmit power, capacity pressure, location matching flag, execution level, and bottleneck ratio.

4. The cloud-edge-device inference scheduling method based on candidate action consequence awareness according to claim 1, characterized in that, The action mask matrix is ​​used to constrain and filter candidate actions and to sample them. When a candidate action meets the basic action feasibility, accuracy, delay, and uplink transmit power constraints, the corresponding action is retained as a sampleable state; otherwise, the corresponding action is masked as an unsampleable state.

5. The cloud-edge-device inference scheduling method based on candidate action consequence awareness according to claim 1, characterized in that, The minimum transmit power matrix is ​​used to record the minimum transmit power required to complete uplink data transmission under the condition of satisfying the delay constraint, and is obtained by back-calculation based on wireless channel state information, data transmission requirements and remaining available uplink transmission time.

6. The cloud-edge-device inference scheduling method based on candidate action consequence awareness according to claim 1, characterized in that, For actions performed on edge servers or in the cloud, based on one or more of the following: computation latency, batch processing waiting latency, backhaul latency, and downlink transmission latency, non-uplink transmission latency is deducted from the task latency constraint to obtain the remaining available uplink transmission time. Then, based on the remaining available uplink transmission time, the uplink transmit power required to satisfy the latency constraint is calculated, and upper and lower bound constraints are applied to the uplink transmit power.

7. The cloud-edge-device inference scheduling method based on candidate action consequence awareness according to claim 1, characterized in that, The multi-agent policy network includes a task feature encoding module, a candidate action consequence encoding module, a global resource encoding module, a region embedding module, a multi-device interaction encoding module, and an action output module; features of each terminal's data are extracted by the task feature encoding module, the candidate action consequence encoding module, and the region embedding module, respectively. Global resource status features are extracted by the global resource coding module; After being fused by the feature fusion module, the features are then processed by the multi-device interaction encoding module to obtain interactive features. These interactive features are constrained by the action mask matrix in the action output module to obtain various output actions. The multi-device interaction encoding module is used to model the resource competition relationship between multiple terminal devices using one of the following: attention network, graph neural network, or Transformer network.

8. The cloud-edge-device inference scheduling method based on candidate action consequence awareness according to claim 1, characterized in that, The joint offloading action is decoupled from the uplink transmit power control process. The discrete offloading action is output by the multi-agent policy network, and the continuous uplink transmit power is determined by the minimum transmit power matrix combined with the power projection strategy, and task scheduling is performed based on the cloud-edge-device batch processing resource status.

9. The cloud-edge-device inference scheduling method based on candidate action consequence awareness according to claim 1, characterized in that, The multi-agent policy network is updated based on reinforcement learning, and the reward function is dynamically penalized and corrected in combination with delay violations, accuracy violations, and batch processing capacity violations during task execution, so as to reduce the constraint violation rate and improve the policy convergence stability.

Images