An a3c algorithm-based large model edge inference scheduling optimization method
By combining an improved Actor network and a Critic network with a structure-aware SGA algorithm, the problem of insufficient module dependency modeling in large model scheduling is solved, achieving efficient edge-end collaborative deployment, improving the accuracy and stability of the scheduling strategy, and making it suitable for heterogeneous edge-end environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENHUA HOLLYSYS INFORMATION TECH CO LTD
- Filing Date
- 2025-12-01
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies fail to effectively model module dependency structures in large-scale model scheduling, resulting in dependency conflicts and execution bottlenecks in scheduling strategies, incomplete resource status awareness, difficulty in predicting device instability, and the policy gradient update of traditional A3C algorithms may lead to convergence oscillations or performance degradation, making it difficult to maintain the optimality and robustness of scheduling in dynamic environments.
An improved Actor network and an improved Critic network are adopted, combined with a structure-aware SGA optimization algorithm. By constructing a joint state vector, efficient collaborative deployment of modules between edge and end-side devices is achieved. An asynchronous parameter update mechanism is introduced, and the dot product of the policy gradient and the state value gradient is used to construct the advantage function. Furthermore, a structure guidance term and an orthogonal projection calibration mechanism are introduced to periodically update the scheduling policy.
It achieves high-precision scheduling strategy generation, improves resource utilization and scheduling adaptability, significantly reduces average inference latency and scheduling failure rate, enhances system stability and robustness, and is suitable for various heterogeneous edge scenarios.
Smart Images
Figure CN121681053B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence model inference scheduling and edge intelligent computing technology, and in particular to a large-scale model edge inference scheduling optimization method based on the A3C algorithm. Background Technology
[0002] With the continuous improvement of edge computing capabilities and the increasing demand for large-scale deep learning models in smart terminals, how to efficiently execute large-scale model inference tasks on heterogeneous edge and end-device devices has become a hot research topic. For scheduling optimization of multi-module deep models, existing technologies mostly adopt static resource allocation strategies or distributed scheduling mechanisms based on traditional reinforcement learning, but these still have the following problems in practical deployment:
[0003] First, most existing methods do not explicitly model the complex module dependency structure within large models, ignoring the topological order and tensor transfer paths between modules, leading to dependency conflicts and execution bottlenecks in scheduling strategies when facing cross-device execution. Second, resource state modeling often only focuses on a single dimension (such as CPU usage or memory availability), making it difficult to form a comprehensive understanding of the real edge environment. Third, most scheduling strategies use a uniform probability distribution or a greedy strategy to allocate nodes, failing to predict the risk of scheduling interruptions caused by unstable devices. Furthermore, during strategy optimization, the policy gradient update direction of the traditional A3C algorithm may violate the model dependency logic, causing convergence oscillations or performance degradation. Finally, existing methods struggle to maintain the optimality and robustness of scheduling in real-world scenarios where resource states are constantly changing.
[0004] Therefore, how to provide a large-scale model edge inference scheduling optimization method based on the A3C algorithm is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] One objective of this invention is to propose a large-model edge-side inference scheduling optimization method based on the A3C algorithm. This invention fully integrates heterogeneous device resource status, module dependency structure, and task scheduling constraint information. It uses an improved Actor network and an improved Critic network for policy generation and value evaluation, and introduces a structure-aware SGA optimization algorithm for asynchronous parameter updates. This enables efficient collaborative deployment of large-model modules between edge and end-side devices, and has the advantages of high scheduling accuracy, high resource utilization, and strong adaptability.
[0006] According to an embodiment of the present invention, a large-model edge inference scheduling optimization method based on the A3C algorithm includes the following steps:
[0007] Collect raw input data including the module dependency structure of the large model, resource status information of edge and end devices, and task scheduling constraint parameters, and construct a joint state vector containing inference structure graph, resource status representation and constraint conditions;
[0008] The joint state vector is input into the improved Actor network, which includes a structural path modeling layer, a heterogeneous resource gating layer, and a failure fallback path prediction layer, and outputs the module scheduling strategy distribution.
[0009] The joint state vector is input into an improved Critic network, which is a multi-objective residual evaluation structure, and outputs the state value evaluation result.
[0010] Based on the distribution of module scheduling strategies and the results of state value assessment, the scheduling node for each module is determined, edge-to-edge collaborative reasoning tasks are executed, and scheduling execution feedback data is collected.
[0011] Based on the joint state vector and scheduling execution feedback data, the A3C algorithm is used to update the parameters of the improved Actor network and the improved Critic network asynchronously. The structure-aware SGA algorithm is used to construct the advantage function by using the directional inner product of the policy gradient and the state value gradient, and a structure guidance term and orthogonal projection calibration mechanism are introduced.
[0012] Based on the scheduling execution feedback data and resource status changes, the original input data is periodically updated and the joint state vector is reconstructed to generate a new module scheduling strategy distribution.
[0013] Optionally, the data collection includes the original input data of the module dependency structure of the large model, resource state information of edge and end-side devices, and task scheduling constraint parameters, and constructs a joint state vector containing inference structure graph, resource state representation, and constraint conditions, specifically including:
[0014] The module dependency structure of the large model to be scheduled is collected. The module dependency structure is a directed graph structure that describes the execution order and input-output dependency relationship of each module. It includes module identifier, connection relationship, computation type and tensor interface information, and is organized in topological order to form an inference structure graph.
[0015] Collect resource status information from edge and end-side devices. The resource status information includes the remaining computing power, memory usage, bandwidth utilization, and current task execution status of the devices. Construct a resource status table by device dimension, and generate a resource status representation after converting it into a unified numerical domain using a linear normalization method.
[0016] Collect task scheduling constraint parameters, which include the maximum allowable delay of tasks, the setting order between modules, the device exclusion list and migration cost constraints, and encode them into a constraint condition vector after being configured by the scheduling controller.
[0017] The reasoning structure graph is encoded into a tensor-quantized structure representation, which is then aligned and concatenated with the resource state representation according to the module dimension to form a joint structure and resource representation. The constraint vector is then merged with the joint structure and resource representation by position expansion and broadcasting to construct a joint state vector.
[0018] Optionally, the improved Actor network includes a structural path modeling layer, a heterogeneous resource gating layer, and a failure fallback path prediction layer:
[0019] The structural path modeling layer receives the inference structure graph from the joint state vector, generates an adjacency matrix based on the adjacency relationship between modules, and obtains a two-hop path reachability matrix through two matrix exponentiation operations. It encodes the path depth, topological order, and module number into a path distance tensor, maps the module index to a position vector of a specific dimension, adds and fuses it with the path distance tensor, and generates an initial structural embedding representation through linear projection. Based on the multi-head self-attention mechanism, it models structural dependencies on the initial embedding representation. Each attention head outputs a module semantic vector, which is concatenated and normalized to output a structural awareness representation tensor.
[0020] The heterogeneous resource gating layer extracts computing frequency, remaining memory, communication bandwidth and current load features from edge and end-side devices. After normalization, these features are concatenated to generate a resource state matrix. The structure-aware representation tensor and the resource state matrix are broadcast and concatenated to obtain a joint tensor. Two layers of one-dimensional convolution are applied to the joint tensor and a sigmoid gating operation is introduced to output a module-device matching score matrix. After softmax normalization, a device selection weight vector is formed.
[0021] The failure fallback path prediction layer counts the failure frequency and failure type of each module from the historical scheduling execution records, generates a failure trend vector of the module using a sliding window mechanism, embeds the maximum latency, energy consumption limit and load tolerance in task scheduling into a representation, and concatenates them with the failure trend vector to form a composite feature tensor. A feedforward neural network is used to perform regression calculation on the composite feature tensor to output the failure probability distribution of each module on different candidate devices, and selects the device numbers with failure probabilities higher than a set threshold to form a fallback device set.
[0022] The device selection weight vector and the set of back-off devices are concatenated and fused together, and then fed into the fully connected mapping layer and the softmax normalization layer along with the structure-aware representation tensor. The output is the scheduling probability vector of each module on all candidate devices, which constitutes the module scheduling strategy distribution.
[0023] Optionally, the improved Critic network includes a target decomposition construction layer, a difference mapping modeling layer, and a residual fusion normalization layer:
[0024] The target decomposition construction layer performs independent channel embedding of the maximum tolerable latency, target energy consumption, and computational load boundary in the joint state vector to generate a target tensor. It encodes the resource state representation into a resource vector matrix containing three-dimensional features of frequency, storage, and bandwidth. It performs bilinear mapping and dimensional broadcasting concatenation on the target tensor and the resource vector matrix to output a target matching tensor.
[0025] The difference mapping modeling layer calculates the normalized difference tensor of the matching tensor between the current device state resources and the target. The normalized difference tensor is compared with the preset performance threshold through a soft masking mechanism, the bottleneck dimension is labeled, and a mask tensor is generated. The mask tensor and the normalized difference tensor are used as joint inputs, and a multi-scale one-dimensional convolution and attention compression mechanism is used to extract the low-dimensional conflict embedding tensor.
[0026] The residual fusion layer concatenates the conflict embedding tensor and the target matching tensor through channel splicing. It then uses a two-layer feedforward neural network and a skip connection mechanism to obtain the state embedding. Finally, it fuses the state embeddings of all modules using max pooling and mean pooling to output the state value evaluation result.
[0027] Optionally, the step of determining the scheduling node for each module based on the module scheduling strategy distribution and state value assessment results, executing edge-to-edge collaborative reasoning tasks, and collecting scheduling execution feedback data specifically includes:
[0028] The module scheduling strategy distribution and state value assessment results are jointly processed, and the scheduling node of each module is determined according to the module dependency order in the inference structure graph.
[0029] A confidence threshold filtering mechanism is adopted. The confidence threshold is the lower limit of the probability of candidate nodes in the scheduling policy distribution for each module. Nodes below the confidence threshold are considered to have insufficient scheduling stability and are removed.
[0030] A bandwidth priority ranking mechanism is introduced, which refers to constructing a priority table based on the network connection bandwidth between edge and end-side devices, and prioritizing the selection of devices with high bandwidth as deployment targets;
[0031] A module scheduling decision table is generated based on the confidence threshold filtering and bandwidth ranking results. It includes the deployment node and corresponding execution order of each module. According to the module scheduling decision table, each module of the large model is deployed to edge or end-side devices, and collaborative inference tasks are executed sequentially according to the dependency relationship of the inference structure graph.
[0032] Collect scheduling execution feedback data, which includes the average delay of module execution, changes in device resource utilization, and interruption records during the scheduling process.
[0033] Optionally, the asynchronous update of the parameters of the improved Actor network and the improved Critic network based on the joint state vector and scheduling execution feedback data using the A3C algorithm specifically includes:
[0034] The module average latency, resource usage change value and scheduling interruption flag in the scheduling execution feedback data are normalized, and after being aligned with the module topology order, they are concatenated with the joint state vector to form training samples.
[0035] The training samples are input into the improved Actor network and the improved Critic network respectively, and a local update operation based on the A3C algorithm is performed to obtain the local gradient vector.
[0036] Optionally, the asynchronous update method based on the A3C algorithm specifically includes:
[0037] Each thread calculates the policy loss and state value loss based on the module scheduling action sequence and scheduling execution feedback data obtained by independent sampling. The policy loss is constructed based on the log probability of each action and the normalized result of the scheduling execution feedback data. The state value loss is constructed based on the state value estimated by the current thread and the aligned cumulative feedback difference in the scheduling execution feedback data.
[0038] Each thread calculates a local gradient vector based on the corresponding policy loss and state value loss. The local gradient vector reflects the update direction of the parameters of the improved Actor network and the improved Critic network by the corresponding thread based on the independent sampling results.
[0039] Each thread writes its corresponding local gradient vector into the global parameter area used to store network parameters without waiting for other threads to complete gradient calculations, and then performs asynchronous updates to the global parameters of the improved Actor network and the improved Critic network based on the local gradients written.
[0040] Optionally, the structure-aware SGA algorithm is adopted, which constructs an advantage function using the directional inner product of the policy gradient and the state value gradient, and introduces a structure guidance term and an orthogonal projection calibration mechanism, specifically including:
[0041] After aggregating the local gradient vectors output by all threads, the structure-aware SGA algorithm is called for parameter optimization. The structure-aware SGA algorithm introduces a structure guidance term, which is derived from the inference structure graph. By calculating the adjacency weights of the dependency paths between modules, a structure constraint matrix is formed.
[0042] Construct an advantage function and use the directional inner product between the policy gradient generated by the improved Actor network and the state value gradient output by the improved Critic network as a metric.
[0043] After fusing the advantage function, an orthogonal projection calibration mechanism is introduced. A legal gradient space projection basis is constructed using a structure guidance term. The original update direction is mapped to a subspace that conforms to the structure dependency using a vector orthogonal decomposition method, and update components that conflict with the inference structure are eliminated.
[0044] The global update gradient, optimized by the structure-aware SGA algorithm and the orthogonal projection calibration mechanism, is applied to all parameters of the improved Actor network and the improved Critic network to achieve synchronous updates, and outputs the updated module scheduling strategy distribution and state value evaluation results.
[0045] Optionally, the step of periodically updating the original input data and reconstructing the joint state vector based on scheduling execution feedback data and resource state changes to generate a new module scheduling strategy distribution specifically includes:
[0046] Collect scheduling execution feedback data and current resource status change information of edge and end-side devices. The resource status changes include processor utilization, memory usage, task queue changes and power consumption levels, which constitute a set of feedback status.
[0047] The feedback state set is aligned with the original input data by field and time, and a sliding window mechanism is used for weighted smoothing to obtain the fused data.
[0048] Based on the fused data, a new module dependency structure graph, resource state representation tensor, and scheduling constraint vector are constructed. These are then encoded into a structured tensor of a unified dimension through a fully connected mapping, generating an updated joint state vector. This vector is then input into the improved Actor network to generate a new round of module scheduling policy distribution, which is used to refresh the execution instruction sequence of the task scheduling controller.
[0049] The update judgment is triggered by setting a dynamic threshold based on the magnitude of state change, and the data collection, state fusion and policy refresh process is executed periodically.
[0050] Optionally, setting the dynamic threshold based on the magnitude of state change specifically includes:
[0051] Based on the feedback data from multiple consecutive rounds of scheduling execution, the changes in processor utilization, memory usage, task queue depth, and power consumption level are calculated for each round, and the directionality of resource status changes is determined based on the sign of the changes in each round.
[0052] The rate of change per unit time is calculated using a sliding window mechanism to measure the change amplitude of each resource status in each round, which serves as a rate measure reflecting how fast the operating status of edge devices changes.
[0053] Based on whether the change direction of each resource status within the same sliding window is consistent, a trend correlation index of multidimensional resource status is calculated. The trend correlation is used to indicate whether the device fluctuates in the same direction simultaneously across multiple resource dimensions.
[0054] The weighted smoothing result of the change magnitude, the change rate, and the trend correlation index are combined according to a preset weight to generate a dynamic threshold. When the consistency of the change direction of the multidimensional resource status is higher, the threshold sensitivity is increased, and when the change rate is greater, the threshold trigger threshold is reduced.
[0055] When the magnitude or rate of change of any resource state exceeds the dynamic threshold, the original input data is updated and the joint state vector is reconstructed. If neither the magnitude nor the rate of change reaches the dynamic threshold, the current scheduling policy is maintained and the next policy refresh is postponed.
[0056] The beneficial effects of this invention are:
[0057] This invention provides a large-scale edge-end inference scheduling optimization method based on the A3C algorithm. Addressing the shortcomings of existing technologies, such as insufficient module dependency modeling, incomplete resource state awareness, lack of failure path prediction, lack of structural guidance for policy updates, and lagging feedback mechanisms, this invention proposes a comprehensive scheduling optimization system combining structure awareness and feedback-driven approaches. By constructing a joint state vector, it comprehensively integrates the inference structure graph, heterogeneous resource states, and scheduling constraints, achieving a fine-grained characterization of the edge environment. The introduction of structural path modeling, resource gating, and failure backoff prediction modules improves the interpretability and stability of the scheduling strategy. Updating the Actor-Critic network through a structure-aware SGA algorithm and orthogonal projection calibration mechanism ensures consistency between the strategy optimization direction and the model structure. Simultaneously, by combining scheduling feedback data and dynamic resource states, the state vector is periodically reconstructed and the scheduling strategy is updated, achieving continuous adaptation between the strategy and the environment. Experimental results show that the proposed method outperforms existing methods in terms of average inference latency, scheduling failure rate, and resource utilization, exhibiting stronger generalization ability and deployment robustness, and is suitable for large-scale model inference tasks in various heterogeneous edge scenarios. Attached Figure Description
[0058] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0059] Figure 1This is a flowchart of a large-model edge inference scheduling optimization method based on the A3C algorithm proposed in this invention;
[0060] Figure 2 This is a schematic diagram of the improved Actor network in the large-model edge inference scheduling optimization method based on the A3C algorithm proposed in this invention;
[0061] Figure 3 This is a schematic diagram of the improved Critic network in the large-model edge inference scheduling optimization method based on the A3C algorithm proposed in this invention. Detailed Implementation
[0062] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0063] refer to Figure 1-3 A method for optimizing edge inference scheduling in large models based on the A3C algorithm includes the following steps:
[0064] Collect raw input data including the module dependency structure of the large model, resource status information of edge and end devices, and task scheduling constraint parameters, and construct a joint state vector containing inference structure graph, resource status representation and constraint conditions;
[0065] The joint state vector is input into the improved Actor network, which includes a structural path modeling layer, a heterogeneous resource gating layer, and a failure fallback path prediction layer, and outputs the module scheduling strategy distribution.
[0066] The joint state vector is input into an improved Critic network, which is a multi-objective residual evaluation structure, and outputs the state value evaluation result.
[0067] Based on the distribution of module scheduling strategies and the results of state value assessment, the scheduling node for each module is determined, edge-to-edge collaborative reasoning tasks are executed, and scheduling execution feedback data is collected.
[0068] Based on the joint state vector and scheduling execution feedback data, the A3C algorithm is used to update the parameters of the improved Actor network and the improved Critic network asynchronously. The structure-aware SGA algorithm is used to construct the advantage function by using the directional inner product of the policy gradient and the state value gradient, and a structure guidance term and orthogonal projection calibration mechanism are introduced.
[0069] Based on the scheduling execution feedback data and resource status changes, the original input data is periodically updated and the joint state vector is reconstructed to generate a new module scheduling strategy distribution.
[0070] In this embodiment, the acquisition of raw input data includes the module dependency structure of the large model, resource status information of edge and end-side devices, and task scheduling constraint parameters. The construction of a joint state vector containing the inference structure graph, resource status representation, and constraint conditions specifically includes:
[0071] The module dependency structure of the large model to be scheduled is collected. The module dependency structure is a directed graph structure that describes the execution order and input-output dependency relationship of each module. It includes module identifier, connection relationship, computation type and tensor interface information. It is organized in topological order to form an inference structure graph, which is used to represent the structural dependency path between modules.
[0072] Collect resource status information from edge and end-side devices. The resource status information includes the remaining computing power, memory usage, bandwidth utilization, and current task execution status of the devices. Construct a resource status table by device dimension, and generate a resource status representation after converting it into a unified numerical domain using a linear normalization method.
[0073] Collect task scheduling constraint parameters, which include the maximum allowable delay of tasks, the setting order between modules, the device exclusion list and migration cost constraints, and encode them into a constraint condition vector after being configured by the scheduling controller.
[0074] The reasoning structure graph is encoded into a tensor-quantized structure representation, which is then aligned and concatenated with the resource state representation according to the module dimension to form a joint structure and resource representation. The constraint vector is then merged with the joint structure and resource representation by position expansion and broadcasting to construct a joint state vector.
[0075] In this embodiment, the improved Actor network includes a structural path modeling layer, a heterogeneous resource gating layer, and a failure fallback path prediction layer:
[0076] The structural path modeling layer receives the inference structure graph from the joint state vector, generates an adjacency matrix based on the adjacency relationship between modules, and obtains a two-hop path reachability matrix through two matrix exponentiation operations. It encodes the path depth, topological order, and module number into a path distance tensor, maps the module index to a position vector of a specific dimension, adds and fuses it with the path distance tensor, and generates an initial structural embedding representation through linear projection. Based on the multi-head self-attention mechanism, it models structural dependencies on the initial embedding representation. Each attention head outputs a module semantic vector, which is concatenated and normalized to output a structural awareness representation tensor.
[0077] The structural path modeling layer constructs a two-hop path matrix and a position embedding vector, and introduces a multi-head structural attention mechanism to achieve deep modeling of module dependencies. Compared with traditional shallow graph convolution methods, it can accurately capture cross-layer connections, module priorities and topological coupling features, and significantly improve the adaptability to complex inference graph structures in scheduling strategy generation.
[0078] The heterogeneous resource gating layer extracts computing frequency, remaining memory, communication bandwidth, and current load features from edge and end-side devices. After normalization, these features are concatenated to generate a resource state matrix. The structure-aware representation tensor and the resource state matrix are broadcast and concatenated to obtain a joint tensor of shape [N,M,D+R], where N is the number of modules, M is the number of devices, D is the structural feature dimension, and R is the resource feature dimension. Two one-dimensional convolutions are applied to the joint tensor, and a sigmoid gating operation is introduced to extract local matching relationships. The module-device matching score matrix is output and formed into a device selection weight vector after softmax normalization.
[0079] The heterogeneous resource gating layer introduces a module-device fusion tensor and gating attention mechanism to support dynamic mapping of modules to heterogeneous device resources. This avoids the resource mismatch problem caused by setting rules or static matching in traditional methods. It innovatively combines structure-aware representation and resource matrix to achieve optimal device selection under the three-dimensional constraints of computation, communication and structure, thereby improving the practicality and generalization ability of edge scheduling.
[0080] The failure fallback path prediction layer counts the failure frequency and failure type of each module from the historical scheduling execution records, generates a failure trend vector of the module using a sliding window mechanism, embeds the maximum latency, energy consumption limit and load tolerance in task scheduling into a representation, and concatenates them with the failure trend vector to form a composite feature tensor. A feedforward neural network is used to perform regression calculation on the composite feature tensor to output the failure probability distribution of each module on different candidate devices, and selects the device numbers with failure probabilities higher than a set threshold to form a fallback device set.
[0081] The failure fallback path prediction layer utilizes sliding window scheduling feedback embedding and task constraint fusion modeling. Through the feedforward risk prediction structure, it actively identifies potential failure paths and outputs fallback suggestion devices. Compared with passive fault tolerance mechanisms, it can provide fault tolerance guidance before scheduling decisions, which greatly improves fault avoidance capabilities and the robustness of edge systems.
[0082] The device selection weight vector and the set of back-off devices are concatenated and fused together, and then fed into the fully connected mapping layer and the softmax normalization layer along with the structure-aware representation tensor. The output is the scheduling probability vector of each module on all candidate devices, which constitutes the module scheduling strategy distribution.
[0083] In this embodiment, the improved Critic network includes a target decomposition construction layer, a difference mapping modeling layer, and a residual fusion normalization layer:
[0084] The target decomposition construction layer performs independent channel embedding of the maximum tolerable latency, target energy consumption, and computational load boundary in the joint state vector to generate a target tensor. It encodes the resource state representation into a resource vector matrix containing three-dimensional features of frequency, storage, and bandwidth. It performs bilinear mapping and dimensional broadcasting concatenation on the target tensor and the resource vector matrix to output a target matching tensor.
[0085] The difference mapping modeling layer calculates the normalized difference tensor of the matching tensor between the current device state resources and the target. The normalized difference tensor is compared with the preset performance threshold through a soft masking mechanism to mark the bottleneck dimension and generate a mask tensor to represent the scheduling conflict type, including excessive latency, energy consumption overflow or bandwidth bottleneck. The mask tensor and the normalized difference tensor are used as joint inputs. A multi-scale one-dimensional convolution and attention compression mechanism is used to extract the low-dimensional conflict embedding tensor, retaining key conflict information and suppressing redundant fluctuations.
[0086] The residual fusion layer concatenates the conflict embedding tensor and the target matching tensor through channel splicing. It then uses a two-layer feedforward neural network and a skip connection mechanism to obtain the state embedding. Finally, it fuses the state embeddings of all modules using max pooling and mean pooling to output the state value evaluation result.
[0087] This implementation significantly improves the state value assessment capability of the Critic network in heterogeneous edge environments by designing three modules: target decomposition and construction, difference mapping modeling, and residual fusion and normalization. The target decomposition and construction module introduces a module-device target matching tensor to achieve a unified representation of scheduling targets and resource states, enhancing the accuracy of scheduling modeling. The difference mapping modeling module extracts bottleneck features based on normalized differences and soft masking mechanisms, possessing strong scheduling conflict identification capabilities. The residual fusion and normalization module combines residual connections and a dual-channel pooling strategy to enhance the response to key risk factors while preserving global trends, significantly improving the accuracy and interpretability of state assessment results. The overall structure possesses comprehensive advantages such as strong structural adaptability, high conflict sensitivity, and excellent generalization ability.
[0088] In this embodiment, the step of determining the scheduling node for each module based on the module scheduling strategy distribution and state value assessment results, executing the edge-to-edge collaborative reasoning task, and collecting scheduling execution feedback data specifically includes:
[0089] The module scheduling strategy distribution and state value assessment results are jointly processed, and the scheduling node of each module is determined according to the module dependency order in the inference structure graph.
[0090] A confidence threshold filtering mechanism is adopted. The confidence threshold is the lower limit of the probability of candidate nodes in the scheduling policy distribution for each module. Nodes below the confidence threshold are considered to have insufficient scheduling stability and are removed.
[0091] A bandwidth priority ranking mechanism is introduced, which refers to constructing a priority table based on the network connection bandwidth between edge and end-side devices, and prioritizing the selection of devices with high bandwidth as deployment targets to reduce communication latency after scheduling.
[0092] A module scheduling decision table is generated based on the confidence threshold filtering and bandwidth ranking results. It includes the deployment node and corresponding execution order of each module. According to the module scheduling decision table, each module of the large model is deployed to edge or end-side devices, and collaborative inference tasks are executed sequentially according to the dependency relationship of the inference structure graph.
[0093] Collect scheduling execution feedback data, which includes the average delay of module execution, changes in device resource utilization, and interruption records during the scheduling process.
[0094] In this embodiment, the asynchronous update of the parameters of the improved Actor network and the improved Critic network based on the joint state vector and scheduling execution feedback data using the A3C algorithm specifically includes:
[0095] The module average latency, resource usage change value and scheduling interruption flag in the scheduling execution feedback data are normalized, and after being aligned with the module topology order, they are concatenated with the joint state vector to form training samples.
[0096] In a multi-threaded asynchronous environment, training samples are input into the improved Actor network and the improved Critic network respectively, and a local update operation based on the A3C algorithm is performed. The asynchronous update method refers to each thread independently sampling the action sequence and feedback trajectory based on the current policy, calculating its own policy loss and state value loss, and obtaining the local gradient vector.
[0097] In this embodiment, the asynchronous update method based on the A3C algorithm specifically includes:
[0098] Each thread calculates the policy loss and state value loss based on the module scheduling action sequence and scheduling execution feedback data obtained by independent sampling. The policy loss is constructed based on the log probability of each action and the normalized result of the scheduling execution feedback data. The state value loss is constructed based on the state value estimated by the current thread and the aligned cumulative feedback difference in the scheduling execution feedback data.
[0099] Each thread calculates a local gradient vector based on the corresponding policy loss and state value loss. The local gradient vector reflects the update direction of the parameters of the improved Actor network and the improved Critic network by the corresponding thread based on the independent sampling results.
[0100] Each thread writes its corresponding local gradient vector into the global parameter area used to store network parameters without waiting for other threads to complete gradient calculation. Based on the local gradients written after writing, the global parameters of the improved Actor network and the improved Critic network are updated asynchronously to maintain the parallelism and asynchronicity of the parameter update process.
[0101] In this embodiment, the structure-aware SGA algorithm is adopted, which constructs an advantage function using the directional inner product of the policy gradient and the state value gradient, and introduces a structure guiding term and an orthogonal projection calibration mechanism, specifically including:
[0102] After aggregating the local gradient vectors output by all threads, the structure-aware SGA algorithm is called for parameter optimization. The structure-aware SGA algorithm introduces a structure guidance term, which is derived from the inference structure graph. By calculating the adjacency weights of the dependent paths between modules, a structure constraint matrix is formed, which is used to apply topological consistency constraints to the directional distribution of the policy gradient.
[0103] An advantage function is constructed, using the directional inner product between the policy gradient generated by the improved Actor network and the state value gradient output by the improved Critic network as a metric. The directional inner product is calculated as the dot product of the normalized policy gradient and the state value gradient, reflecting the consistency of the update direction. It is used to screen high-quality update paths and serve as the optimization target of the structure-aware SGA algorithm.
[0104] After fusing the advantage function, an orthogonal projection calibration mechanism is introduced. A legal gradient space projection basis is constructed using a structure guidance term. The original update direction is mapped to a subspace that conforms to the structure dependency by using a vector orthogonal decomposition method. Update components that conflict with the inference structure are eliminated, thereby improving the stability and structural consistency of policy convergence.
[0105] The global update gradient, optimized by the structure-aware SGA algorithm and the orthogonal projection calibration mechanism, is applied to all parameters of the improved Actor network and the improved Critic network to achieve synchronous updates, and outputs the updated module scheduling strategy distribution and state value evaluation results.
[0106] By combining the asynchronous update mechanism of the A3C algorithm with the structure-aware SGA algorithm, the adaptive capability of the scheduling policy in dynamic edge environments is improved. The structure-aware SGA algorithm constructs the dominance function by the directional inner product of the policy gradient and the state gradient, and introduces a structure guidance term to strengthen module dependency constraint modeling, which can accurately identify key update directions and avoid policy drift. At the same time, the orthogonal projection calibration mechanism effectively reduces gradient interference and improves training convergence stability. Compared with traditional optimization methods, this method exhibits higher scheduling accuracy and energy efficiency in complex topology and heterogeneous resource scenarios.
[0107] In this embodiment, the step of periodically updating the original input data and reconstructing the joint state vector based on scheduling execution feedback data and resource status changes to generate a new module scheduling strategy distribution specifically includes:
[0108] Collect scheduling execution feedback data and current resource status change information of edge and end-side devices. The resource status changes include processor utilization, memory usage, task queue changes and power consumption levels, which constitute a set of feedback status.
[0109] The feedback state set is aligned with the original input data by field and time, and a sliding window mechanism is used for weighted smoothing to obtain the fused data.
[0110] Based on the fused data, a new module dependency structure graph, resource state representation tensor, and scheduling constraint vector are constructed. These are then encoded into a structured tensor of a unified dimension through a fully connected mapping, generating an updated joint state vector. This vector is then input into the improved Actor network to generate a new round of module scheduling policy distribution, which is used to refresh the execution instruction sequence of the task scheduling controller.
[0111] The system triggers update judgments by setting dynamic thresholds based on the magnitude of state changes, and periodically executes data acquisition, state fusion, and policy refresh processes to enable the system to quickly adapt to and continuously optimize changes in the edge inference environment.
[0112] In this embodiment, setting the dynamic threshold based on the magnitude of state change specifically includes:
[0113] Based on the feedback data from multiple consecutive rounds of scheduling execution, the changes in processor utilization, memory usage, task queue depth, and power consumption level are calculated for each round, and the directionality of resource status changes is determined based on the sign of the changes in each round.
[0114] The rate of change per unit time is calculated using a sliding window mechanism to measure the change amplitude of each resource status in each round, which serves as a rate measure reflecting how fast the operating status of edge devices changes.
[0115] Based on whether the change direction of each resource status within the same sliding window is consistent, a trend correlation index of multidimensional resource status is calculated. The trend correlation is used to indicate whether the device fluctuates in the same direction simultaneously across multiple resource dimensions.
[0116] The weighted smoothing result of the change magnitude, the change rate, and the trend correlation index are combined according to a preset weight to generate a dynamic threshold. When the consistency of the change direction of the multidimensional resource status is higher, the threshold sensitivity is increased, and when the change rate is greater, the threshold trigger threshold is reduced.
[0117] When the magnitude or rate of change of any resource state exceeds the dynamic threshold, the original input data is updated and the joint state vector is reconstructed. If neither the magnitude nor the rate of change reaches the dynamic threshold, the current scheduling policy is maintained and the next policy refresh is postponed.
[0118] Example 1:
[0119] To verify the feasibility of this invention in practice, it was applied to a typical industrial-grade intelligent edge computing platform to deploy and schedule a large, multi-module model. This model is a Transformer series model that integrates semantic segmentation and multimodal object detection tasks, with a total of over 110 million parameters and 23 sub-structures. The experimental scenario involved multiple edge and end-side heterogeneous devices collaboratively performing urban traffic monitoring tasks. The task input consisted of high-frequency video frames and sensor fusion data. The model was required to respond within 100ms, and the system exhibited typical edge environment characteristics such as bandwidth fluctuations and computing node jitter.
[0120] The actual deployment environment includes 4 edge servers and 12 end-side processing nodes. The performance of each node varies significantly, and resource status includes CPU availability, memory usage, communication bandwidth, and current task queue depth. The goal of task scheduling is to improve overall inference efficiency, reduce node failure rate, and increase resource utilization. To compare the results, a traditional static scheduling scheme is introduced as a control group. This scheme only allocates modules based on initial device load balancing, lacks dynamic feedback capabilities, and does not incorporate structural dependencies or failure prediction mechanisms.
[0121] During deployment, this invention first generates a directed graph-based inference structure diagram based on the module dependency structure of the large model, and constructs a joint state vector by encoding the resource status and task constraint parameters of each device. The scheduling controller inputs the state vector into the improved Actor network and Critic network, outputs the module scheduling policy distribution and state value evaluation results, and collects feedback data after each round of execution, including module-level average latency, interrupt records, power consumption data, and resource usage changes. Then, it performs asynchronous parameter updates through a structure-aware SGA algorithm. The entire scheduling policy is dynamically refreshed over multiple scheduling cycles, forming a feedback loop.
[0122] Table 1 Comparison of Core Performance Indicators
[0123]
[0124] As shown in Table 1, the method of this invention significantly outperforms traditional solutions in several key performance indicators. First, regarding average inference latency, because this invention can accurately model the structural dependencies between modules and optimize scheduling strategies in real time based on resource status, it significantly reduces latency caused by waiting or resource conflicts between modules, resulting in an overall average latency reduction of 36.5%. Second, regarding scheduling failure rate, this invention introduces a failure fallback path prediction mechanism, which can predict unstable nodes and avoid high-risk devices based on historical failure behavior, thus reducing the failure rate from 7.5% to 1.3%, significantly improving system stability. Regarding resource utilization, thanks to the combination of a heterogeneous resource gating layer and a device selection weighting mechanism, this invention can rationally allocate resources based on the remaining computing frequency and bandwidth of devices, effectively improving the utilization rate of edge devices, achieving a resource utilization rate of 85.4%, an improvement of more than 22 percentage points compared to traditional solutions.
[0125] Furthermore, regarding training efficiency, by introducing a structure-aware SGA algorithm combined with an orthogonal projection calibration mechanism, this invention achieves scheduling policy convergence within 470 rounds, far less than the 980 rounds required by traditional schemes, verifying that this method has faster optimization speed and stronger convergence in the policy learning phase. In terms of fault tolerance, traditional methods do not model fallback paths, and devices cannot be quickly switched after scheduling failures. This invention, however, constructs a fallback device set and automatically switches to a backup node when the failure probability exceeds a threshold. Therefore, the number of abnormal fallback triggers is significantly reduced from 27 to 6, significantly improving scheduling stability and fault recovery capabilities. Regarding overall task completion time, this invention continuously adapts to resource fluctuations through a dynamic policy refresh mechanism, rationally scheduling task execution order and node distribution, ultimately reducing the overall task processing time from 41.8 seconds to 27.5 seconds, resulting in a significant overall performance improvement.
[0126] Based on the deployment results, this invention exhibits the following significant advantages: First, it enables intelligent scheduling that integrates structure awareness and resource status, effectively adapting to heterogeneous edge environments. Second, it enhances system stability and robustness through backtracking path prediction and structure-guided updates. Third, the introduction of structural information and orthogonal projection into the strategy optimization process improves convergence speed and final performance. Fourth, the feedback-driven scheduling strategy refresh mechanism ensures the model's adaptability to dynamic resource changes during long-term deployment. These performance advantages make this invention widely applicable to complex environments requiring large-scale edge inference scheduling, such as intelligent manufacturing, edge video analytics, and multimodal perception systems.
[0127] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for optimizing edge inference scheduling in large models based on the A3C algorithm, characterized in that, Includes the following steps: Collect raw input data including the module dependency structure of the large model, resource status information of edge and end devices, and task scheduling constraint parameters, and construct a joint state vector containing inference structure graph, resource status representation and constraint conditions; The joint state vector is input into the improved Actor network, which includes a structural path modeling layer, a heterogeneous resource gating layer, and a failure fallback path prediction layer, and outputs the module scheduling strategy distribution. The joint state vector is input into an improved Critic network, which is a multi-objective residual evaluation structure, and outputs the state value evaluation result. Based on the distribution of module scheduling strategies and the results of state value assessment, the scheduling node for each module is determined, edge-to-edge collaborative reasoning tasks are executed, and scheduling execution feedback data is collected. Based on the joint state vector and scheduling execution feedback data, the A3C algorithm is used to update the parameters of the improved Actor network and the improved Critic network asynchronously. The structure-aware SGA algorithm is used to construct the advantage function by using the directional inner product of the policy gradient and the state value gradient, and a structure guidance term and orthogonal projection calibration mechanism are introduced. Based on the scheduling execution feedback data and resource status changes, the original input data is periodically updated and the joint state vector is reconstructed to generate a new module scheduling strategy distribution; The structure-aware SGA algorithm is employed, which constructs an advantage function using the directional inner product of the policy gradient and the state gradient, and introduces a structure guidance term and an orthogonal projection calibration mechanism, specifically including: After aggregating the local gradient vectors output by all threads, the structure-aware SGA algorithm is called for parameter optimization. The structure-aware SGA algorithm introduces a structure guidance term, which is derived from the inference structure graph. By calculating the adjacency weights of the dependency paths between modules, a structure constraint matrix is formed. Construct an advantage function and use the directional inner product between the policy gradient generated by the improved Actor network and the state value gradient output by the improved Critic network as a metric. After fusing the advantage function, an orthogonal projection calibration mechanism is introduced. A legal gradient space projection basis is constructed using a structure guidance term. The original update direction is mapped to a subspace that conforms to the structure dependency using a vector orthogonal decomposition method, and update components that conflict with the inference structure are eliminated. The global update gradient, optimized by the structure-aware SGA algorithm and the orthogonal projection calibration mechanism, is applied to all parameters of the improved Actor network and the improved Critic network to achieve synchronous updates, and outputs the updated module scheduling strategy distribution and state value evaluation results.
2. The large-model edge inference scheduling optimization method based on the A3C algorithm according to claim 1, characterized in that, The data collection includes the original input data of the module dependency structure of the large model, resource status information of edge and endpoint devices, and task scheduling constraint parameters. A joint state vector is constructed, comprising an inference structure graph, resource status representation, and constraint conditions. Specifically, this includes: The module dependency structure of the large model to be scheduled is collected. The module dependency structure is a directed graph structure that describes the execution order and input-output dependency relationship of each module. It includes module identifier, connection relationship, computation type and tensor interface information, and is organized in topological order to form an inference structure graph. Collect resource status information from edge and end-side devices. The resource status information includes the remaining computing power, memory usage, bandwidth utilization, and current task execution status of the devices. Construct a resource status table by device dimension, and generate a resource status representation after converting it into a unified numerical domain using a linear normalization method. Collect task scheduling constraint parameters, which include the maximum allowable delay of tasks, the setting order between modules, the device exclusion list and migration cost constraints, and encode them into a constraint condition vector after being configured by the scheduling controller. The reasoning structure graph is encoded into a tensor-quantized structure representation, which is then aligned and concatenated with the resource state representation according to the module dimension to form a joint structure and resource representation. The constraint vector is then merged with the joint structure and resource representation by position expansion and broadcasting to construct a joint state vector.
3. The large-model edge inference scheduling optimization method based on the A3C algorithm according to claim 1, characterized in that, The improved Actor network includes a structural path modeling layer, a heterogeneous resource gating layer, and a failure fallback path prediction layer: The structural path modeling layer receives the inference structure graph from the joint state vector, generates an adjacency matrix based on the adjacency relationship between modules, and obtains a two-hop path reachability matrix through two matrix exponentiation operations. It encodes the path depth, topological order, and module number into a path distance tensor, maps the module index to a position vector of a specific dimension, adds and fuses it with the path distance tensor, and generates an initial structural embedding representation through linear projection. Based on the multi-head self-attention mechanism, it models structural dependencies on the initial embedding representation. Each attention head outputs a module semantic vector, which is concatenated and normalized to output a structural awareness representation tensor. The heterogeneous resource gating layer extracts computing frequency, remaining memory, communication bandwidth and current load features from edge and end-side devices. After normalization, these features are concatenated to generate a resource state matrix. The structure-aware representation tensor and the resource state matrix are broadcast and concatenated to obtain a joint tensor. Two layers of one-dimensional convolution are applied to the joint tensor and a sigmoid gating operation is introduced to output a module-device matching score matrix. After softmax normalization, a device selection weight vector is formed. The failure fallback path prediction layer counts the failure frequency and failure type of each module from the historical scheduling execution records, generates a failure trend vector of the module using a sliding window mechanism, embeds the maximum latency, energy consumption limit and load tolerance in task scheduling into a representation, and concatenates them with the failure trend vector to form a composite feature tensor. A feedforward neural network is used to perform regression calculation on the composite feature tensor to output the failure probability distribution of each module on different candidate devices, and selects the device numbers with failure probabilities higher than a set threshold to form a fallback device set. The device selection weight vector and the set of back-off devices are concatenated and fused together, and then fed into the fully connected mapping layer and the softmax normalization layer along with the structure-aware representation tensor. The output is the scheduling probability vector of each module on all candidate devices, which constitutes the module scheduling strategy distribution.
4. The large-model edge inference scheduling optimization method based on the A3C algorithm according to claim 1, characterized in that, The improved Critic network includes a target decomposition construction layer, a difference mapping modeling layer, and a residual fusion normalization layer: The target decomposition construction layer performs independent channel embedding of the maximum tolerable latency, target energy consumption, and computational load boundary in the joint state vector to generate a target tensor. It encodes the resource state representation into a resource vector matrix containing three-dimensional features of frequency, storage, and bandwidth. It performs bilinear mapping and dimensional broadcasting concatenation on the target tensor and the resource vector matrix to output a target matching tensor. The difference mapping modeling layer calculates the normalized difference tensor of the matching tensor between the current device state resources and the target. The normalized difference tensor is compared with the preset performance threshold through a soft masking mechanism, the bottleneck dimension is labeled, and a mask tensor is generated. The mask tensor and the normalized difference tensor are used as joint inputs, and a multi-scale one-dimensional convolution and attention compression mechanism is used to extract the low-dimensional conflict embedding tensor. The residual fusion layer concatenates the conflict embedding tensor and the target matching tensor through channel splicing. It then uses a two-layer feedforward neural network and a skip connection mechanism to obtain the state embedding. Finally, it fuses the state embeddings of all modules using max pooling and mean pooling to output the state value evaluation result.
5. The large-model edge inference scheduling optimization method based on the A3C algorithm according to claim 1, characterized in that, Based on the module scheduling strategy distribution and state value assessment results, the scheduling node for each module is determined, edge-to-edge collaborative reasoning tasks are executed, and scheduling execution feedback data is collected. Specifically, this includes: The module scheduling strategy distribution and state value assessment results are jointly processed, and the scheduling node of each module is determined according to the module dependency order in the inference structure graph. A confidence threshold filtering mechanism is adopted. The confidence threshold is the lower limit of the probability of candidate nodes in the scheduling policy distribution for each module. Nodes below the confidence threshold are considered to have insufficient scheduling stability and are removed. A bandwidth priority ranking mechanism is introduced, which refers to constructing a priority table based on the network connection bandwidth between edge and end-side devices, and prioritizing the selection of devices with high bandwidth as deployment targets; A module scheduling decision table is generated based on the confidence threshold filtering and bandwidth ranking results. It includes the deployment node and corresponding execution order of each module. According to the module scheduling decision table, each module of the large model is deployed to edge or end-side devices, and collaborative inference tasks are executed sequentially according to the dependency relationship of the inference structure graph. Collect scheduling execution feedback data, which includes the average delay of module execution, changes in device resource utilization, and interruption records during the scheduling process.
6. The large-model edge inference scheduling optimization method based on the A3C algorithm according to claim 1, characterized in that, The method of asynchronously updating the parameters of the improved Actor network and the improved Critic network based on the joint state vector and scheduling execution feedback data using the A3C algorithm specifically includes: The module average latency, resource usage change value and scheduling interruption flag in the scheduling execution feedback data are normalized, and after being aligned with the module topology order, they are concatenated with the joint state vector to form training samples. The training samples are input into the improved Actor network and the improved Critic network respectively, and a local update operation based on the A3C algorithm is performed to obtain the local gradient vector.
7. The large-model edge inference scheduling optimization method based on the A3C algorithm according to claim 6, characterized in that, The asynchronous update method based on the A3C algorithm specifically includes: Each thread calculates the policy loss and state value loss based on the module scheduling action sequence and scheduling execution feedback data obtained by independent sampling. The policy loss is constructed based on the log probability of each action and the normalized result of the scheduling execution feedback data. The state value loss is constructed based on the state value estimated by the current thread and the aligned cumulative feedback difference in the scheduling execution feedback data. Each thread calculates a local gradient vector based on the corresponding policy loss and state value loss. The local gradient vector reflects the update direction of the parameters of the improved Actor network and the improved Critic network by the corresponding thread based on the independent sampling results. Each thread writes its corresponding local gradient vector into the global parameter area used to store network parameters without waiting for other threads to complete gradient calculations, and then performs asynchronous updates to the global parameters of the improved Actor network and the improved Critic network based on the local gradients written.
8. The large-model edge inference scheduling optimization method based on the A3C algorithm according to claim 1, characterized in that, The step of periodically updating the original input data and reconstructing the joint state vector based on scheduling execution feedback data and resource status changes to generate a new module scheduling strategy distribution specifically includes: Collect scheduling execution feedback data and current resource status change information of edge and end-side devices. The resource status changes include processor utilization, memory usage, task queue changes and power consumption levels, which constitute a set of feedback status. The feedback state set is aligned with the original input data by field and time, and a sliding window mechanism is used for weighted smoothing to obtain the fused data. Based on the fused data, a new module dependency structure graph, resource state representation tensor, and scheduling constraint vector are constructed. These are then encoded into a structured tensor of a unified dimension through a fully connected mapping, generating an updated joint state vector. This vector is then input into the improved Actor network to generate a new round of module scheduling policy distribution, which is used to refresh the execution instruction sequence of the task scheduling controller. The update judgment is triggered by setting a dynamic threshold based on the magnitude of state change, and the data collection, state fusion and policy refresh process is executed periodically.
9. The large-model edge inference scheduling optimization method based on the A3C algorithm according to claim 8, characterized in that, The specific steps of setting a dynamic threshold based on the magnitude of state change include: Based on the feedback data from multiple consecutive rounds of scheduling execution, the changes in processor utilization, memory usage, task queue depth, and power consumption level are calculated for each round, and the directionality of resource status changes is determined based on the sign of the changes in each round. The rate of change per unit time is calculated using a sliding window mechanism to measure the change amplitude of each resource status in each round, which serves as a rate measure reflecting how fast the operating status of edge devices changes. Based on whether the change direction of each resource status within the same sliding window is consistent, a trend correlation index of multidimensional resource status is calculated. The trend correlation is used to indicate whether the device fluctuates in the same direction simultaneously across multiple resource dimensions. The weighted smoothing result of the change magnitude, the change rate, and the trend correlation index are combined according to a preset weight to generate a dynamic threshold. When the consistency of the change direction of the multidimensional resource status is higher, the threshold sensitivity is increased, and when the change rate is greater, the threshold trigger threshold is reduced. When the magnitude or rate of change of any resource state exceeds the dynamic threshold, the original input data is updated and the joint state vector is reconstructed. If neither the magnitude nor the rate of change reaches the dynamic threshold, the current scheduling policy is maintained and the next policy refresh is postponed.