A microservice resource scheduling method for heterogeneous cloud-edge collaborative environments

Abstract

This invention discloses a microservice resource scheduling method for heterogeneous cloud-edge collaborative environments. It utilizes reinforcement learning to collect physical resource indicators and microservice topology constraints of heterogeneous edge nodes in real time, constructing a bimodal action space with numerical tensor-semantic description to obtain a set of valid candidate deployment actions. The method evaluates the value of these candidate actions and uses hard-constraint action masks to eliminate invalid nodes, outputting feasible and safe deployment actions. It compares the expected benefits of semantic suggestions and numerical strategies in the value evaluation and decision output deployment actions in real time, dynamically adjusting the decision weights by calculating the value difference between the two, and outputting the scheduling result. This invention addresses the modal disconnect between unstructured semantic planning and structured numerical control in existing methods by constructing a hierarchical heterogeneous collaborative mechanism.

Description

Technical Field

[0001] This invention relates to the field of cloud computing, edge computing and artificial intelligence operation and maintenance technology, and specifically to a microservice resource scheduling method for heterogeneous cloud-edge collaborative environments. Background Technology

[0002] With the rapid development of the Internet of Things (IoT) and 5G / 6G communication technologies, the computing paradigm is undergoing a historic shift from centralized cloud computing to ubiquitous collaborative computing across the cloud, edge, and endpoint. As the core carrier of cloud-native technology, microservices architecture significantly improves the agile delivery and horizontal scalability of software systems by decoupling traditional monolithic applications into a group of loosely coupled, single-function, and lightweight service units that interact via lightweight protocols (such as gRPC and RESTful). In typical heterogeneous cloud-edge collaborative scenarios, due to the high degree of constraint and heterogeneity of edge nodes in terms of computing resources (CPU instruction cycles, memory capacity), network bandwidth, and energy efficiency budgets, and the physical constraints of long transmission latency faced by abundant cloud resources, how to efficiently and reasonably map massive numbers of microservice instances to physical computing nodes has become a key bottleneck determining the system's end-to-end latency and service reliability. The multi-dimensional combinatorial optimization challenges of resource scheduling in production environments are essentially a high-dimensional, multi-objective, and highly dynamic NP-hard combinatorial optimization problem. Its complexity is mainly reflected in the following dimensions: Microservices have intricate call chains and data dependency topologies. Cross-node communication overhead fluctuates non-linearly with changes in deployment location, and resource bottlenecks in a single critical path service often lead to a service quality avalanche across the entire application through cascading failures. Dramatic fluctuations in user traffic distribution over time and space, random interference in wireless links, and dynamic online / offline states of heterogeneous nodes require the scheduling system to have millisecond-level real-time perception and response capabilities. The system needs to find the optimal Pareto optimality between ensuring strict service level agreements (SLAs, such as request response times below a threshold) and minimizing operational costs (energy consumption optimization, reduction of resource fragmentation). Traditional static rule-based (such as bin-packing) or metaheuristic algorithms (such as genetic algorithms and particle swarm optimization) often struggle in such highly dynamic environments due to their large search space and lack of online learning capabilities, easily falling into local optima or experiencing severe decision lag. Therefore, intelligent scheduling methods based on data-driven and deep reinforcement learning have gradually become the core technical path to solve the above problems.

[0003] The existing technology is as follows: 202410753704.X discloses a microservice deployment method that coordinates cloud and multiple edge network nodes. This method focuses on the cloud-edge collaborative scheduling problem under heterogeneous topology. First, it constructs a refined system model that covers microservice computing nodes, distributed databases, and various heterogeneous communication protocols, and sets end-to-end latency, computing resource consumption, and service deployment success rate as joint optimization objectives. In response to the pain point of low efficiency in traditional reinforcement learning algorithms in exploring a large discrete action space, this scheme innovatively introduces a Thompson Sampling with Clipped Gaussian Approximation exploration mechanism. This mechanism maintains the probability distribution of action value rather than a single deterministic value, and uses Bayesian posterior inference to achieve an adaptive balance between "exploitation" and "exploration," effectively avoiding the blindness of the epsilon-greedy strategy. In addition, to solve the credit allocation problem in sparse reward environments, this scheme introduces the Eligibility Traces strategy, which accelerates the propagation and convergence of the value function through a multi-step bootstrapping mechanism, significantly improving the learning efficiency of the model in complex dynamic environments.

[0004] Paper 202510594273.1 discloses an adaptive microservice deployment resource optimization method for cloud-edge-device collaboration. This method aims to address the problems of sparse microservice performance data, high annotation costs, and imbalanced samples in production environments, proposing a hybrid technical path of "generative augmentation + hyperheuristic optimization". At the data processing level, the method introduces a generative adversarial network (GAN) to perform distribution learning and augmentation generation on resource-performance data with small samples, constructing a high-quality synthetic dataset. Subsequently, a self-trained semi-supervised performance prediction model (SMPP) is trained based on this dataset, achieving accurate fitting of the nonlinear mapping relationship between microservice response time and resource configuration. At the decision level, when SMPP predicts potential QoS default risks, the scheme abandons a single heuristic rule and instead adopts an adaptive window search hyperheuristic algorithm (AWSH). This algorithm operates on the policy domain, dynamically managing and selectively scheduling a set of low-level heuristic operators (variants of genetic algorithms and ant colony algorithms). It switches the most efficient operator in real time based on environmental feedback, thus achieving a resource adjustment strategy that is more robust and generalizable than a single algorithm under dynamic load fluctuations.

[0005] Paper 202510920476.5 discloses a microservice deployment method for cloud-edge fusion environments based on multi-agent reinforcement learning. This method addresses the strong spatiotemporal dynamics of user requests by proposing a multi-agent reinforcement learning (MARL) deployment architecture that deeply integrates spatiotemporal feature extraction. At the state-aware layer, the method designs an LSTM-MHA (Long Short-Term Memory Network combined with Multi-Head Attention) composite neural network module. The LSTM layer is responsible for capturing the temporal dependency of request traffic, while the MHA mechanism focuses on extracting the spatial correlation between different edge nodes, thus constructing a full-spatiotemporal environmental state representation. Simultaneously, to eliminate redundant noise in the high-dimensional state space, a Variable Selection Network (VSN) is introduced to adaptively weight and filter the input features. At the decision layer, the method employs a multi-agent collaborative framework, decomposing the overall network deployment problem into multiple regional sub-problems. By designing a Critic network that incorporates an attention mechanism, agents can share observation information and evaluate the value of joint actions during training, thereby achieving synergistic optimization of long-term system energy efficiency and service latency in the decentralized execution phase.

[0006] Paper 202311621139.3 discloses a performance-aware microservice adaptive deployment and resource allocation method and system in a cloud-edge environment. This method proposes a resource allocation framework based on interference mining and game theory. Its core logic lies in explicitly modeling and handling interference and communication constraints between microservices. In the preprocessing stage, the method constructs a weighted execution dependency graph of microservices, uses spectral clustering to segment the graph, identifies service clusters with high-frequency interactions (strong I / O dependencies), and uses these clusters as soft constraints to favor deployment on the same node to minimize network overhead. Simultaneously, an exponential fitting interference prediction model is constructed to quantify the performance degradation caused by CPU cache contention and memory bandwidth competition when different microservices are co-located. In the decision-making stage, the scheme models the resource allocation problem as a Markov game and employs the MADDPG (Multi-Agent Deep Deterministic Policy Gradient) algorithm. The algorithm adopts a centralized training and distributed execution (CTDE) architecture, which enables each microservice agent to converge to the Nash equilibrium point through a game process while ensuring differentiated QoS, thereby achieving adaptive deployment that balances individual QoS guarantee and global resource utilization.

[0007] 202410753704.X A microservice deployment method for cloud and multi-edge network node collaboration focuses on improving the exploration efficiency of reinforcement learning. It innovatively introduces a Thompson sampling mechanism based on pruned Gaussian approximation, uses Bayesian posterior probability to replace the traditional epsilon-greedy strategy, and combines eligibility traces to accelerate value propagation in sparse reward environments. The main drawback of this method is that it still doesn't escape the framework of blind trial and error. Although Thompson sampling optimizes the statistical efficiency of exploration, in the early stages of training or when facing entirely new environments, the agent is essentially still in a random exploration phase without prior knowledge. In production-grade edge computing scenarios, this blind exploration may lead to critical services being scheduled to faulty or high-latency nodes, causing serious SLA defaults. Furthermore, this method is highly sensitive to environmental topology; once edge nodes are added or removed, or the network structure changes, the original value network becomes invalid, requiring time-consuming retraining, making it difficult to adapt to the rapid iteration of cloud-native environments.

[0008] The existing technology has the following problems: The main drawback of the method disclosed in 202510594273.1 lies in the uncertainty and latency caused by model complexity. First, while the introduction of GANs alleviates the problem of insufficient data, if the synthetic data fails to perfectly cover the long-tail distribution of the real scene, it will introduce undetectable systematic biases into subsequent prediction models. Second, hyperheuristic algorithms are essentially iterative search mechanisms. When faced with millisecond-level drastic load fluctuations, their search time often leads to delays in issuing decision instructions, failing to meet the stringent real-time requirements of edge computing.

[0009] The main drawback of the method disclosed in 202510920476.5 lies in its over-reliance on the accuracy of predictions. In the real world, user traffic is often highly sudden and unpredictable (such as sudden hotspot events). Once a "black swan" traffic surge occurs, the LSTM prediction model based on historical data will instantly fail, leading to a complete collapse of the scheduling strategy based on the prediction results. In addition, this solution also suffers from a serious "modal fragmentation" problem, utilizing only the spatiotemporal characteristics of traffic and resources while completely ignoring textual knowledge such as system logs and architecture documents, thus limiting the system's generalization ability.

[0010] The method disclosed in 202311621139.3 performs well in handling numerical interference metrics, but its main drawback lies in its lack of awareness of business semantics. The MADDPG algorithm can only process numerical state inputs such as CPU utilization and bandwidth usage, and cannot understand unstructured textual information such as the functional descriptions and data sensitivity levels of microservices. This "semantic blind spot" makes the model prone to making erroneous decisions lacking common sense when faced with services with similar numerical features but drastically different business constraints (such as ordinary log services and core payment services). In addition, as the scale of microservices grows, the state space of multi-agent games expands exponentially, making it difficult for the model to converge in complex dynamic scenarios, and online training is extremely computationally intensive.

[0011] Despite the multi-dimensional intelligent explorations made by the aforementioned existing technologies in the field of cloud-edge resource scheduling, common technical bottlenecks remain insurmountable when facing highly dynamic, heterogeneous, and complex production-level environments with complex business constraints. These bottlenecks include a widespread "semantic-numerical" modal separation, leading to a lack of business common sense in decision-making. Existing solutions (such as patents 202311621139.3 and 202510920476.5) primarily rely on numerical indicators such as CPU utilization and link bandwidth to construct the state space, completely ignoring textual semantic information such as service function descriptions and architecture documents. This "semantic blind spot" prevents the model from understanding complex hard business constraints such as data compliance and hardware affinity, causing the algorithm to easily make scheduling decisions that violate common sense when faced with services with similar numerical characteristics but vastly different business logics. Secondly, reinforcement learning is inefficient in the cold start phase and easily falls into conservative local optima. Although some solutions (such as patent 202410753704.X) introduce statistical exploration mechanisms, without prior knowledge guidance, the agent is essentially still in a state of blind trial and error. In edge environments with extremely high reliability requirements, RL agents often exhibit a strong risk-averse tendency to avoid SLA breach penalties caused by trial and error. They tend to transmit all tasks back to the cloud, thus losing the core advantage of low latency in edge computing. This leads to the system converging to an inefficient local optimum in a fully cloud-based deployment for a long time. Furthermore, they over-rely on model prediction and offline training, lacking real-time resilience to sudden events. Most solutions (such as patent 202510594273.1) are based on accurate predictions of future load or failure probabilities and rely on complex generative artificial intelligence (GANs) or temporal networks (LSTMs). However, real traffic is often highly bursty and unpredictable; once the prediction fails, the prediction-based scheduling strategy will completely collapse. In addition, the high coupling between deep models and specific topologies means that even minor environmental changes (such as adding or removing nodes) require expensive model retraining, failing to meet the agile iteration requirements of cloud-native architectures.

[0012] In summary, existing technical solutions exhibit a significant binary contradiction between numerical accuracy and semantic generalization. Traditional reinforcement learning algorithms focus on data-driven numerical fitting but lack contextual awareness of complex business logic, resulting in inefficient exploration within a vast state space and frequent suboptimal strategies due to a lack of guidance. On the other hand, while large language models excel at handling unstructured semantic information, their outputs possess inherent probabilistic uncertainty, lack strict constraints on underlying physical resource quotas, and their inference time is ill-suited to the millisecond-level response requirements of edge computing. The lack of a mechanism that organically couples the logical planning of the semantic layer with the precise control of the numerical layer is the core challenge hindering breakthroughs in current microservice resource scheduling performance.

[0013] Invention The purpose of this invention is to address the aforementioned problems by providing a microservice resource scheduling method for heterogeneous cloud-edge collaborative environments. By constructing a hierarchical heterogeneous collaborative mechanism, this invention solves the modal disconnect between "unstructured semantic planning" and "structured numerical control" in existing technologies. First, this invention utilizes domain prior knowledge from LLM to perform semantic pruning on the action space, reducing the trial-and-error cost and sample complexity of reinforcement learning in high-dimensional state spaces. Second, it uses the real-time state feedback loop of RL to dynamically calibrate the inherent static inference bias and uncertainty of generative models. Even under the complex dynamic load of heterogeneous cloud-edge environments, the system can achieve a dynamic balance between global semantic exploration and local numerical development through adaptive arbitration logic, thereby improving resource allocation efficiency while ensuring system robustness.

[0014] The technical solution adopted in this invention is as follows: A microservice resource scheduling method for heterogeneous cloud-edge collaborative environments, the method comprising: Semantic prior guidance and reinforcement learning are used to collect physical resource indicators and microservice topology constraints of heterogeneous edge nodes in real time, construct a bimodal action space with numerical tensor-semantic description, and obtain a set of effective candidate deployment actions; Value assessment and decision-making: The value of the acquired candidate deployment action set is assessed, and invalid nodes are eliminated by combining hard constraint action masks, and feasible and safe deployment actions are output. Dynamic hybrid arbitration compares the expected benefits of semantic suggestions and numerical strategies in the deployment actions of value assessment and decision output in real time. By calculating the value difference between the two, the decision weight is dynamically adjusted and the scheduling result is output.

[0015] Furthermore, the semantic prior guidance is constructed based on a large language model (LLM), wherein the collection of heterogeneous edge nodes specifically includes: Construct a hierarchically normalized multidimensional state vector Suppose that the edge computing cluster consists of N heterogeneous nodes, the first... t State vector at time step The tensor concatenation defined as the feature vectors of all nodes is as follows: (1) In the formula, For the first i Local feature vectors of each physical node For nodes i The normalized load pressure is used to quantify the current node's busy level; For nodes i Network topology cost coefficient between the network topology and the currently deployed microservice; For nodes i The resource capacity surplus rate is the ratio of the remaining resources of a node to the current demand of a microservice.

[0016] Furthermore, the semantically prior-guided microservice topology constraints specifically include a dynamic action masking process based on hard constraints for designing the action space, specifically including: Action space For discrete sets, corresponding to clusters N The index of each physical node introduces a dynamic action mask based on hard constraints, at each time step. t Generate a mask vector based on the current resource snapshot. The vector is logically determined as follows: (2) The set of physically feasible nodes is identified. At the output layer of the policy network, this mask is used to reshape the action probability distribution, as shown in the following equation: (3) The logical value of invalid actions is forced to be negative infinity, so that the probability of them being sampled is strictly zero; the search space of reinforcement learning (RL) is reduced from N dimensions to an effective K dimensions. ).

[0017] Furthermore, the microservice topology constraints also include a composite reward function that incorporates positive incentives and negative penalties. We construct a multi-objective optimization trade-off method, as follows: (4) In the formula, Given the link cost of the selected node, this guides the agent to choose the node with the lowest communication overhead, directly optimizing end-to-end latency. This is the standard deviation of the load on each node in the current cluster. This term serves as a regularization factor, penalizing the behavior of excessively concentrating the load on a single node. For indicator functions, This is the penalty coefficient.

[0018] Furthermore, the semantic prior guidance also includes a value evaluation network architecture built using a Dueling Network Architecture, specifically including: The architecture decouples the network output into a state-value stream V(S) and an action-advantage stream. The evaluation is divided into two parts: assessing the inherent advantages and disadvantages of the microservice environment and the relative benefits of specific actions, using a decentralized aggregation formula as follows: (5) The network output is decomposed into two streams: the state value function. Assess the inherent good or bad of the current environmental state S, and the action advantage function. Assess the specific action to take in state S. a Compared to the additional returns of the average strategy; minus the mean term This ensures that the state value stream V(S) can closely approximate the true value of the state.

[0019] Furthermore, the scheduling method also includes an enhancement scheme based on the Large Language Model (LLM) and employing domain instruction fine-tuning, specifically including: A domain-specific dataset including environment snapshots and expert decision pairs was constructed. Input Sample Semantic construction: The system converts real-time collected numerical state snapshots into structured natural language prompts; output labels. Expert knowledge injection: The label part consists of offline optimization algorithms and optimal policy experience during reinforcement learning RL inference, including the results and the thought chain reasoning process.

[0020] Furthermore, the optimization process based on domain instruction fine-tuning employs LoRA low-rank adaptive technology, specifically as follows: Model layer output h This can be formally expressed as: (6) In the formula, the main weights of the pre-trained model freeze, x Given an input vector, a low-rank factorization matrix is ​​trained only in the attention layer bypass. and ,in The fine-tuned forward propagation process is represented as: (7) LoRA employs an initialization strategy. Since matrix B is initialized to 0, at the initial moment of training (Step 0): (8) (9) This ensures that the model behaves exactly the same as the original pre-trained model at the start of fine-tuning, guaranteeing effective gradient backpropagation.

[0021] Furthermore, the objective function in the fine-tuning is to maximize the conditional probability, as shown in the following equation: (10) During the inference phase, the trained BA matrix can be compared with the original weights. To merge, that is It gains domain-specific knowledge enhancement without increasing inference latency, enabling the generalized LLM to learn microservice scheduling-specific terminology, topology understanding, and multi-objective trade-off logic, while retaining the model's strong generalization ability.

[0022] Furthermore, the value assessment and decision-making specifically include: At each decision-making step t The Dueling DQN agent receives a normalized numerical state vector. After forward propagation through the neural network, the action value distribution corresponding to each physical node is output. And select the benchmark action. The fine-tuned LLM receives semantic context description text. Heuristic suggestions for action are generated through thought chain reasoning. First, use action masks. right Perform hard constraint verification as follows: (11) If the verification fails, that is The LLM recommendation will be discarded directly, and a forced regression to the reinforcement learning (RL) baseline policy will be implemented to ensure the system's security at the physical level.

[0023] Furthermore, the dynamic hybrid arbitration specifically includes: If the LLM recommendation passes the validation, a mature value network trained using RL is used to perform cross-model value projection on the LLM recommendation and calculate its expected return. The final action is determined based on the following confidence level arbitration inequality. : (12) In the formula, For the quantitative value assessment of semantic suggestions, the heuristic actions generated by LLM based on semantic reasoning are characterized. The numerical result after mapping to the value function space learned by the RL agent.

[0024] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: This invention proposes a microservice resource scheduling method for heterogeneous cloud-edge collaborative environments. By constructing a two-layer architecture guided by large-scale model prior knowledge and using reinforcement learning for numerical optimization, it aims to solve the resource adaptation challenge of heterogeneous microservices under dynamic load environments. Addressing the characteristics of strong resource heterogeneity and complex task dependencies in cloud-edge collaborative computing, this invention designs a hybrid intelligent scheduling engine. The system first uses a lightweight large language model to semantically understand the current cluster state, such as remaining node resources, network topology, and characteristics of the microservices to be deployed, eliminating invalid or high-risk scheduling actions and generating a high-quality candidate action set. Subsequently, a Dueling DQN agent is used to perform refined policy search within the reduced action space, thereby minimizing system energy consumption and operating costs while ensuring service quality. Attached Figure Description

[0025] Figure 1 This is a diagram of the microservice deployment system architecture in the method of this invention; Figure 2 This is a schematic diagram of the LoRA fine-tuning technology in the method of this invention; Figure 3 This is a comparison chart of cloud costs for different deployment algorithms in the method of this invention under different system loads; Figure 4 This is a comparison chart of the average end-to-end latency under different system loads for different deployment algorithms in the method of this invention; Figure 5 This is a comparison chart of the total system energy consumption under different system loads for different deployment algorithms in the method of this invention; Figure 6 This is a comparison chart of network traffic consumption under different system loads for different deployment algorithms in the method of this invention; Figure 7 This is a comparison chart of the cloud costs of each module in the method of the present invention under different system loads; Figure 8 This is a comparison chart of the average end-to-end latency of each module in the method of the present invention under different system loads; Figure 9 This is a comparison chart of the total system energy consumption of each module in the method of the present invention under different system loads; Figure 10 This is a comparison chart of network traffic consumption of each module in the method of the present invention under different system loads; Figure 11This is a convergence graph of the LLM fine-tuning loss function in the method of this invention. Detailed Implementation

[0026] The present invention will now be described in detail with reference to the accompanying drawings.

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0028] As computing paradigms evolve towards ubiquitous cloud-edge collaboration, microservice-based distributed architectures have become mainstream. However, the strict resource constraints of edge nodes (heterogeneous computing power, bandwidth limitations) and the complex call chain topology of microservices pose significant challenges to resource scheduling. Existing technologies generally suffer from rigid decision-making models and insufficient adaptability when dealing with such dynamic and complex environments: traditional rule-based algorithms (such as greedy and heuristic algorithms) rely on static thresholds, lack understanding of business semantics, and struggle to cope with sudden traffic surges and topology changes; while deep reinforcement learning possesses dynamic adaptability, it faces severe blind exploration and cold start problems in the early stages of training, and its long convergence period easily leads to service SLA breaches; although LLM has strong logical reasoning capabilities, it suffers from the risk of illusion and is insensitive to real-time numerical indicators, making it difficult to guarantee the feasibility of physical resources.

[0029] To address the core technical challenges faced by microservice deployment in heterogeneous cloud-edge collaborative environments, such as rigid deployment strategies, fragmented awareness of heterogeneous resources, and limitations of single-model decision-making, such as... Figure 1 As shown, this invention proposes a microservice resource adaptive deployment method based on a two-layer collaborative arbitration architecture of "semantic prior guidance-numerical auditing". Through a heterogeneous collaborative mechanism, it organically integrates the generalized semantic understanding capability of LLM for business topology and architectural constraints with the precise control capability of deep reinforcement learning for underlying physical resources.

[0030] To address the convergence difficulties of reinforcement learning due to the lack of prior guidance and the inability of a single decision model to balance semantic reasoning breadth and numerical control accuracy, the main objective of this invention is to provide a heterogeneous cloud-edge microservice adaptive scheduling method and system that combines global business awareness with the determinacy of underlying physical boundaries. This invention proposes a two-layer collaborative arbitration architecture based on semantic prior guidance and real-time numerical auditing, abandoning single data-driven or single logic generation. The core design concept of this architecture is not to train a perfect, all-encompassing single model, but to construct a cross-modal balance and collaboration framework between a knowledge-driven semantic reasoning mechanism and a data-driven numerical decision-making mechanism. Addressing the inefficiency of traditional scheduling algorithms in exploring complex topologies, this invention does not rely on unguided trial and error starting from scratch, but instead utilizes a large language model for deep understanding of business logic and network topology. This invention employs a layered understanding approach, directly injecting high-value domain prior features at the initial decision-making stage. This mechanism effectively guides the system out of local extreme value domains, such as centralized cloud deployments, which traditional algorithms often fall into, greatly expanding the effective search space of strategies. Simultaneously, addressing the output uncertainty risk and logical distortion problems commonly found in generative large models in industrial control, this invention refuses to directly delegate scheduling execution to the semantic model. Instead, the system establishes a rigorous numerical audit and physical boundary verification mechanism at the underlying level. It focuses on real-time hard constraint filtering and value quantification of any upper-level semantic suggestions, thereby fundamentally avoiding the risk of illegal deployments that violate resource quotas. By introducing an adaptive arbitration logic based on the difference in expected returns, this invention achieves a layered closed-loop control: enabling the system to dynamically balance global exploration in the generalized semantic space and local optimization in the constrained numerical space based on its real-time operating status. This mechanism completely breaks the rigid trade-off between policy flexibility and execution security in traditional scheduling systems. When facing highly heterogeneous and dynamically changing cloud-edge environments, the system has both the planning ability to cope with long-tail complex scenarios and the control ability to ensure the bottom line of physical resources. Ultimately, it achieves a double leap in resource scheduling efficiency and system robustness.

[0031] Specifically, this invention relates to a microservice resource adaptive scheduling method that utilizes the semantic reasoning capabilities of Large Language Models (LLM) and the numerical decision-making capabilities of Deep Reinforcement Learning (DRL) for collaborative optimization. Its core technical logic includes: First, constructing a "semantic prior guidance module" based on Large Language Models (LLM); addressing the complex call chain dependencies and unstructured business constraints of microservices, this invention utilizes the semantic reasoning capabilities and CoT (Cooperation of Thought) mechanism of LLM to perform semantic-level preprocessing and pruning of the high-dimensional action space; this module does not directly output the final decision, but rather, based on an understanding of the business topology, filters out options that violate hard business rules, outputting a set of effective candidate actions containing high-confidence deployment suggestions, thereby reducing the search space from the original N dimensions to an effective K dimensions; Second, based on the numerical state evaluation and physical constraint guarantees of Dueling DQN, the Dueling DQN agent serves as the numerical optimization engine within the reduced candidate action set. This module handles real-time state vectors that fluctuate frequently, such as CPU load and network bandwidth, and quantifies the value assessment score (Q-Value) of each candidate action in the physical resource dimension. Simultaneously, a physical feasibility masking mechanism is introduced at the output layer to force the logical value of actions that violate underlying resource quotas to negative infinity, establishing deterministic hard constraint boundaries to ensure the physical legality of all output instructions. Finally, an adaptive confidence fusion arbitration mechanism based on dynamic risk thresholds is implemented. The expected returns of "semantic suggestions" and "numerical strategies" are compared in real-time within a closed-loop decision flow. The algorithm dynamically adjusts the decision weights by calculating the value gap between the two: when the potential long-term return of the LLM suggestion is significantly higher than the current RL strategy and passes security audit, the system adopts the semantic suggestion to escape local optima; otherwise, it adheres to the robust numerical strategy of RL.

[0032] Example This embodiment provides a microservice resource scheduling method for heterogeneous cloud-edge collaborative environments. The specific implementation process is as follows: By collecting physical resource metrics and microservice topology constraints from heterogeneous edge nodes in real time, a bimodal state space of "numerical tensor-semantic description" is constructed. This provides both accurate normalized numerical input for reinforcement learning and structured business context for large language models, solving the problem of the disconnect between underlying resources and upper-layer business perception. The specific process is as follows: In heterogeneous and complex cloud-edge computing environments, accurate and comprehensive perception of system status is a prerequisite for optimal scheduling. This embodiment addresses the microservice resource scheduling problem in heterogeneous cloud-edge environments by establishing a Constrained Markov Decision Process (CMDP) model. This model consists of a quadruple... The formal definition aims to learn a policy function that maps system states to optimal deployment actions through a data-driven approach. .

[0033] To overcome the information gaps caused by traditional methods relying on a single load metric, this embodiment designs a hierarchical normalized multidimensional state vector. Assuming the edge computing cluster contains N heterogeneous nodes, the state vector at time t is... Defined as a tensor concatenation of all node feature vectors: (1) In the formula, Let be the local feature vector of the i-th physical node. The physical meaning of each component is as follows: (Normalized Load Pressure): Represents the normalized load pressure of a node, used to quantify the current node's workload. Its calculation formula is: (13) A weighted average is used to combine CPU and memory loads to prevent single resource bottlenecks from being masked. The closer the value is to 1, the more congested the node. The RL agent should learn that... A higher probability decreases the likelihood of selecting that node.

[0034] (Topological Link Cost): Represents the network topology cost coefficient between a node and the microservice to be deployed. This metric quantifies the data transmission cost between the microservice to be deployed and its deployed predecessor dependent services or data sources. It not only measures physical distance but also implies network latency information. It is defined as follows: (14) In the formula, It is a dynamically changing relative value. This means excellent "data locality" (such as local deployment); This means transmission across wide area networks; this feature guides agents to inherently weigh computing resources against network overhead when making decisions, allowing agents to prioritize the use of nearby computing resources at the edge, thereby reducing end-to-end latency.

[0035] (Resource Margin Ratio): Represents the resource margin ratio of a node, i.e., the ratio of the node's remaining resources to the current microservice demand; its calculation aims to avoid the risk of "resource fragmentation," and the formula is: (15) Unlike the usual amount of surplus resources, It is a dimensionless ratio; The system indicates insufficient physical resources and absolutely cannot be deployed. This indicates that resources are plentiful; incorporating it into the state space helps the agent identify pseudo-idle nodes that are "although there are surpluses, they are insufficient to accommodate the current large-scale service"; the higher this indicator, the stronger the system stability after the node accepts the task.

[0036] After establishing the state representation, this embodiment designs a hard-constraint action masking mechanism for the action space to address the problem of frequent illegal actions generated during the early stages of reinforcement learning due to blind exploration; action space It is a discrete set, corresponding to the indices of N physical nodes in the cluster. To address the problem of training convergence difficulties caused by frequent "illegal actions" (i.e., scheduling to nodes with insufficient resources) due to blind exploration in the early stages of RL training, this embodiment introduces a dynamic action masking technique based on hard constraints. At each time step t, the system generates a mask vector based on the current resource snapshot. The vector is determined through logical judgment: (2) The system identifies the set of physically feasible nodes; at the output layer of the policy network, it uses this mask to reshape the action probability distribution. (3) The logical value of invalid actions is forced to be negative infinity, making its sampling probability strictly zero. This mechanism constructs a deterministic hard constraint boundary in the algorithm's decision manifold, which not only fundamentally eliminates the generation of illegal scheduling instructions such as resource overselling and ensures the physical feasibility of all issued instructions, but also reduces the search space of RL from N dimensions to an effective K dimension. ).

[0037] To guide the agent in finding the optimal strategy under multiple constraints, this embodiment further designs a composite reward function that includes positive incentives and negative penalties. This function is not a simple linear superposition, but rather constructs a multi-objective optimization trade-off mechanism: (4) The physical meaning and mechanism of action of each component are as follows: (Performance incentive) is the link cost of the selected node; this incentive guides the agent to choose the node with the lowest communication overhead (such as the edge) and directly optimize end-to-end latency. (Balanced incentive term) is the standard deviation of the load of each node in the current cluster; this term acts as a regularization factor, penalizing the behavior of over-concentrating the load on a single node, and promoting the long-term stability of the system. For indicator functions, The penalty coefficient is set to, for example, 10; once a decision causes the service response time to exceed the SLO threshold, a huge penalty is imposed; this reward and punishment mechanism forces the agent to maintain the bottom line of service quality when exploring high-yield strategies.

[0038] Finally, considering the characteristics of microservice scenarios where the value difference between actions is not obvious under certain conditions, such as overall cluster overload, this embodiment adopts a Dueling Network Architecture to construct a value evaluation network; this architecture decouples the network output into a state value flow V(S) and an action advantage flow. The evaluation is divided into two parts: assessing the inherent advantages and disadvantages of the environment and the relative benefits of specific actions; this is done using a decentralized aggregation formula. (5) The network output is decomposed into two streams: a state value function and a second stream. Assess the inherent goodness or badness of the current environmental state S (independent of specific actions); action advantage function Evaluate the additional benefit of taking a specific action a in state S compared to the average strategy; subtract the mean term. This ensures that V(S) can approximate the true value of the state; this means that the agent can keenly identify which performance fluctuations are caused by environmental degradation (captured by V) and which are caused by scheduling decisions (captured by A); this separation improves the robustness and convergence speed of the model in dynamic environments.

[0039] To enable general-purpose large language models (such as LLaMA and GPT-series) to possess expert knowledge and reasoning capabilities in the microservice resource scheduling domain, this embodiment adopts an enhancement scheme based on Domain-Specific Instruction Tuning. This scheme aims to allow the large model not only to understand physical concepts such as node load and bandwidth bottlenecks, but also to generate deployment suggestions that conform to engineering logic based on complex business descriptions. High-quality instruction data is the foundation of fine-tuning; this embodiment constructs a domain-specific dataset containing environment snapshots and expert decision pairs. The aim is to teach the model to map heterogeneous physical environments to reasonable scheduling strategies; input samples ( Semantic construction of ) : The system converts real-time collected numerical state snapshots into structured natural language Prompt; output labels ( The expert knowledge injection: The tag part consists of the optimal solution searched by offline optimization algorithms such as genetic algorithms, RL training, and the optimal strategy experience during inference, including decision results and the reasoning process of the thought chain.

[0040] Considering that full fine-tuning of LLM directly would result in huge computational resource consumption and storage pressure, in order to maintain the model's general semantic capabilities while giving it expertise in the field of microservice resource scheduling and deployment, this embodiment adopts LoRA (Low-Rank Adaptation) technology for model optimization.

[0041] like Figure 2 The core difference between full fine-tuning and LoRA fine-tuning in terms of weight update strategy is shown below: Full fine-tuning (left figure): In the traditional fine-tuning mode, the model needs to update all parameters of the pre-trained weight matrix W. Its forward propagation process is represented as follows: Among them, the weight update matrix The dimension is exactly the same as the original weight W. This approach not only consumes a lot of GPU memory, but also requires saving and loading complete model copies when switching between different downstream tasks, such as different microservice scenarios, resulting in high deployment costs. LoRA fine-tuning (right figure): LoRA is based on the assumption that the pre-trained model has low intrinsic dimensionality. It freezes the pre-trained weights W (gray squares in the figure, which do not participate in gradient updates) and simulates the change in weights by introducing bypass branches. Specifically, LoRA will It can be decomposed into the product of two low-rank matrices A and B, i.e. The dimension of the reduced-dimensional matrix A is Gaussian distribution is adopted Perform random initialization; increase the dimension of matrix B. Initialized as a matrix of all zeros ( The rank r is used as a hyperparameter. .

[0042] After applying LoRA, the output h of the model layer in this embodiment can be formally expressed as: (6) In the formula, the main weights of the pre-trained model Freeze, x is the input vector; train the low-rank factorization matrix only in the attention layers. and (in The fine-tuned forward propagation process is represented as follows: (7) To ensure the stability of the model's output during the initial training phase, LoRA employs a special initialization strategy; since matrix B is initialized to 0, at the initial training time (Step 0): (8) (9) This ensures that the model behaves exactly like the original pre-trained model when fine-tuning begins, thus avoiding performance collapse caused by random initialization and guaranteeing effective gradient backpropagation.

[0043] The objective function for fine-tuning is to maximize the conditional probability: (10) Using this method, the number of parameters that can be fine-tuned in this embodiment is typically only 1% or even less of the total number of parameters, significantly reducing memory overhead. During the inference phase, the trained BA matrix can be compared with the original weights. To merge, that is This allows for domain-specific knowledge enhancement without increasing inference latency, enabling the generalized LLM to learn specific terminology for microservice scheduling, topology understanding, and multi-objective trade-off logic, while retaining the model's strong generalization capabilities.

[0044] Numerical evaluation and decision-making based on the Dueling DQN architecture focuses on accurately evaluating the value of deployment actions. It also incorporates hard-constraint action masking technology to rigorously eliminate resource-insufficient invalid nodes from the algorithm's underlying layer, ensuring that all output strategies are absolutely feasible and secure at the physical level. The specific process is as follows: To address the challenge of balancing flexibility and security in a single model, this invention designs a two-layer collaborative arbitration mechanism. This mechanism establishes a dynamic coupling between unstructured semantic reasoning and structured numerical optimization. The specific workflow is as follows: At each decision time step t, the system triggers two decision streams in parallel: the Dueling DQN agent receives a normalized numerical state vector. After forward propagation through the neural network, the action value distribution corresponding to each physical node is output. And select the benchmark action. The fine-tuned LLM receives semantic context description text. Heuristic suggestions for action are generated through thought chain reasoning. To completely avoid the illusion risks that LLM may produce (such as recommending non-existent nodes or nodes that have exhausted their resources), the system first utilizes the aforementioned action mask. right Perform hard constraint verification.

[0045] (11) If the verification fails (i.e.) The system will discard the suggestion directly and force a fallback to the RL baseline policy, thereby ensuring the system's security at the physical level.

[0046] The value-based dynamic hybrid arbitration model operates a set of confidence-based dynamic arbitration logic. It does not mechanically execute a single model output, but rather compares the potential value (Q-value) of the RL baseline strategy with the LLM heuristic recommendations in real time. The system only adopts LLM recommendations when they pass a security audit and have significant expected benefits. This ensures the system's real-time responsiveness while effectively overcoming local optima bottlenecks and maximizing deployment benefits. The specific process is as follows: If the LLM recommendation passes validation, the system will proceed to the core value arbitration phase. At this stage, the system utilizes a mature value network trained with RL to perform cross-model value projection on the LLM recommendation and calculate its expected return. The final action will be determined based on the following confidence level arbitration inequality. : (12) In the formula, This is a quantitative value assessment of semantic suggestions, which characterizes the heuristic actions generated by LLM based on semantic reasoning. The numerical result is mapped to the value function space learned by the RL agent; even if LLM provides suggestions, the system must use historical experience to verify whether these suggestions are truly beneficial in the long run. Although The decision originates from logical reasoning based on textual descriptions, but the system must measure its long-term effectiveness based on objective performance metrics (such as latency and energy consumption, which are internalized in the Q-function). This is achieved through calculations... Achieving a unified quantitative measure from discrete semantic space to continuous value space ensures the comparability of heterogeneous decision sources. (Confidence threshold) is a non-negative hyperparameter. ).when At this time, this parameter introduces a hysteresis comparator. That is, it only works if the heuristic suggestion ( The expected return of the strategy is statistically significant, exceeding that of the benchmark strategy. The system only performs a policy switch when the threshold for error is reached. This effectively suppresses policy jitter caused by model prediction fluctuations, ensuring the control stability of the system in a steady-state environment. This mechanism allows the system to avoid getting trapped in local optima (e.g., when the RL algorithm gets stuck in a local optimum). When the overall value is low and a conservative strategy is adopted, the global perspective of LLM can be used to discover potential high-value actions, thereby achieving effective escape from local optima.

[0047] In summary, this embodiment constructs a joint decision boundary of semantic reasoning and numerical optimization at the mathematical level, ensuring that the system fully utilizes external expert knowledge to optimize decisions while strictly adhering to objective evaluation criteria based on long-term returns (RL Value).

[0048] Verification Example To verify the technical advantages of this invention, a high-fidelity cloud-edge collaborative experimental environment was constructed on the iFogSim simulation platform, and background loads of different intensities were introduced. To simulate real system pressure; this invention is compared with the following prior art solutions: Round Robin (RR) scheduling algorithm: This is a stateless, static scheduling strategy. Its strategy is to sequentially distribute all arriving microservice requests to available nodes in the cluster (including edge nodes and cloud nodes) in order. Once a request is assigned to the last node, the round-robin process restarts from the first node.

[0049] Worst Fit (WF) algorithm: This is a greedy strategy based on remaining resources. Its strategy is to iterate through all available nodes, calculate the remaining resources of each node, and always select the node with the most remaining resources to deploy the microservice, aiming to average out resource consumption within the cluster.

[0050] Best Fit (BF) algorithm: This is another classic greedy strategy, the opposite of WF. Its strategy is to find nodes that can satisfy the resource requirements of the microservice while minimizing remaining resources. This strategy attempts to compactly pack tasks into nodes to minimize resource fragmentation.

[0051] The default Kubernetes scheduler (K8s) uses a rule-based greedy strategy. This strategy involves two steps: first, filtering out nodes with insufficient resources; then, scoring the remaining nodes, prioritizing the node with the highest resource idle rate, and finally scheduling the task to the node with the highest score.

[0052] Ant Colony Optimization (ACO) is a probabilistic search algorithm that simulates swarm intelligence. Its strategy involves using "digital ants" to construct paths in the solution space. The ants select the next hop node based on the pheromone concentration along the path and heuristic information (such as node resources and link bandwidth). As iterations proceed, pheromones accumulate on high-quality paths, guiding the algorithm to converge to a better deployment scheme.

[0053] Genetic Algorithm (GA) is a global search algorithm that simulates the biological evolution process. Its strategy involves encoding the scheduling scheme as chromosomes, initializing the population, and iteratively optimizing the population's fitness function value by repeatedly performing selection, crossover, and mutation operations until the maximum number of iterations is reached, outputting the optimal solution in the current population.

[0054] The following are the definitions and calculations of the four key performance indicators for the experiment: Cloud Cost: This metric quantifies only the computing rental costs incurred when tasks are offloaded to the public cloud; edge nodes are considered zero-marginal-cost facilities. The calculation formula is as follows: (16) In the formula, This represents the set of all microservice tasks (Tuples) that have been assigned by the scheduling algorithm to be executed in the cloud data center. It is the computation length of a single task, measured in millions of instructions (MI). It refers to the processing power of cloud servers, measured in MIPS (Million Instructions Per Second). It is the unit price for cloud resource rental.

[0055] Total Energy Consumption: This metric is based on Dynamic Voltage Frequency Scaling (DVFS) theory and uses a linear power consumption model to calculate the total energy consumption of all physical nodes; the calculation formula is as follows: (17) Wherein, power function Defined as: (18) In the formula, It is the collection of all physical nodes in the system, including cloud servers, edge gateways, and edge nodes. It is the total duration of the experimental simulation. This represents the baseline power consumption of node n in the idle state. It is the peak power consumption of node n under full CPU load (100% utilization). It is the instantaneous CPU utilization of node n at time t. .

[0056] Average End-to-End Latency: This metric measures the total closed-loop time of a microservice application from data generation to processing completion; the calculation formula is as follows: (19) The latency of a single request is composed of three cumulative parts: (20) In the formula, This is the set of all application requests that were successfully completed within the experimental period. Timestamps for the raw data emitted by the sensor. The timestamp for when the executor receives the final processing result; This refers to the microservice call chain that request r traverses. The actual execution time of a microservice instance on the CPU. The queuing time of a task in the node scheduling queue. Network latency caused by data transmission between nodes.

[0057] Total Network Usage: This metric tracks the total amount of data transmitted across physical nodes and is used to assess the network bandwidth usage of scheduling strategies. The calculation formula is as follows: (twenty one) In the formula, This is a collection of all microservice instances; Let i be the set of all data tuples sent from microservice i to microservice j. The size of the k-th data tuple; The physical node ID where the microservice is currently located; For indicator functions, when When the function is active, its value is 1 (it generates network traffic); when... When the function value is 0 (local memory communication, not included in network traffic), the function value is 0.

[0058] As shown in Figure 3, calculations based on the cloud billing model indicate that this invention has the lowest cost among all compared solutions, while the baseline algorithms RR and WF have the highest costs. Compared to Kubernetes and various greedy strategies, this invention achieves optimal economic benefits by reducing unnecessary cloud offloading through more precise edge resource scheduling. Figure 4 As shown, traditional strategies such as RR, WF, and BF have extremely high latency, ranging from 470ms to 510ms. Although the latency of K8s is reduced, it is still significantly higher than that of intelligent algorithms. The average end-to-end latency of the present invention is reduced to about 230ms, and its latency performance is significantly better than that of traditional algorithms. It is also on par with offline global search algorithms GA and ACO under different loads. This proves that the present invention, as a decision-making method, has successfully achieved a service response speed close to the theoretical optimal solution.

[0059] like Figure 5 As shown, the total system energy consumption of the RR, WF, and BF algorithms remains high, ranging from 14.4 MJ to 14.8 MJ. The total system energy consumption of this invention is reduced to 13.2 MJ. Compared to the traditional benchmark, this invention achieves energy savings of approximately 10% or more. This invention effectively reduces unnecessary CPU idling and high-energy-consuming data transmission.

[0060] like Figure 6 As shown, the random strategy and RR generated a high traffic volume of approximately 4.0GB; even with BF and WF, the traffic remained around 3.9GB. The network traffic consumption of this invention is controlled at around 3.1GB; the minimum traffic consumption of this invention indicates that the scheduling strategy successfully achieves data locality optimization, tends to deploy microservices on nearby nodes, and significantly alleviates the bandwidth pressure on the core network.

[0061] Based on the above data, it was found that the best-fit algorithm (BF) performs reasonably well under low load, but under high load (… The previous approach resulted in poor decision-making quality—cloud costs surged to 8.7, while latency remained high at 500ms. This was because the greedy strategy over-filled edge nodes, leading to resource saturation and task queuing, forcing subsequent tasks to overflow to the cloud. This invention achieves both the lowest cost and lowest latency; this is thanks to the forward-looking planning capability of RL-LLM. The agent learns to reserve margins at edge nodes to cope with sudden traffic surges, thus avoiding congestion collapse in the BF algorithm and achieving true multi-objective Pareto optimization. Data shows that algorithms with high network traffic (e.g., RR, WF > 3.9GB) inevitably have high total energy consumption (> 14.4MJ). This confirms that cross-node data movement is one of the main sources of system energy consumption; this invention directly reduces total energy consumption by significantly reducing network flow. This indicates that this invention successfully extracts the topological dependencies between microservices, proactively deploying frequently interacting services within the same local area network, achieving system-level energy saving from the source. When the background load surged from 0.1 to 0.5, the performance metrics of the stateless algorithm (RR, Random) deteriorated drastically.

[0062] To further verify the effectiveness of the proposed large model-assisted reinforcement learning two-layer architecture and demonstrate the applicability of the lightweight model (3B) on the edge side, under the same background load intensity ( The system underwent module decoupling and model replacement, and the following ablation experiments were conducted: Pure RL: Only uses Dueling DQN, without LLM guidance; Pure LLM (3B / 14B): Only uses a large language model for direct scheduling, without real-time RL fine-tuning; Proposed (3B / 14B): The hybrid architecture of this invention, equipped with models with 3B and 14B parameters respectively.

[0063] like Figure 7 and Figure 9As shown, Pure RL performs worse than hybrid architectures in terms of cloud cost and energy consumption. This is because reinforcement learning is inefficient in exploring a large action space and has difficulty converging to the global optimum quickly. Although Pure LLM outperforms Pure RL in cost optimization due to its semantic reasoning capabilities, its average end-to-end latency is significantly higher. This indicates that relying solely on LLM lacks adaptability to dynamic environmental feedback (such as transient network congestion). The present invention (Proposed 3B) achieves optimal or near-optimal results in all metrics. Compared to Pure RL, its cost is reduced by approximately 15%; compared to Pure LLM, its latency is reduced by approximately 40%. Specifically, LLM is used for semantic pruning, pre-eliminating high-cost and constraint-violating poor-quality actions based on business logic, effectively constraining the search space of RL and significantly alleviating its inefficient exploration problem in sparse reward environments (reducing Cost / Energy). Simultaneously, the RL agent makes numerical decisions, utilizing high-frequency real-time interaction feedback with the environment to dynamically correct the biases and lags caused by LLM's static knowledge-based reasoning, successfully compensating for the high inference latency and insensitivity to dynamic loads of generative models (reducing latency). Figure 7 and Figure 9 As shown, Proposed (14B) does indeed demonstrate stronger logical reasoning capabilities. The scheduling strategy generated by the 14B model further reduces cloud costs and system energy consumption by approximately 3% to 5% compared to the 3B model. This indicates that models with a larger number of parameters can discover more refined resource optimization paths.

[0064] like Figure 8 In the comparison of average end-to-end latency data shown, there is no significant difference between Proposed (14B) and Proposed (3B); as Figure 10 The difference in network traffic between the two models is minimal (<2%), indicating that the 3B model is sufficient to capture the topological dependencies of microservices. This is a typical phenomenon of diminishing marginal utility. Although the 14B model has stronger inference capabilities and can save a small amount of additional cost, its inference overhead and memory usage become a heavy cost burden. In the cloud-edge collaboration scenario, Proposed (3B) achieves more than 95% of the decision quality of the 14B model with lower resource consumption and inference time, realizing the best balance between decision accuracy and inference speed. Therefore, this invention ultimately selects the 3B lightweight model as the best implementation scheme on the edge side.

[0065] To verify the domain adaptation potential of large language models in the field of microservice scheduling and to further demonstrate the rationality of the lightweight model (3B) selected in this invention, instruction tuning was performed on base models with different parameter amounts, and the loss curves during the training process were recorded.

[0066] like Figure 11 As shown, Figure 11 (a) and Figure 11 (b) shows the training convergence of the 3B lightweight model and the 14B large parameter model on the same dataset; where the light-colored curve represents the raw loss and the black solid line represents the smoothed loss.

[0067] like Figure 11 (a) and Figure 11 As shown in (b), in the initial stage of instruction fine-tuning (the first 100 steps), the loss functions of both parameter-scale models exhibit significant descent gradients. This convergence characteristic indicates that the model has efficient domain adaptability and rapidly completes feature transfer from the general semantic space to the microservice scheduling task domain. As the training process progresses, both convergence curves smoothly approach the low numerical range (Loss < 1.0). This strongly verifies that the "topology description-scheduling suggestion" instruction dataset constructed in this invention has high signal-to-noise ratio, and that the model establishes a robust generalization boundary during training without overfitting or gradient anomalies, successfully constructing a high-dimensional nonlinear mapping relationship from "business topology constraints" to "resource scheduling strategies".

[0068] Comparative analysis shows that the 14B model ( Figure 11 b) has a slightly lower final convergence error than the 3B model ( Figure 11 a) This phenomenon aligns with the scaling law of large models, meaning that a larger parameter scale leads to a deeper semantic representation capacity, enabling it to capture more subtle scheduling rule features. However, from a performance-efficiency perspective, the convergence trajectory of the 3B model demonstrates that it already possesses the domain representation completeness required to meet the complexity of current scheduling tasks. Considering the memory and computing power constraints of edge computing nodes, the slight loss advantage of the 14B model is offset by its larger inference latency (as shown in the ablation experiment described earlier). In contrast, the 3B model exhibits a perfect balance between excellent convergence and lower resource consumption. Therefore, this invention ultimately establishes the 3B model as the implementation subject on the edge side, achieving a significant reduction in system deployment resource overhead while ensuring scheduling intelligence.

[0069] In summary, this invention constructs a decoupled and parallel dual-modal environment perception system of "business semantics and physical numerical values": it abandons the complex topological feature extraction algorithms in traditional methods (such as complex operators or graph neural networks based on graph theory) and innovatively proposes a natural language topological understanding mechanism based on LLM. LLM directly transforms the call dependencies, affinity constraints, and business architecture descriptions of microservices into natural language text prompts. Utilizing the semantic reasoning capabilities of LLM (such as Qwen-3B), it directly understands the complex business topology and constraints from the text, generating candidate suggestions containing topological priors. RL collects physical resource indicators such as CPU and bandwidth of edge nodes in real time, standardizing them into high-dimensional numerical tensors for state mapping by the reinforcement learning agent (Dueling DQN). This decoupled perception architecture effectively avoids the high computational complexity of traditional topological algorithms while fully utilizing LLM's ability to understand unstructured business logic. This paper proposes a two-layer collaborative decision-making architecture combining large-scale model prior pruning and reinforcement learning numerical optimization: addressing the modal disconnect between semantic business planning and numerical resource scheduling in heterogeneous environments. The invention designs a two-layer collaborative decision-making mechanism. Using LLM (Limited Learning Model) based on semantic understanding of the business topology, preprocessing and pruning are performed in a large action space. By eliminating deployment options that violate hard business constraints or are logically infeasible, a convergent set of high-confidence, effective candidate actions is output. Then, a Dueling DQN (Dualizing Quantization) agent is used to evaluate the value of the candidate actions within the reduced candidate action subspace based on real-time physical resource states, selecting the final node that maximizes long-term cumulative benefits. This architecture leverages the prior knowledge of LLM to avoid blind exploration in reinforcement learning, while using feedback from reinforcement learning to compensate for the limitation of LLM's inability to perceive real-time load.

[0070] A hard-constraint secure execution mechanism based on action masks: To ensure the security and effectiveness of scheduling strategies in actual deployment, this invention introduces hard-constraint pre-filtering and lightweight adaptation techniques at the execution layer. In the decision output stage, action masking technology is introduced. The system detects the remaining resource quota of each edge node in real time and generates a dynamic mask vector to directly block all nodes with insufficient resources or abnormal states. This ensures that regardless of the strategy output by LLM or RL, the final issued instructions are always legal and secure at the physical level, fundamentally preventing scheduling failures or resource over-purchasing.

[0071] This article uses specific embodiments to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A microservice resource scheduling method for heterogeneous cloud-edge collaborative environments, characterized in that, The method includes: Semantic prior guidance and reinforcement learning are used to collect physical resource indicators and microservice topology constraints of heterogeneous edge nodes in real time, construct a bimodal action space with numerical tensor-semantic description, and obtain a set of effective candidate deployment actions; Value assessment and decision-making: The value of the acquired candidate deployment action set is assessed, and invalid nodes are eliminated by combining hard constraint action masks, and feasible and safe deployment actions are output. Dynamic hybrid arbitration compares the expected benefits of semantic suggestions and numerical strategies in the deployment actions of value assessment and decision output in real time. By calculating the value difference between the two, the decision weight is dynamically adjusted and the scheduling result is output.

2. The microservice resource scheduling method for heterogeneous cloud-edge collaborative environments according to claim 1, characterized in that, The semantic prior guidance is constructed based on the Large Language Model (LLM), and the collection of heterogeneous edge nodes specifically includes: Construct a hierarchically normalized multidimensional state vector Suppose that the edge computing cluster consists of N heterogeneous nodes, the th... t State vector at time step The tensor concatenation defined as the feature vectors of all nodes is as follows: （1） In the formula, For the first i Local feature vectors of each physical node For nodes i The normalized load pressure is used to quantify the current node's busy level; For nodes i Network topology cost coefficient between the microservice and the currently deployed microservice; For nodes i The resource capacity surplus rate is the ratio of the remaining resources of a node to the current demand of a microservice.

3. The microservice resource scheduling method for heterogeneous cloud-edge collaborative environments according to claim 1, characterized in that, The semantically prior-guided microservice topology constraints specifically include a process for designing dynamic action masks based on hard constraints for the action space, specifically including: Action space For discrete sets, corresponding to clusters N The index of each physical node introduces a dynamic action mask based on hard constraints, at each time step. t Generate a mask vector based on the current resource snapshot. The vector is determined logically as follows: （2） The set of physically feasible nodes is identified. At the output layer of the policy network, this mask is used to reshape the action probability distribution, as shown in the following equation: （3） The logical value of invalid actions is forced to be negative infinity, so that the probability of them being sampled is strictly zero; the search space of reinforcement learning (RL) is reduced from N dimensions to an effective K dimensions. ).

4. The microservice resource scheduling method for heterogeneous cloud-edge collaborative environments according to claim 3, characterized in that, The microservice topology constraints also include a composite reward function that incorporates positive incentives and negative penalties. We construct a multi-objective optimization trade-off method, as follows: （4） In the formula, Given the link cost of the selected node, this guides the agent to choose the node with the lowest communication overhead, directly optimizing end-to-end latency. This is the standard deviation of the load on each node in the current cluster. This term serves as a regularization factor, penalizing the behavior of excessively concentrating the load on a single node. For indicator functions, This is the penalty coefficient.

5. A microservice resource scheduling method for heterogeneous cloud-edge collaborative environments according to claim 3, characterized in that, The semantic prior guidance also includes a value evaluation network architecture built using Dueling Network Architecture, specifically including: The architecture decouples the network output into a state-value stream V(S) and an action-advantage stream. The evaluation is divided into two parts: assessing the inherent advantages and disadvantages of the microservice environment and the relative benefits of specific actions, using a decentralized aggregation formula as follows: （5） The network output is decomposed into two streams: the state value function. Assess the inherent good or bad of the current environmental state S, and the action advantage function. Assess the specific action to take in state S. a Compared to the additional returns of the average strategy; minus the mean term This ensures that the state value stream V(S) can closely approximate the true value of the state.

6. A microservice resource scheduling method for heterogeneous cloud-edge collaborative environments according to claim 1, characterized in that, The scheduling method also includes an enhancement scheme based on the Large Language Model (LLM) and employing domain instruction fine-tuning, specifically including: A domain-specific dataset consisting of environment snapshots and expert decision pairs was constructed. Input Sample Semantic construction: The system converts real-time collected numerical state snapshots into structured natural language prompts; output labels. Expert knowledge injection: The label part consists of offline optimization algorithms and optimal policy experience during reinforcement learning RL inference, including the results and the thought chain reasoning process.

7. A microservice resource scheduling method for heterogeneous cloud-edge collaborative environments according to claim 6, characterized in that, The optimization process based on domain instruction fine-tuning employs LoRA low-rank adaptive technology, specifically as follows: Model layer output h This can be formally expressed as: （6） In the formula, the main weights of the pre-trained model freeze, x Given an input vector, a low-rank factorization matrix is ​​trained only in the attention layer bypass. and ,in The fine-tuned forward propagation process is represented as: （7） LoRA employs an initialization strategy. Since matrix B is initialized to 0, at the initial moment of training (Step 0): （8） （9） This ensures that the model behaves exactly the same as the original pre-trained model at the start of fine-tuning, guaranteeing effective gradient backpropagation.

8. A microservice resource scheduling method for heterogeneous cloud-edge collaborative environments according to claim 7, characterized in that, The objective function in the fine-tuning is to maximize the conditional probability, as shown in the following equation: （10） During the inference phase, the trained BA matrix can be compared with the original weights. To merge, that is It gains domain-specific knowledge enhancement without increasing inference latency, enabling the generalized LLM to learn microservice scheduling-specific terminology, topology understanding, and multi-objective trade-off logic, while retaining the model's strong generalization ability.

9. A microservice resource scheduling method for heterogeneous cloud-edge collaborative environments according to any one of claims 1 to 8, characterized in that, The value assessment and decision-making specifically include: At each decision-making step t The Dueling DQN agent receives a normalized numerical state vector. After forward propagation through the neural network, the action value distribution corresponding to each physical node is output. And select the benchmark action. The fine-tuned LLM receives semantic context description text. Heuristic suggestions for action are generated through thought chain reasoning. First, use action masks. right Perform hard constraint verification as follows: （11） If the verification fails, that is The LLM recommendation will be discarded directly, and a forced regression to the reinforcement learning (RL) baseline policy will be implemented to ensure the system's security at the physical level.

10. A microservice resource scheduling method for heterogeneous cloud-edge collaborative environments according to claim 9, characterized in that, The dynamic hybrid arbitration specifically includes: If the LLM recommendation passes the validation, a mature value network trained using RL is used to perform cross-model value projection on the LLM recommendation and calculate its expected return. The final action is determined based on the following confidence level arbitration inequality. : （12） In the formula, For the quantitative value assessment of semantic suggestions, the heuristic actions generated by LLM based on semantic reasoning are characterized. The numerical result after mapping to the value function space learned by the RL agent.

Images