Network slice resource allocation method based on tsallis-maac

Abstract

The application discloses a network slice resource allocation method based on Tsallis-MAAC, comprising: acquiring real-time state characteristics and global resource states of each network slice; calculating dynamic priority weights of each slice; constructing a Tsallis divergence regularization term with asymmetric geometric morphology based on the dynamic priority weights and a reference strategy, wherein the priority weights are mapped as geometric morphology parameters to regulate the sensitivity of the strategy constraint; predicting SLA violation risk probabilities of each slice, combining the priority weights with the resource states to generate adaptive regularization coefficients; coupling the regularization term with the adaptive coefficients to obtain a total regularization loss, combining a strategy gradient target to construct a total strategy loss function, and updating and executing resource allocation strategies of each slice according to the total strategy loss function. The application realizes fine network slice resource allocation under differentiated SLA guarantee through the cooperation of the priority-driven geometric constraint and the risk-perceived adaptive adjustment double channels.

Description

Technical Field

[0001] This invention relates to the field of wireless communication and network resource management technology, and in particular to a network slice resource allocation method based on Tsallis-MAAC. Background Technology

[0002] In 6G pre-research, integrated space-ground computing and network convergence has become a key trend. LEO satellites, acting as distributed computing nodes, collaborate with terrestrial networks to provide end-to-end resource guarantees for differentiated services through network slicing. In this scenario, the highly dynamic changes in space-ground links and the highly limited onboard resources require resource allocation algorithms to possess high agility and adaptability. How to utilize multi-agent reinforcement learning technology to ensure high-priority service SLAs (Service Level Agreements) while also considering the overall resource utilization efficiency of the system is the key technology for achieving space-ground computing and network collaborative scheduling.

[0003] Current mainstream multi-agent reinforcement learning resource allocation schemes typically employ policy optimization frameworks based on entropy regularization (such as Soft-Actor-Critic, SAC) or fixed-coefficient penalty terms (such as the penalized proximal policy optimization algorithm PPO-Penalty) to balance exploration and utilization. Some schemes attempt to introduce static weights or rule-based priority mechanisms to weight resource requests from different slices. These methods perform reasonably well in terrestrial networks with relatively fixed topologies and stable service demands, achieving basic load balancing and resource scheduling.

[0004] However, in highly dynamic heterogeneous networks such as satellite-ground integrated networks, existing technologies have the following limitations: First, the geometric form of regularization constraints is singular and fixed. Existing methods mostly use fixed-form regularization terms, such as standard KL divergence or Shannon entropy, which are difficult to dynamically adjust the constraint sensitivity of the head and tail of the strategy distribution based on the rapid on / off of satellite-ground links and the drastic fluctuations in service priorities. This results in high-priority services struggling to quickly lock in resources during transit windows, or insufficient exploration of low-priority services. Second, there is a lack of adaptive adjustment mechanisms for risk perception. Existing solutions mostly rely on real-time state feedback adjustments, lacking the ability to predict SLA default risks. When the quality of satellite-ground links deteriorates due to weather or elevation angle, traditional fixed or linearly adjusted regularization coefficients are insufficient to form sufficiently strong nonlinear constraints on high-risk slices, easily triggering vicious resource contention among multiple slices. Third, the coupling strength of single-dimensional constraints is insufficient. Existing methods typically only simply superimpose constraints on amplitude, lacking deep coupling between geometric form and amplitude strength, making it difficult to provide strong protection with gating effects for critical services under extreme resource constraints. Summary of the Invention

[0005] The purpose of this invention is to provide a network slice resource allocation method based on Tsallis-MAAC to solve at least one of the aforementioned problems in the prior art.

[0006] Technical solution: A network slice resource allocation method based on Tsallis-MAAC, comprising:

[0007] Obtain the real-time network slice status characteristics of each network slice and the global resource status of the resource pool;

[0008] Based on the real-time network slice state characteristics, the dynamic priority weight of each network slice is calculated.

[0009] Based on dynamic priority weights and pre-maintained reference strategies, a first regularization term with an asymmetric geometric shape is constructed; wherein, the dynamic priority weights are mapped to the geometric shape parameters of the first regularization term to regulate the constraint sensitivity of the first regularization term to different regions of the strategy distribution.

[0010] Based on the real-time network slice state characteristics, predict the SLA default risk probability of each network slice, and generate adaptive regularization coefficients by combining dynamic priority weights and global resource status.

[0011] The policy gradient objective is calculated based on the historical interaction experience data of each network slice. The first regularization term is coupled with the adaptive regularization coefficient to obtain the total regularization loss. The total policy loss function is then constructed by combining the policy gradient objective.

[0012] The resource allocation strategy for each network slice is updated using the overall policy loss function, and resource allocation is performed based on the updated resource allocation strategy.

[0013] Beneficial effects: This invention achieves strong anchoring of high-priority business strategies and flexible release of exploration space for low-priority businesses, realizes strong preventive constraints before risks occur, ensures that scarce on-board computing power is prioritized for high-value slices, effectively suppresses ineffective resource contention for ordinary slices, and realizes refined network slice resource allocation under differentiated SLA guarantees. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the overall process of the network slice resource allocation method based on Tsallis-MAAC provided in the embodiments of this application.

[0015] Figure 2 This is a schematic diagram of the process for calculating the dynamic offset representing SLA performance pressure provided in the embodiments of this application.

[0016] Figure 3 This is a schematic diagram of the order parameter process for mapping dynamic priority weights to Tsallis divergence provided in an embodiment of this application.

[0017] Figure 4 This is a schematic diagram of the process for generating adaptive regularization coefficients by combining dynamic priority weights, provided in an embodiment of this application. Detailed Implementation

[0018] Example 1 describes the overall technical framework of the network slice resource allocation method based on Tsallis-MAAC (Tsallis divergence regularization multi-agent Actor-Critic), such as... Figure 1 As shown, an end-to-end resource allocation process is constructed, and a dual-channel priority modulation mechanism is used to solve the game equilibrium problem of multiple slices in a resource-constrained environment.

[0019] In this embodiment, the application scenario is set as a space-ground integrated computing and network convergence environment or a multi-tenant terrestrial mobile communication network environment. In this environment, there are N concurrent network slices, each controlled by an independent agent. The system architecture adopts a multi-agent reinforcement learning framework with centralized training and decentralized execution of CTDE, specifically including: each slice is configured with an independent actor network, an independent risk prediction network, and a shared or independent commentator network.

[0020] Step 101: Obtain the real-time network slice status characteristics of each network slice and the global resource status of the resource pool.

[0021] In this embodiment, the real-time network slice status feature is a composite vector characterizing the current operating status of the slice. Specifically, the system collects real-time Quality of Service (QoS) metrics (such as throughput, end-to-end latency, packet loss rate, and number of connections) and resource usage information (such as vCPU utilization, memory usage, and bandwidth consumption) for each slice from the network management plane. The collected raw data undergoes min-max normalization processing and is mapped to the [0,1] interval to eliminate dimensional differences. The normalized QoS features are concatenated with the resource features to form a dimension d. s slice state feature vectors s i (t).

[0022] Global resource status refers to the total capacity and current allocated amount of the underlying physical infrastructure resource pool (such as base stations, satellite nodes, and edge servers). The system aggregates the utilization ratios of computing resources, storage resources, and bandwidth resources to form a global resource utilization view.

[0023] Building upon this, this embodiment also predefines a discrete action space. For D types of allocable resources, such as computing power and bandwidth, the continuous allocation of each resource is uniformly quantized into K. dThere are 10 discrete tiers. For example, bandwidth resources are quantized into 10 tiers: {10%, 20%, ..., 100%}. All discrete tiers across all resource dimensions are combined using a Cartesian product to form a group of size |A| = |A|. d=1 D K d The joint discrete action space (|A| is the size of the joint discrete action space, i.e., the total number of actions). Each action index corresponds to a specific resource allocation combination scheme.

[0024] Step 102: Calculate the dynamic priority weight of each network slice based on the real-time network slice state characteristics.

[0025] In this embodiment, the dynamic priority weight α i (t) is the driving variable for subsequent dual-channel regularization. This weight is not a fixed value, but is jointly determined by the slice's static service category (such as ultra-reliable low-latency communication uRLLC, enhanced mobile broadband eMBB) and real-time SLA compliance pressure. The system calculates the dynamic offset in real time based on the deviation between the slice's QoS index and the SLA target at the current moment, and adds it to the static baseline weight. For example, when a slice's latency index is about to default, its dynamic priority weight will increase significantly, gaining a stronger resource competition advantage in subsequent steps. The specific calculation formula and process will be explained in detail in Example 2.

[0026] Before training begins, the reference policy for each slice is initialized to a uniform distribution in the action space; the experience replay buffer is pre-filled by executing a random policy, with a filling amount of 10 times the batch size; the risk prediction network is pre-trained using the experience data collected by the initial random policy, with a pre-training step count of 1000 steps.

[0027] Step 103: Based on the dynamic priority weights and the pre-maintained reference strategy, construct a first regularization term with an asymmetric geometric shape; wherein, the dynamic priority weights are mapped to the geometric shape parameters of the first regularization term to adjust the sensitivity of the first regularization term to the constraints of different regions of the strategy distribution.

[0028] In this embodiment, the first regularization term specifically uses Tsallis divergence. The geometric morphology parameter is specifically the order parameter q of the Tsallis divergence. The system uses a linear mapping function to obtain the dynamic priority weight α. i (t) is converted to the order parameter q iHigh-priority slices are mapped to larger q values ​​(q>1), making the Tsallis divergence more sensitive to the high-probability regions (peaks) of the policy distribution, strongly anchoring policies near the reference policy; low-priority slices are mapped to smaller q values ​​(q<1), making the Tsallis divergence more tolerant to the tails (low-probability regions) of the policy distribution, allowing it to explore more extensively.

[0029] Reference Strategy π ref It is a smoothed estimate of the historical behavior of the opposing intelligent agent group, maintained through the concept of virtual game theory. The specific order parameter mapping logic and Tsallis divergence calculation process will be explained in detail in Example 3.

[0030] Step 104: Predict the SLA default risk probability of each network slice based on the real-time network slice state characteristics, and generate adaptive regularization coefficients by combining dynamic priority weights and global resource status.

[0031] In this embodiment, the system uses a pre-built lightweight risk prediction network to perform forward inference on the slice state features and outputs the probability value p of SLA default occurring within a future period. risk This risk prediction network typically employs a lightweight, fully connected structure, such as an input layer -> a 64-neuron hidden layer -> a 32-neuron hidden layer -> a sigmoid output layer, to ensure real-time performance.

[0032] After obtaining the risk probability, the system maps it to an adaptive regularization coefficient λi using a parameterized sigmoid function. The shape (slope and center point) of this mapping function is controlled by dynamic priority weights, making high-priority slices more sensitive to risk. Furthermore, the system introduces global resource utilization as a macro-adjustment factor, amplifying the regularization coefficient as a whole when resources are scarce. The specific coefficient generation and adjustment logic will be detailed in Example 4.

[0033] Step 105: Calculate the policy gradient objective based on the historical interaction experience data of each network slice, couple the first regularization term with the adaptive regularization coefficient to obtain the total regularization loss, and construct the total policy loss function in combination with the policy gradient objective.

[0034] In this embodiment, the coupling method specifically adopts slice-by-slice multiplication coupling. That is, for each slice i, its total regularization loss R i (t) is calculated as follows:

[0035] R i (t)=λ i (t)*D q T (π i ||π i ref );

[0036] Among them, R i (t) represents the total regularization loss, λ i (t) represents the adaptive regularization coefficient generated in step 104, D q T The Tsallis divergence term constructed in step 103, with the superscript T indicating that the divergence is constructed based on the Tsallis entropy, is distinct from other divergence forms such as KL divergence, π i Let π be the current policy distribution of the agent in the i-th network slice at time t. i ref The reference policy corresponding to the i-th network slice (usually obtained by aggregating other agent policies or smoothing historical policies).

[0037] Through multiplicative coupling, this embodiment achieves dual-channel priority modulation of geometry and amplitude intensity. High-priority slices not only face stricter geometric constraints (determined by q), but also a larger amplitude penalty coefficient (determined by λ). The multiplication of the two produces a superlinear constraint enhancement effect.

[0038] Based on this, construct the total policy loss function L. i actor :

[0039] L i actor =L i PG +R i (t);

[0040] Among them, L i PG The standard policy gradient objective is used, such as PG loss based on the advantage function. During training, the system updates the actor network parameters by minimizing this total loss function.

[0041] Step 106: Update the resource allocation strategy of each network slice using the total policy loss function, and perform resource allocation based on the updated resource allocation strategy.

[0042] In this embodiment, given the discreteness of the action space, the policy update employs the Gumbel-Softmax reparameterization technique. Specifically, the actor network outputs the Logits vector l of the action. i In calculating the policy gradient L i PG At that time, Gumbel noise is introduced and Softmax relaxation is performed to generate a differentiable approximate one-hot action vector y:

[0043] y k =exp((l i,k+g k ) / τ) / Σ j exp((l i,j +g j ) / τ);

[0044] Among them, y k Let l be the differentiable approximate probability value corresponding to the k-th discrete action. i,k In the actor network, the subscript i represents the i-th network slice agent, the subscript k represents the k-th candidate discrete action, such as allocating a certain type of resource block, selecting a certain bandwidth, etc., and the subscript j is the index for traversing all possible discrete actions, g k The noise is sampled from the Gumbel(0,1) distribution, and τ is the temperature coefficient.

[0045] It should be noted that the regularization term D in calculation step 105... q T At that time, the standard Softmax probability distribution output by the actor network was used instead of the Gumbel sample values ​​to utilize the accurate probability distribution for constraint.

[0046] The updated policy network directly selects the action index with the highest probability during the execution phase, maps it to a specific resource allocation instruction, such as allocating 20% ​​bandwidth and 4 CPU cores, and issues it to the Network Slice Management Orchestrator (MANO) for execution. The environment feeds back new status and rewards based on the execution results, forming a closed loop.

[0047] According to one aspect of this application, in this embodiment, the actor network for each slice agent adopts a fully connected network structure. The input of this network is d. s A slice of state feature vectors in dimension d, where d s The value is determined by the sum of the number of QoS metrics M and the number of resource dimensions D, and is typically 5 to 15 in a typical network slicing scenario. The specific architecture of this network is as follows:

[0048] The first hidden layer contains 128 neurons, using the ReLU activation function; the second hidden layer contains 64 neurons, also using the ReLU activation function; the output layer contains |A| neurons, where |A| equals the size of the discrete action space, and outputs the original logit vector l. i (s i Then, it is converted into an action probability distribution π by the Softmax function. i (a|s i Each slice is configured with an independent actor network instance, meaning that actor networks in different slices do not share parameters, in order to capture the differentiated decision-making patterns of different slices. The network parameters are initialized uniformly using Xavier.

[0049] The system also maintains a target actor network with the same architecture as the current actor network for each slice. The parameters of the target actor network are not updated directly through gradient descent, but are slowly tracked through the Polyak soft update mechanism in Example 7, providing a stable policy benchmark for the temporal difference objective computation of the critic network.

[0050] In this embodiment, the QoS indicator dimension in the slice state feature vector is extracted and denormalized, and then compared with the SLA target parameters index by index to obtain the QoS compliance gap. The QoS compliance gap is scaled and truncated by the offset sensitivity coefficient, and then superimposed on the slice category baseline weight to obtain the dynamic priority weight.

[0051] The deterioration of the real-time operating status of a network slice directly drives the increase of its priority weight. The dynamic priority weight is transformed into the order parameter q of the Tsallis divergence through a linear mapping. The magnitude of the order parameter q directly determines the exponent of the power function in the Tsallis divergence formula, altering the sensitivity distribution of the divergence to different probability regions of the policy distribution. Higher slice priorities result in a larger order parameter, making the divergence more sensitive to deviations from high-probability action regions, and the policy is more strongly anchored near the reference policy.

[0052] The dynamic priority weights parameterize both the slope and center point of the sigmoid mapping function, controlling the shape of the response curve that transforms the SLA default probability into the regularized coefficient base value. The same level of default risk produces a larger coefficient output for higher priority slices and a smaller coefficient output for lower priority slices.

[0053] During the online execution phase, the SLA default labels are stored in a buffer along with the empirical tuples and periodically used as supervisory signals for incremental training of the risk prediction network. This training gradually brings the network's output default probabilities closer to the distribution of real default patterns, making the generation of adaptive regularization coefficients more accurate.

[0054] This creates a closed-loop positive feedback loop where actual default experience drives improvements in network prediction capabilities, improved prediction accuracy leads to more accurate regularization coefficients, and more accurate coefficients drive more effective strategy optimization.

[0055] In other implementations, the actor network may also employ a recurrent network architecture including LSTM layers to capture temporal dependencies, or a multi-head attention network to model interactions between slices. Furthermore, actor network parameters can be shared for slices of the same type to reduce training computational overhead.

[0056] Example 2 describes the specific calculation logic and establishment mechanism of hierarchical dynamic priority weights in network slice resource allocation. Through a hierarchical architecture that combines static baselines and dynamic offsets, it solves the technical problems that traditional fixed priorities are difficult to respond to real-time SLA fulfillment pressures and that pure dynamic priorities are prone to system oscillations, thus achieving robust and adaptive adjustment of slice priorities.

[0057] Step 201: Obtain the preset slice category baseline weight of the service category to which each network slice belongs; calculate the dynamic offset representing the SLA fulfillment pressure based on the real-time network slice status characteristics; superimpose the dynamic offset onto the slice category baseline weight, and obtain the dynamic priority weight through preset boundary constraint processing.

[0058] In this embodiment, the slice category baseline weight is a static value pre-set by the network operator based on the business value or Service Level Agreement (SLA) level of the service. This value reflects the macro-level position of different service categories in long-term resource competition. For example, for ultra-reliable low-latency communication (uRLLC) slices, which are more sensitive to latency, a higher baseline weight is usually assigned, specifically 0.6; for enhanced mobile broadband (eMBB) slices, which have high bandwidth requirements but relatively high tolerance, a baseline weight can be set to 0.3; and for massive machine-type communication (mMTC) slices, a baseline weight can be set to 0.1. This static baseline ensures that even when network load is low, high-priority services still maintain basic resource priority.

[0059] The dynamic offset is a correction value calculated based on the real-time operating status of the slice, used to reflect the degree of SLA default risk faced by the slice at the current moment. When the slice performance degrades and approaches the SLA threshold, the offset is positive, used to temporarily increase priority to seize resources; when the slice performance is far better than the SLA requirements, the offset is negative, used to proactively release resources to other slices that urgently need them.

[0060] The specific formulas for superposition and boundary constraint calculation are as follows:

[0061] α i (t)=max(w cls +clip(γ*g i (t),-δ,δ),α lb );

[0062] Where, α i (t) represents the dynamic priority weight of slice i at time t, w cls γ is the baseline weight of the category to which slice i belongs, clip is the truncation function, γ is the offset sensitivity coefficient, and g i (t) represents the dynamic offset characterizing SLA performance pressure, δ is the symmetric limit value of the offset, and αlb This is the lower bound of the priority weight.

[0063] In this formula, the truncation function limits the magnitude of the dynamic offset to within ±δ, preventing drastic priority jumps due to instantaneous network fluctuations and ensuring system stability. For example, δ can be set to 0.15, indicating that the dynamic adjustment magnitude will not exceed ±0.15 of the baseline weight. The parameter α... lb It is usually set to a small positive number, such as 0.01, to ensure that the priority of all slices is always greater than zero, avoiding division by zero errors or gradient vanishing problems in subsequent calculations. The sensitivity coefficient γ is used to control the system's response speed to SLA pressure, and its value is usually between 0.5 and 2.0.

[0064] Step 202: Obtain the preset SLA target parameters and corresponding direction identifiers; extract real-time QoS indicators from the real-time network slice state features, and calculate the normalized deviation between the real-time QoS indicators and the SLA target parameters; correct the sign of the normalized deviation using the direction identifiers to unify the optimization direction of different types of indicators; based on the preset indicator importance weights, perform weighted aggregation on the corrected normalized deviations to obtain the dynamic offset, such as... Figure 2 As shown.

[0065] In this embodiment, because the key performance indicators (KPIs) of network slicing have different physical dimensions and optimization directions, directly calculating the deviation makes it difficult to uniformly measure the performance pressure. For example, the throughput indicator is better the higher the value, while the end-to-end latency indicator is better the lower the value. To solve this problem, this embodiment introduces a direction identifier mechanism.

[0066] Specifically, for metrics where higher values ​​are better, such as throughput or number of connections, the direction identifier is set to +1; for metrics where lower values ​​are better, such as latency or packet loss rate, the direction identifier is set to -1. The system calculates the normalized deviation after unifying the direction using the following formula, and then weights and aggregates it to obtain the dynamic offset:

[0067] g i (t)=Σ m (ω m *d m *(QoS tgt_m -QoS real_m (t)) / QoS tgt_m );

[0068] Among them, g i (t) represents the dynamic offset of slice i, m is the index of the KPI indicator, and ω m Let d be the importance weight of the m-th indicator. m The direction identifier for the m-th metric, QoStgt_m Let m be the SLA target value for the m-th metric, and QoS be the QoS target value for the m-th metric. real_m (t) represents the current measured value of the m-th indicator.

[0069] To illustrate the calculation process more clearly, a specific numerical calculation example is provided below. Assume a uRLLC slice focuses on two metrics: end-to-end latency and throughput. Metric 1 is latency, measured in milliseconds, which is a metric where lower is better; therefore, d1 = -1. The SLA target value is QoS. tgt_1 =10ms, current measured QoS value real_1 =12ms. At this point, the measured value is greater than the target value, and the performance does not meet the standard. Substitute into the formula to calculate the deviation term:

[0070] Term1 = -1 * (10 - 12) / 10 = -1 * (-0.2) = +0.2. The result is positive, which correctly reflects the existence of a performance pressure of 0.2, or 20%.

[0071] Indicator 2 is throughput, measured in Mbps, which is a "the higher the better" indicator; therefore, d2 = +1. SLA target value QoS tgt_2 =100Mbps, current measured QoS value real_2 =80Mbps. At this point, the measured value is less than the target value, and the performance does not meet the standard. Substituting into the formula to calculate the deviation term: Term2=+1*(100-80) / 100=+1*(0.2)=+0.2. The result is also positive, which correctly reflects the existence of a 20% performance pressure.

[0072] Assuming the two indicators have equal weights, i.e., ω1 = ω2 = 0.5, then the final dynamic offset is:

[0073] g i (t) = 0.5 * 0.2 + 0.5 * 0.2 = 0.2.

[0074] If the baseline weight w of the slice cls =0.6, sensitivity coefficient γ=1.0, cutoff threshold δ=0.15, then the superimposed dynamic priority weights are:

[0075] α i (t)=0.6+clip(1.0*0.2,-0.15,0.15)=0.6+0.15=0.75.

[0076] As can be seen, due to the failure to meet the SLA metric, the priority of this slice was increased from 0.6 to 0.75, giving it an advantage in subsequent resource allocation.

[0077] In some alternative implementations, for the aggregation calculation of dynamic offsets, in addition to using a weighted summation method, a maximum value aggregation strategy can also be employed. Specifically, the system selects only the KPI with the largest deviation among all current KPI indicators as the overall performance pressure of the slice. This strategy is suitable for business scenarios that are sensitive to a single indicator, such as some industrial control slices, where failure to meet latency standards is considered a complete failure, regardless of throughput. In this case, g i (t)=max m (d m *(QoS tgt_m -QoS real_m (t)) / QoS tgt_m ).

[0078] Furthermore, when calculating the normalized bias, to avoid division-by-zero errors, for metrics with a target SLA value of zero, such as a packet loss rate target of 0%, the system adds a small smoothing term to the denominator, or uses the absolute difference instead of a relative ratio for calculation. In addition, for certain non-linear QoS-aware characteristics, a non-linear transformation can be applied to the normalized bias, for example, using an exponential function to amplify the pressure value near the default threshold, making the system more sensitive to critical states.

[0079] Example 3 describes the algorithm and mathematical principles for constructing asymmetric geometric regularization terms in a multi-agent game environment. By mapping dynamic priorities to the order parameters of Tsallis divergence, differentiated geometric constraints are achieved on the policy update trajectories of agents in different slices, effectively resolving the contradictory requirement that high-priority slices need to converge quickly while low-priority slices need to maintain exploration.

[0080] Step 301: The first regularization term is the Tsallis divergence regularization term defined based on the Tsallis relative entropy; mapping the dynamic priority weights to the geometric morphology parameters of the first regularization term specifically includes: mapping the dynamic priority weights to the order parameters of the Tsallis divergence; wherein, the order parameters determine the distribution of the penalty weights of the Tsallis divergence regularization term on high-probability action regions and low-probability action regions.

[0081] In this embodiment, the system uses the Tsallis relative entropy based on a q-th power function as the anchoring constraint for policy updates. Compared to standard KL divergence, Tsallis divergence introduces an additional degree of freedom, namely the order parameter q. This parameter directly changes the convexity and geometry of the divergence function, and can impose nonlinear penalty weights on different regions of the policy distribution.

[0082] Specifically, for slice i, its current policy distribution π i With reference strategy π refThe Tsallis divergence regularization term D between them qi T The calculation formula is as follows:

[0083] ;

[0084] Among them, D qi T Let π be the Tsallis divergence value. i (a|s i π represents the probability of the current action output by the actor's network. i ref (a) represents the action probability of the reference policy, q i Let be the order parameter, and ∑ represent the summation over all discrete actions in the action space A.

[0085] In this formula, the order parameter q i The physical meaning is very clear, when q i When x > 1, the power function x q Exhibiting a convex downward characteristic, the contribution of high-probability actions is significantly amplified. This indicates that the regularization term primarily penalizes the deviation between the head region (i.e., high-probability actions) in the policy distribution and the reference policy, forcing the primary decisions of high-priority slices to be strictly anchored near the equilibrium point, reducing unnecessary oscillations. Conversely, when q... i When <1, the power function x q More sensitive to changes in low-probability values, the weight of the regularization term shifts to the tail of the policy distribution. This indicates that the system allows low-priority slices to maintain a certain degree of conformity in major decisions, but encourages them to explore extensively in low-probability action regions to discover potential better solutions.

[0086] Step 302: Map the dynamic priority weights to the order parameters of the Tsallis divergence, such as... Figure 3 As shown, the process includes: obtaining a preset range of order parameter values ​​and a singularity avoidance threshold; converting dynamic priority weights into initial order parameters through linear mapping, such that the initial order parameters increase as the dynamic priority weights increase; detecting whether the initial order parameters fall within the singularity avoidance threshold range centered at 1; if they fall within the range, shifting and correcting the initial order parameters in a direction away from 1 to obtain the final order parameters, in order to avoid numerical singularities in Tsallis divergence calculation.

[0087] In this embodiment, in order to make the dynamic priority weight α i (t) is transformed into a specific geometric shape control signal. The system first defines the range of values ​​for the order parameter [q]. min ,q max For example, let q min =0.6, q max=1.8. The priority weight interval [α] is calculated using linear interpolation. min ,α max Mapping to the order parameter range:

[0088] q i raw =q min +(q max -q min )*(α i (t)-α min ) / (α max -α min );

[0089] Where, q i raw Let α be the initial order parameter. min and α max These are the minimum and maximum priority weights of all slices at the current time, respectively.

[0090] However, observing the Tsallis divergence formula reveals that the denominator contains (q) i -1) item. When q i As the expression approaches 1, the formula exhibits a division-to-zero singularity, which, although its limit is the KL divergence, can lead to overflow in numerical computation. This embodiment proposes a singularity avoidance mechanism. A small avoidance threshold ε is set. q For example, 0.05. The system detects q. i raw Does it fall within the interval [1-ε]? q ,1+ε q ].

[0091] The specific correction logic is as follows:

[0092] If 1-ε q i raw If q is less than 1, then q will be forcibly set. i =1-ε q ;

[0093] If 1≤q i raw <1+ε q Then force setting q i =1+ε q .

[0094] By performing the above-mentioned push-off operation, the monotonicity of the order parameter changing with priority is preserved, and the risk of instability caused by division by zero in numerical calculation is effectively reduced.

[0095] ​For example, suppose at a certain moment slice A has a priority of 0.9 (highest) and slice B has a priority of 0.1 (lowest). After mapping, slice A obtains q. A =1.8, its regularization term will pay close attention to the alignment of high-probability actions to ensure that key business operations do not deviate; slice B obtains q B =0.6, its regularization term is lenient towards tail exploration, allowing it to try unpopular resource combinations.

[0096] Step 303: The pre-maintained reference policy is updated in the following way: obtain the real-time policy distribution of agents in other network slices besides the current network slice; calculate the aggregated average policy of agents in other network slices as the opponent group policy; and use the exponentially weighted moving average method to smoothly update the reference policy of the previous time step based on the opponent group policy to obtain the reference policy of the current time step.

[0097] In this embodiment, the reference strategy π i ref The construction is based on virtual games in game theory. In order to find the Nash equilibrium in a non-stationary multi-agent environment, the agents should not only optimize for the instantaneous actions of their opponents, but also for the long-term average behavior patterns of their opponents.

[0098] Specifically, for slice i, the system first aggregates the policy distributions of all other slices j (j≠i) in the current state, and calculates the arithmetic mean as the aggregated policy π of the opponent group. -i (t):

[0099] π -i (a|t)=(1 / (N-1))*Σ j≠i π j (a|s j, t);

[0100] Where N is the total number of slices, π j (a|s j, t) represents the current policy distribution of the opponent slice j;

[0101] A reference strategy for updating slice i using the exponentially weighted moving average (EWMA) method:

[0102] π i ref (t)=β*π i ref (t-1)+(1-β)*π -i (t);

[0103] Here, β is the forgetting factor, which typically takes a value between 0.9 and 0.999 and is used to control the degree to which historical information is retained.

[0104] This update mechanism is equivalent to maintaining a model of the opponent's collective behavior for each agent. This is achieved by updating the current policy π. i Pulling towards this smoothly changing reference strategy π i ref The Tsallis regularization term actually suppresses the agent's overreaction to transient noise in the environment, guiding the system to converge to a stable equilibrium state.

[0105] In some optional implementations, to further enhance numerical stability, the system applies a lower bound protection to the probability values ​​in the reference policy. Specifically, after each EWMA update, the probability values ​​p of all actions are adjusted to max(p, ε). π ), where ε π It is a very small positive number, such as 10. -6 This operation prevents the probability of actions that have not been selected for a long time from decaying to zero, thus avoiding numerical errors when calculating the Tsallis divergence (which involves division or logarithmic operations).

[0106] According to one aspect of this application, in this embodiment, Tsallis divergence is chosen instead of standard KL divergence as the first regularization term for the following considerations: Standard KL divergence is mathematically equivalent to Tsallis divergence when the order parameter q approaches 1, and it does not possess the additional degree of freedom to adjust the constraint geometry through parameters. In multi-slice differentiated management scenarios, if all slices share the same form of KL divergence regularization, the policy anchoring constraints required by high-priority slices and the policy exploration degrees of freedom required by low-priority slices are difficult to satisfy simultaneously within the same regularization framework. Tsallis divergence, by introducing the additional degree of freedom of the order parameter q, makes the geometry of the regularization constraint itself a controllable dimension driven by priority, achieving differentiated constraint behavior among slices under a unified divergence framework.

[0107] When q i When >1, the power function term in the numerator of the Tsallis divergence formula This amplifies the contribution of high-probability actions to the summation, while the denominator... The division effect on the high-probability region of the reference strategy further amplifies the measurement of deviation in that region. The combined effect of these two factors makes the divergence more sensitive to deviations in the main part of the strategy distribution, i.e., the high-probability action region, which is equivalent to imposing a stricter equilibrium anchoring constraint on the decision region of the high-priority slice.

[0108] When q i When <1, the lower power of the power function weakens the dominance of high-probability actions, making the divergence more tolerant of changes in the tail of the distribution, i.e., the region of low-probability actions, and giving low-priority slices greater freedom in strategy exploration.

[0109] During backpropagation in the training phase, the Tsallis divergence varies with respect to the actor network parameters θ. i The gradient needs to be obtained through the current policy probability π. i (a|s i The partial derivative of the Tsallis divergence with respect to the probability components of a single action a is:

[0110] ;

[0111] in, Let q be the partial derivative of the Tsallis divergence with respect to the policy probability of action a. i Let π be the Tsallis order parameter of slice i. i (a|s i ) for actor network in state s i The softmax probability of output for action a, π i ref (a) represents the probability value of action a in the reference strategy.

[0112] The partial derivative is further propagated in a chain through the Jacobian matrix of the Softmax function to the logit vector of the actor network output layer, and then backpropagated through the fully connected layer to all network parameters. This gradient calculation process can be automatically completed by the automatic differentiation mechanism of mainstream deep learning frameworks.

[0113] According to one aspect of this application, the denominator in the above linear mapping formula includes (α) max -α min The denominator approaches zero when the dynamic priority weights of all slices are substantially equal, leading to a division-by-zero anomaly. The system checks α before performing the linear mapping. max With α min Is the difference less than the preset minimum positive number ε? α (For example, take 10) -6 If α max -α min Less than ε α If all slice priority weights are substantially equal, then linear mapping is not performed; instead, the order parameter of all slices is uniformly set to the median value of the range.

[0114] q i =(q min +q max ) / 2;

[0115] Where, q i Let q be the order parameter of slice i. min A lower bound is preset for the order parameter, q maxAn upper bound is preset for the order parameter. This median value ensures that all slices have the same regularized geometry when there is no difference in priority, which is a natural choice consistent with the semantics of treating all slices equally.

[0116] The aforementioned aggregated policy distribution is obtained by taking the arithmetic mean of the conditional policy distributions output by each adversary agent in its respective state. However, from a strictly probabilistic perspective, since these conditional distributions correspond to different states, their direct arithmetic mean does not constitute a valid conditional probability distribution, but is merely an approximate estimate of the marginal action tendency of the adversary group.

[0117] This approximation method is widely used in Mean-Field multi-agent reinforcement learning frameworks, mainly due to the following considerations: Under the same network slice operating environment, the state distribution faced by each slice agent has obvious correlation - all are driven by the same underlying physical resource pool capacity and macro-level business load, and the output of the opponent's policy under different states exhibits an aggregateable behavior pattern; the reference policy in this invention is positioned as the anchor point of Tsallis divergence regularization, that is, the soft constraint target of policy update, rather than the basis for accurate probabilistic inference, and the accuracy requirement is lower than that of the probability estimation task; the smoothing mechanism of exponentially weighted moving average further reduces the impact of aggregation error at a single time step on the stability of the reference policy, so that even if there is a certain deviation in the aggregation at a certain time step, the actual impact on the reference policy after exponential decay smoothing by EWMA is very limited.

[0118] In other implementations, the first regularization term may also be a family of generalized divergences with adjustable order parameters, such as the Rényi divergence, whose order parameters can also be driven by dynamic priority weights.

[0119] Example 4 describes the specific implementation of the amplitude channel in the dual-channel regularization mechanism, constructing a complete risk perception and coefficient adjustment system. It uses a lightweight network to predict SLA default risk in real time and leverages a parameterized Sigmoid mapping and macro-resource feedback mechanism to dynamically generate adaptive regularization coefficients highly adapted to slice priorities. This mechanism ensures that high-priority slices receive strong game equilibrium constraints when risk increases, while low-priority slices maintain strategic flexibility when resources are abundant.

[0120] Step 401: Input the real-time network slice state features into the pre-constructed lightweight risk prediction network; through the forward inference of the lightweight risk prediction network, output a value representing the probability of service level agreement (SLA) default in the current state, as the SLA default risk probability.

[0121] In this embodiment, the lightweight risk prediction network, independent of the actor and critic networks, is used to capture the nonlinear mapping between slice states and SLA defaults. To reduce computational overhead and meet real-time requirements, the network employs a shallow, fully connected architecture.

[0122] Specifically, the input layer receives data in dimension d. s The slice state feature vector; the first hidden layer contains 64 neurons and uses the ReLU activation function; the second hidden layer contains 32 neurons and also uses the ReLU activation function; the output layer contains 1 neuron and uses the Sigmoid activation function to map the output to the [0,1] interval.

[0123] In actual operation, the system will normalize the slice state feature vector s i (t) Input the network, perform forward propagation computation, and output a scalar value p between 0 and 1. i risk (t). This value quantifies the probability that a slice will fail to meet QoS standards, i.e., breach the SLA, in the future under its current state. For example, an output value of 0.85 indicates that the slice faces a high risk of breach, and the system needs to intervene immediately.

[0124] Step 402: Generate adaptive regularization coefficients by combining dynamic priority weights, such as... Figure 4 As shown, the process includes: constructing a parameterized Sigmoid mapping function, where the slope parameter and center point parameter of the parameterized Sigmoid mapping function are functions of dynamic priority weights; adjusting the slope parameter and center point parameter using the dynamic priority weights so that higher priority weights correspond to steeper slopes and lower center point thresholds; and using the adjusted parameterized Sigmoid mapping function to map the SLA default risk probability to the base value of the adaptive regularization coefficient, so as to achieve differentiated risk responses to different priority slices.

[0125] In this embodiment, to achieve differentiated risk response, the present invention proposes a priority-parameterized Sigmoid mapping mechanism. Unlike traditional fixed-shape mapping functions, the geometry of the mapping function in this embodiment dynamically changes with the slice priority.

[0126] Specifically, the slope parameter k(α) of the Sigmoid function is defined. i (t) and center point parameter μ(α) i (t) represents the dynamic priority weight α. i A linear function of (t):

[0127] k(α i (t))=k0+k1*α i (t);

[0128] μ(α i (t))=μ0-μ1*α i (t);

[0129] Where k0 is the slope reference value, k1 is the slope adjustment coefficient, μ0 is the center point reference value, and μ1 is the center point offset coefficient.

[0130] Based on this, the base value λ of the regularization coefficient is calculated. i base (t):

[0131] λ i base (t)=λ max / (1+exp(-k(α i (t))*(p i risk (t)-μ(α i (t)))));

[0132] Where, λ i base (t) represents the base value of the regularization coefficient for slice i, λ max p is the upper limit of the coefficient. i risk (t) represents the SLA default risk probability output in step 401, and exp is the exponential function.

[0133] In a certain scenario, suppose the preset parameter is: λ max =1.0, k0=5, k1=10, μ0=0.8, μ1=0.4.

[0134] Scenario 1: High-priority slice A, with priority weight α A =0.8. The slope k is calculated. A =5 + 10 * 0.8 = 13 (very steep), center point μ A =0.8 - 0.4 * 0.8 = 0.48 (the threshold is relatively low). If the current risk probability p risk =0.5 (medium risk). Since 0.5 > 0.48 and the slope is relatively large, substituting into the formula, we can calculate λ. A base ≈0.56. This indicates that for high-priority slices, even if the risk just exceeds the midline, the system will quickly apply strong regularization constraints.

[0135] Scenario 2, low-priority slice B, with priority weight α B =0.2. The slope k is calculated. B =5 + 10 * 0.2 = 7 (relatively flat), center point μ B=0.8 - 0.4 * 0.2 = 0.72 (the threshold is relatively high). If the current risk probability is also p... risk =0.5, since 0.5 < 0.72, substituting into the formula, we can calculate λ. B base ≈0.17. This indicates that for low-priority slices, medium risk is insufficient to trigger strong constraints, and the system still allows it to maintain a relatively large degree of freedom in policy exploration.

[0136] Step 403: Calculate the current global resource utilization rate based on the global resource status; when the global resource utilization rate exceeds the preset resource stress threshold, the base value of the adaptive regularization coefficient is amplified and adjusted using the global resource utilization rate to obtain the adjusted adaptive regularization coefficient, so as to enhance the game constraint on resource competition under resource stress; when the global resource utilization rate does not exceed the resource stress threshold, the base value is used as the adjusted adaptive regularization coefficient.

[0137] In this embodiment, besides the risk status of individual slices, the stress level of the global resource pool is also a crucial factor determining the regularization strength. The system first aggregates the occupancy of underlying physical resources and calculates the weighted average global resource utilization rate U. global (t).

[0138] The following formula is used to macroscopically adjust the base values ​​of the coefficients:

[0139] λ i adj (t)=λ i base (t)*max(ε,1+η*(U global (t)-U th ));

[0140] Where, λ i adj (t) represents the adjusted adaptive regularization coefficient, ε is the lower limit of the adjustment factor, η is the adjustment sensitivity coefficient, and U th This is the threshold for resource scarcity.

[0141] For example, set U th =0.7, which is 70%, η=2.0. When the global resource utilization rate soars to 0.9, which is 90%, the adjustment factor is 1+2.0*(0.9-0.7)=1.4. This means that the regularization coefficient of all slices will be uniformly amplified by 1.4 times on the original basis, tightening the constraints at the macro level, suppressing excessive resource contention between slices, and preventing system crashes.

[0142] Step 404: Calculate the sum of the adjusted adaptive regularization coefficients of all network slices as the current total regularization strength; check whether the total regularization strength exceeds the preset upper bound of the total regularization strength; if it exceeds the upper bound, perform joint pruning and redistribution of the adjusted adaptive regularization coefficients based on the dynamic priority weights of each network slice, so that the proportion of coefficients retained by each network slice is positively correlated with its dynamic priority weights, and obtain the final adaptive regularization coefficients; if it does not exceed the upper bound, directly use the adjusted adaptive regularization coefficients as the final adaptive regularization coefficients.

[0143] In this embodiment, in order to prevent the total regularization term from becoming too large due to the superposition of high risks from multiple slices, thereby masking the policy gradient signal, the system introduces a total amount constraint mechanism.

[0144] Specifically, the system first calculates the total regularization strength Σ j λ j adj (t). If the sum exceeds the preset upper bound Λ max Then, priority-based proportional cropping will be performed:

[0145] λ i final (t)=Λ max *(α i (t)*λ i adj (t)) / (Σ j (α j (t)*λ j adj (t)));

[0146] Where, λ i final (t) represents the final adaptive regularization coefficient, Σ j This indicates summing over all slices.

[0147] This formula indicates that when the regularization budget is limited, the system prioritizes ensuring the regularization strength of high-priority slices to maintain sufficient game-theoretic constraints, while being forced to reduce the coefficients of low-priority slices. This mechanism ensures that, even under high-load scenarios, the system's control signals can still reflect differences in business priorities, achieving refined management with limited resources.

[0148] In some alternative implementations, if the total regularization strength does not exceed the upper bound, then λ is directly set. i final (t)=λ i adj (t), no cropping is performed. In addition, to prevent cropping from causing the coefficients of some slices to be too small, a small minimum value can be introduced into the formula.

[0149] According to one aspect of this application, the numerical examples above provide a clearer understanding of the differences in curve morphology of the parameterized Sigmoid mapping under different priorities. The Sigmoid curve corresponding to a high-priority slice exhibits a steep left-shifted shape, meaning the center point of the curve is located at a lower risk probability and the transition zone is narrow. This indicates that the coefficient rapidly rises to near its upper limit at a lower risk probability level, achieving a high-response protection effect for low-risk slices. The Sigmoid curve corresponding to a low-priority slice exhibits a gentle right-shifted shape, meaning the center point of the curve is located at a higher risk probability and the transition zone is wide. The coefficient only begins to increase significantly when the risk probability rises significantly, preserving ample strategic flexibility for low-priority slices in the low-to-medium risk range. The intersection of the two curves corresponds to the response range for medium risk and medium priority. By dynamically controlling both the slope and center point of the curve through dynamic priority weights, this parameterized Sigmoid mapping achieves a more refined differentiated risk response than adjusting only a single parameter.

[0150] In other implementations, the parameterized Sigmoid mapping can also be replaced with a piecewise linear mapping or an exponential mapping function.

[0151] Example 5 describes a dual-channel priority modulation architecture and its specific implementation in the policy optimization process. By using a multiplicative coupling mechanism, the two regularization channels of geometric shape and amplitude intensity are merged to construct a total regularization loss that is highly sensitive to priority. Combined with the Gumbel-Softmax reparameterization technique, the gradient propagation problem in the discrete action space is solved, and end-to-end policy learning is achieved.

[0152] Step 501: Using a slice-by-slice multiplication method, the first regularization term is multiplied by the adaptive regularization coefficient to obtain the total regularization loss; dual-channel priority modulation is achieved through multiplication, wherein the first regularization term provides priority-based geometric shape constraints, and the adaptive regularization coefficient provides priority-based amplitude intensity constraints. The two work together to apply differentiated game equilibrium guidance to network slices with different priorities.

[0153] In this embodiment, the system couples the asymmetric Tsallis divergence term (geometric channel output) with the adaptive regularization coefficient (amplitude channel output) through slice-by-slice multiplication. This coupling method is not a simple numerical superposition, but rather constructs a nonlinear modulation relationship.

[0154] The specific calculation formula is as follows:

[0155] R i (t)=λ i final (t)*D qi T (π i ||πi ref );

[0156] Among them, R i (t) represents the total regularization loss of slice i at time t, λ i final (t) represents the final adaptive regularization coefficient, D qi T Based on the order parameter q i The Tsallis divergence term.

[0157] In this formula, the physical effect of dual-channel modulation is that, for high-priority slices, it not only has a larger order parameter q i (leading to D) qi T (It is more sensitive to high probability regions) and also has a large coefficient λ. i final (This leads to a doubling of the overall penalty intensity). The product of the two produces a superlinear constraint effect of 1+1>2, forcibly locking the policy of high-priority slices tightly within the Nash equilibrium neighborhood of the reference policy, preventing them from interfering with system stability due to overexploration. For low-priority slices, their order parameter q... i Smaller (focus on tail exploration), and coefficient λ i final Smaller (weaker penalty). The product of the two results in a lower numerical value for the regularization term, which effectively grants low-priority slices the privilege of extensive exploration in non-critical regions, allowing them to flexibly fill resource gaps left by high-priority slices.

[0158] For example, in the extreme case, if additive coupling is used, even if the coefficient λ is small, as long as the divergence term D is large, the total loss will still be large, causing the exploration of low-priority slices to be unexpectedly suppressed. However, when multiplicative coupling is used, as long as the coefficient λ approaches zero, such as in low-risk and low-priority situations, no matter how the divergence term fluctuates, the total regularization loss will be filtered out by the gating effect, truly achieving differentiated strategy guidance.

[0159] Assume the system contains a high-priority slice A and a low-priority slice B. The Tsallis divergence value (geometric channel output) of slice A is D. A =0.8, the adaptive regularization coefficient (amplitude channel output) is λ. A =0.9; the Tsallis divergence value of slice B is D. B =0.3, the adaptive regularization coefficient is λ B =0.2. Under multiplicative coupling, the regularization loss of slice A is R. A =0.9 × 0.8 = 0.72, the regularization loss of slice B is R. B=0.2×0.3=0.06, the ratio of the two is 12 to 1. If additive coupling is used instead, i.e. R=D+λ, then the regularization loss of slice A is 0.8+0.9=1.7, and the regularization loss of slice B is 0.3+0.2=0.5, the ratio of the two is only 3.4 to 1.

[0160] Therefore, multiplicative coupling amplifies the regularization difference between high-priority and low-priority slices from 3.4 times to 12 times, producing a significant superlinear amplification effect. Furthermore, under multiplicative coupling, when the coefficient λ of the low-priority slice... B When it approaches zero (such as in low-risk and low-priority cases), regardless of its Tsallis divergence value D B Regardless of fluctuations, the total regularization loss is effectively suppressed to near zero. This gating effect is not present in additive coupling – in additive mode, even with small coefficients, fluctuations in the divergence value are directly transmitted to the total loss, potentially interfering with policy exploration for low-priority slices.

[0161] Step 502: Perform Gumbel-Softmax reparameterization on the output of the policy network for each network slice to generate differentiable approximate discrete actions; wherein, the policy network is a neural network configured for each network slice to output the probability distribution of actions; calculate the policy gradient objective based on the approximate discrete actions, and update the policy network parameters through backpropagation; wherein, the first regularization term and the Shannon entropy regularization term are calculated based on the standard Softmax probability output of the policy network to use the accurate probability distribution for constraint.

[0162] In this embodiment, since the resource allocation action of network slicing is discrete, for example, selecting bandwidth level 3, the direct sampling operation (argmax or classification sampling) is non-differentiable, making it difficult to directly calculate the policy gradient through backpropagation. This embodiment introduces Gumbel-Softmax.

[0163] Specifically, let l be the original Logits vector output by the actor network. i During the forward propagation phase of training, the system first generates an independent and identically distributed Gumbel noise vector g with the same dimension as Logits, where:

[0164] g k =-log(-log(u k )), u k It follows a uniform distribution in (0,1).

[0165] An approximate one-hot action vector y is generated using the Softmax function with a temperature coefficient τ:

[0166] y k =exp((l i,k +gk ) / τ) / Σ j exp((l i,j +g j ) / τ);

[0167] Among them, y k Let l be the approximate probability value of the k-th action. i,k For the actor network, the subscript i represents the i-th network slice agent, the subscript k represents the k-th candidate discrete action, the subscript j is the index for traversing all possible discrete actions, and τ is the temperature coefficient.

[0168] When τ approaches 0, the vector y approaches the true one-hot vector, i.e., one bit is 1 and the rest are 0, simulating discrete selection behavior; when τ is large, y approaches a uniform distribution. In the early stages of training, a large τ (such as 1.0) is usually set to encourage exploration, and it is gradually decayed to a smaller value (such as 0.1) as training progresses.

[0169] The generated approximate action vector y is fed into the critic network to calculate the action value Q(s,y), and then to calculate the policy gradient objective L. i PG Since y is differentiable with respect to the network parameters, the gradient can be backpropagated from the critic network to the actor network via y using the chain rule.

[0170] Furthermore, when calculating the regularization loss (including Tsallis divergence and optional Shannon entropy), the system does not use the noisy Gumbel output y mentioned above, but instead directly uses the accurate probability distribution π of the actor network based on the standard Softmax output. i (a|s i )=Softmax(l i The regularization term is used to constrain the shape of the policy distribution itself. Directly using a noise-free probability distribution can provide a more accurate gradient signal with low variance.

[0171] Total policy loss function L i actor At the code implementation level, it consists of two parts: one is the policy gradient loss calculated based on Gumbel sampling (responsible for maximizing cumulative reward), and the other is the regularization loss calculated based on standard probability (responsible for maintaining game equilibrium). Both update the parameters θ of the actor network during backpropagation. i .

[0172] According to one aspect of this application, in the above-described coupling calculation, the adaptive regularization coefficient λ i final(t) is a scalar value calculated based on real-time state observations at the current training time step, while the batch mean of the Tsallis divergence is the mean calculated on the states of multiple different historical time steps sampled from the empirical replay buffer. The two are not perfectly aligned in time.

[0173] The design choice is based on the following considerations: After multiple smoothing processes such as parameterized Sigmoid mapping, global utilization adjustment, and upper bound pruning, the adaptive regularization coefficient changes slowly between adjacent time steps, and the coefficient value of the current time step is highly representative of recent historical time steps; the function of the regularization coefficient is to control the overall magnitude of the regularization loss, rather than accurately matching each historical state. The regularization strength of the entire batch is uniformly calibrated by the global risk perception level of the current time step, which is conceptually consistent with the common practice of evaluating historical experience with the current policy in off-policy training; if the coefficients of each state in the batch are calculated independently for the corresponding time step, the coefficient values ​​of historical time steps need to be stored in the experience tuple, increasing storage overhead and introducing high variance of the coefficients within the same batch, which is detrimental to the stability of gradient estimation.

[0174] According to one aspect of this application, the complete closed-loop process of a single training iteration in the training process of this invention is as follows:

[0175] The system collects real-time state features and global resource status of each slice from the network environment, corresponding to the data acquisition and feature construction in step 101 of Example 1. Based on the current state features, the dynamic priority weights of each slice are calculated, corresponding to Example 2. Subsequently, two branches are executed in parallel: the first branch constructs the asymmetric Tsallis divergence regularization term for each slice based on the dynamic priority weights, corresponding to Example 3; the second branch generates adaptive regularization coefficients for each slice based on the state features and dynamic priority weights, corresponding to Example 4. After the two branches merge, the total regularization loss is constructed through multiplicative coupling and assembled with the policy gradient objective into the total policy loss function, which updates the parameters of the actor network for each slice, corresponding to Example 5. Simultaneously, the centralized commentator network is updated synchronously based on global information. The updated actor network guides each slice to generate resource allocation actions and distributes them for execution. The environment provides feedback on the state information and immediate reward for the next time step, experience tuples are stored in a buffer, and the state information for the next time step is fed back to the data acquisition stage, driving the next iteration. This constitutes a complete training loop from data acquisition to policy update to environment interaction and back to data acquisition.

[0176] The termination condition for the training process can be one of the following strategies or a combination thereof, setting an upper limit on the total number of training iterations, with a recommended range of 10. 5 Up to 10 6The training is considered converged and terminated early when the average SLA default rate of all slices is below a preset threshold (e.g., 5%) for several consecutive evaluation periods (e.g., 100 consecutive evaluation periods, each containing several training steps). After training, each slice agent retains the final actor network parameters for online deployment. During the online execution phase, each agent performs forward inference and action sampling based solely on its local state features, eliminating the need for the commentator network and global information.

[0177] Example 6 describes the flexible variations of the total policy loss function in different application scenarios and its adaptive switching mechanism, and provides a general loss function architecture. By introducing an optional Shannon entropy regularization term, the system can achieve fast convergence using a two-term loss pattern in low-dimensional simple scenarios, and prevent policy collective collapse using a three-term loss pattern in high-dimensional complex scenarios. Furthermore, it provides an automated switching algorithm based on entropy threshold to achieve a dynamic balance between exploration and exploitation.

[0178] Step 601, the total policy loss function also selectively includes a Shannon entropy regularization term; the Shannon entropy regularization term is constructed based on the Shannon entropy of the current policy distribution and is used to provide undirected policy exploration incentives to prevent the policy distribution from collapsing prematurely.

[0179] In this embodiment, in addition to the policy gradient objective and Tsallis divergence regularization term constructed in the previous embodiments, the system also introduces Shannon entropy as a third optional loss component. For the current policy distribution π of slice i... i (·|s i ), its Shannon entropy H i (t) is calculated as follows:

[0180] H i (t)=-Σ a∈A π i (a|s i )*ln(π i (a|s i ));

[0181] Among them, H i (t) represents the entropy value of the current policy, and ln is the natural logarithm.

[0182] This term measures the randomness or uncertainty of the policy distribution. Unlike the Tsallis divergence regularization term, which is a directional constraint requiring the policy to distribute like a reference policy, the Shannon entropy regularization term is an undirected incentive, only requiring the policy not to become overly concentrated, i.e., maintaining a certain exploration width, without specifying a specific exploration direction. Functionally, the two complement each other: the Tsallis divergence is responsible for pulling the policy towards the neighborhood of Nash equilibrium, while the Shannon entropy is responsible for preventing the policy from prematurely converging to suboptimal deterministic actions before reaching equilibrium.

[0183] Specifically, in the integrated satellite-ground scenario, the sliced ​​state feature vector, in addition to the conventional M QoS indicators (such as throughput and latency) and D-dimensional resource occupancy features, also includes extended dimensions such as the normalized satellite elevation angle, the normalized satellite-ground link signal-to-noise ratio, and the remaining transit time window percentage, among other satellite-ground link feature dimensions. The state feature vector dimension d in the satellite-ground scenario... s Typically, it's between 12 and 20. In a typical space-ground computing network converged deployment, the resource dimension D can be set to 3, corresponding to on-board computing resources, space-ground link bandwidth, and ground edge caching, respectively. The quantization levels K for each dimension are... d The values ​​can be set to 8, 6, and 5 respectively, then the action space size |A| = 8 × 6 × 5 = 240, corresponding to 240 discrete joint resource allocation schemes, which can be processed by the standard discrete strategy network adopted in this invention.

[0184] Furthermore, the periodicity of satellite transit windows causes SLA defaults to exhibit a quasi-periodic pattern related to the orbital period. The risk prediction network learns this periodic pattern gradually through incremental training, and can predict the risk of link interruption in advance when the satellite is about to leave the transit window, which shortens the remaining transit time window, thus triggering a preventive increase in the adaptive regularization coefficient.

[0185] Step 602: The total policy loss function is in a bi-term loss mode, consisting only of the policy gradient objective and the total regularization loss. The bi-term loss mode is suitable for low-dimensional action spaces or resource allocation scenarios that require fast convergence, and directly guides policy updates by utilizing the anchoring effect of the first regularization term.

[0186] In this embodiment, when the application scenario is relatively simple, such as when the action space dimension is low (e.g., only 3 to 5 discrete levels) or the network environment is relatively stable, the system adopts a bi-term loss mode. In this case, the coefficient of the Shannon entropy regularization term is explicitly set to zero.

[0187] Total policy loss function L i dual Defined as:

[0188] L i dual (θ i )=Li PG (θ i )+R i (t);

[0189] Among them, L i PG For the policy gradient objective, θ i Let R be the network parameters of the agent in the i-th network slice, typically a negative advantage function or value estimate. i (t) represents the total regularization loss including dual-channel modulation.

[0190] In this mode, policy updates are driven by reward signals and anchored by Tsallis divergence. Due to the small action space and the inherently high entropy lower bound of the policy distribution, the constraints provided by Tsallis divergence are sufficient to maintain the necessary diversity. Introducing additional entropy terms might introduce unnecessary noise and slow down convergence. The two-term mode is the preferred option when high execution efficiency is desired.

[0191] Step 603: The total policy loss function is in a three-term loss mode, consisting of the policy gradient objective, the total regularization loss, and a weighted Shannon entropy regularization term. The three-term loss mode is suitable for resource allocation scenarios with high-dimensional action spaces or where there is a risk of collective policy collapse. By subtracting the weighted Shannon entropy regularization term from the total policy loss function, the randomness of the policy distribution is forcibly maintained.

[0192] In this embodiment, when the application scenario is complex, such as when the action space has a high dimension (e.g., more than 50 discrete combinations) or when there are a large number of agents playing concurrently, a collective collapse phenomenon is likely to occur. That is, all agents converge prematurely to the same suboptimal action, causing the reference strategy to become fixed and the Tsallis divergence to fail. To address this, the system activates three loss modes.

[0193] Total policy loss function L i triple Defined as:

[0194] L i triple (θ i )=L i PG (θ i )+R i (t)-α H *H i (t);

[0195] Where, α H H is the Shannon entropy regularization coefficient. i (t) is the entropy value calculated in step 601.

[0196] Since the optimization objective is to minimize the total loss L i triple And entropy H i (t) needs to be maximized to encourage exploration, hence the entropy term has a negative sign in the formula. In this mode, even if the Tsallis divergence attempts to pull the policy toward a collapsed reference policy, the Shannon entropy term will generate a reverse repulsive force, forcing the policy distribution to maintain a certain width and retain the ability to escape local optima.

[0197] Step 604: The weight coefficients of the weighted Shannon entropy regularization term are adaptively adjusted based on the current policy state; the average policy entropy of the current policy in each network slice is monitored and compared with a preset entropy threshold; when the average policy entropy is lower than the preset entropy threshold, the weight coefficients are increased to activate the three-term loss mode and break the policy collapse; when the average policy entropy is higher than the preset entropy threshold, the weight coefficients are decreased or set to zero to switch back to the two-term loss mode and accelerate policy convergence.

[0198] In this embodiment, to achieve the optimal balance between exploration and development, the present invention provides a state-aware adaptive switching algorithm. This algorithm does not require manually specifying a fixed mode, but dynamically adjusts α based on the system's real-time entropy level. H .

[0199] Specifically, the system first calculates the average policy entropy H of all slice agents at the current time. avg (t):

[0200] H avg (t)=(1 / N)*Σ i H i (t);

[0201] Based on a preset entropy threshold H th Calculate the current Shannon entropy coefficient α (e.g., 20% of the maximum possible entropy). H (t):

[0202] α H (t)=α H,0 *max(0,1-H avg (t) / H th );

[0203] Where, α H,0 This is the baseline value for the entropy coefficient (e.g., 0.01).

[0204] The adjustment logic of this formula is as follows: when H avg (t)>H th When the strategy is sufficiently random, all terms within the parentheses are negative, the max function outputs 0, and at this time α H(t)=0. The system automatically operates in bivariate loss mode, focusing on rapid convergence using known information. When H avg (t) <H th When the strategy tends to be certain, there is a risk of collapse, and the terms in parentheses are positive, α H (t) becomes positive and follows H avg The entropy penalty increases as (t) decreases. The system automatically switches to the three loss modes, and the more severe the collapse, the greater the entropy penalty, forcibly pushing the strategy back to a high-entropy state.

[0205] Through the aforementioned closed-loop feedback mechanism, this embodiment achieves fully automatic mode switching, which avoids blind exploration in the early stages of training and prevents deadlock in the later stages of training, thereby improving the robustness of the algorithm in dynamic environments.

[0206] Example 7 describes the specific process of environmental interaction mechanism, multi-dimensional reward function design, network architecture configuration and parameter update in the model training closed loop, and completes the key closed loop mechanism from online policy execution to offline model optimization, so as to ensure the long-term stable operation and continuous evolution of the entire multi-agent reinforcement learning system.

[0207] Step 701: The pre-built lightweight risk prediction network is continuously updated through an incremental training mechanism. During the resource allocation process, SLA default labels are generated based on the comparison results between the real-time QoS indicators and SLA target parameters of each network slice. Empirical data containing real-time network slice state characteristics and SLA default labels are collected and stored in the empirical replay buffer. Empirical data is periodically sampled from the empirical replay buffer, and the parameters of the lightweight risk prediction network are updated using the binary cross-entropy loss function so that it can continuously track the dynamic evolution of network slice SLA default patterns.

[0208] In this embodiment, in order to solve the SLA default mode drift problem caused by the non-stationarity of the network slicing environment, the lightweight risk prediction network is not fixed after one-time training, but adopts an incremental training mode.

[0209] Specifically, the system maintains a fixed-capacity experience replay buffer. At each time step t, the system assembles the interaction data of all slices into experience tuples and stores them in the buffer.

[0210] e t =(s(t),a(t),r(t),s(t+1),v(t),done(t));

[0211] Where s(t) is the joint state, a(t) is the joint action, r(t) is the reward vector, v(t) is the SLA default label vector, and done(t) is the termination flag.

[0212] SLA breach of contract label vi (t) is a binary variable. The system automatically labels the variables based on the comparison between real-time QoS indicators and SLA targets: if any key KPI fails to meet the target, then v... i (t)=1, otherwise v i (t)=0.

[0213] Every fixed training interval T train (For example, every 10 time steps), the system randomly samples a batch of samples of size |B| from the buffer. For the risk prediction network, the binary cross-entropy (BCE) loss function is used for updates:

[0214] L i risk =(-1 / |B|)*Σ (s,v)∈B [v i *log(p i risk (s))+(1-v i )*log(1-p i risk (s))];

[0215] Among them, L i risk That is, the binary cross-entropy of the risk prediction network for the i-th network slice, p i risk (s) represents the predicted default probability output by the risk prediction network in state s, v i These are the actual observed default labels.

[0216] By minimizing this loss function, the risk prediction network can continuously adapt to new traffic patterns and interference characteristics, providing accurate risk signals for the generation of adaptive regularization coefficients in the online phase.

[0217] Step 702 also includes training a centralized commentator network; constructing a centralized commentator network, which uses the concatenation of real-time network slice state features of all network slices as joint state input and the concatenation of resource allocation actions of all network slices as joint action input, to evaluate the value of the joint state and joint action of all network slices; while updating the resource allocation policy based on the total policy loss function, calculating the temporal difference error based on the immediate reward obtained after each network slice performs resource allocation and the value estimate of the centralized commentator network, and using the temporal difference error to update the parameters of the centralized commentator network, providing a value signal for the calculation of the policy gradient objective.

[0218] In this embodiment, the system employs a centralized training and distributed execution (CTDE) architecture. While each slice agent makes independent decisions, its critic network Q... i(s,a) can observe the global joint state s and joint action a, solving the problem of environmental nonstationarity in multi-agent environments.

[0219] Specifically, the architecture of the critic network is designed as follows, with the input layer receiving data in dimension N*(d). s The joint feature vector of +|A|); the first hidden layer contains 256 neurons and uses the ReLU activation function; the second hidden layer contains 128 neurons and uses the ReLU activation function; the output layer contains 1 linear neuron and outputs a scalar action value Q. i .

[0220] During training, the system first uses the target network to calculate the temporal difference, i.e., the TD target value y. i :

[0221] y i =r i +γ'*Q i target (s',a target ');

[0222] Where, r i For immediate reward, γ' is the discount factor (e.g., 0.99), s' is the global joint state at the next time step (or subsequent time step), and Q i target For the target critic network, a target 'The next moment's action output to the target actor via network;'

[0223] Calculate the TD error (time series difference error) and construct the mean squared error, i.e., the MSE loss function:

[0224] L i critic =(1 / |B|)*Σ (s,a,r,s')∈B (Q i (s,a)-y i ) 2 ;

[0225] Among them, L i critic This is the mean squared error of the commentator network for the i-th network slice.

[0226] Minimize L using gradient descent. i critic Update the critic network parameters. The accuracy of the critic network directly determines the policy gradient L. i PG The effectiveness of the algorithm is the foundation for the convergence of the entire reinforcement learning algorithm.

[0227] Step 703: Construct a comprehensive reward function containing multi-dimensional performance indicators to quantify the immediate benefits of network slices in the resource allocation process and to serve as the optimization objective of the reinforcement learning algorithm.

[0228] In this embodiment, the design of the reward function directly guides the learning direction of the agent. This invention constructs a comprehensive reward function r that includes QoS satisfaction, SLA violation penalty, and resource efficiency penalty. i (t). This function not only focuses on achieving quality of service, but also takes into account the economy of resource usage.

[0229] The specific reward calculation formula is as follows:

[0230] r i (t)=Σ m (ω m *ρ i,m (t))-c vio *v i (t)-c res *(res i alloc (t) / res i cap );

[0231] Where, ω m Let ρ be the importance weight of the m-th indicator. i,m (t) represents the satisfaction rate of the m-th QoS indicator, v i (t) represents the default label, res i alloc (t) represents the allocated resource amount, res i cap c represents the total amount of resources. vio c is the penalty coefficient for breach of contract (e.g., 2.0). res This is the resource efficiency coefficient (e.g., 0.1).

[0232] Satisfaction rate ρ i,m (t) also introduces the direction identifier d m To be processed uniformly:

[0233] ρ i,m (t)=min((QoS real / QoS tgt ) dm ,1);

[0234] This formula ensures that when the target is not met, the satisfaction level is less than 1 and decreases as the gap increases; when the target is met or exceeded, the satisfaction level is truncated to 1. This design prevents agents from wasting valuable network resources in pursuit of meaningless excess performance (such as lower latency far exceeding demand).

[0235] Step 704: A soft update mechanism is used to maintain the parameters of the target actor network and the target critic network to stabilize the temporal difference target estimation during the training process.

[0236] In this embodiment, to reduce oscillations during training, the system does not directly use the current network to calculate the TD target, but instead uses a target network with lagging parameters. The parameters θ of the target network... target Maintenance is performed using the Polyak average, or soft update, method:

[0237] θ target =τ soft *θ current +(1-τ soft )*θ target ;

[0238] Where, θ current τ is the parameter of the current network (actor or critic). soft This is the soft update coefficient, which is usually taken as a very small value, such as 0.005.

[0239] This mechanism ensures that the parameters of the target network change slowly, providing a relatively stable regression target (i.e., a dynamic reference target) for the learning of the Critic network, and improving the convergence stability of the algorithm in complex and ever-changing network slicing environments.

[0240] Optionally, each network (actor, critic, risk predictor) uses the Adam optimizer for parameter updates. The learning rate for the actor network is set to 3 × 10⁻⁶. -4 The learning rate of the critic network was set to 1×10. -3 The learning rate of the risk prediction network is set to 1×10. -3 The capacity of the experience replay buffer is set to 10. 5 Empirical tuples. The batch size |B| for each training sample is set to 256.

[0241] This embodiment details the offline training process, network configuration parameters, reward function construction, and target network update mechanism, which, together with the aforementioned embodiments, constitute a complete and reproducible technical method.

[0242] According to one aspect of this application, the reward function does not include explicit priority weighting; that is, the reward calculation logic for all slices is the same, and different reward weights are not assigned based on different slice priorities. In a preferred embodiment, the differentiated coordination between slices is handled by the Tsallis divergence regularization path, and the reward function is only responsible for measuring the local service quality performance and resource utilization efficiency of each slice. The two signaling paths are functionally complementary—the reward signal guides the agent to learn how to improve its own performance, and the regularization signal guides the agent to learn how to coordinate and coexist with other slices in a game environment. If priority weighting is repeatedly introduced into the reward function, it will cause coupling interference between the reward signal and the regularization signal in the priority dimension, increasing the complexity of hyperparameter tuning and making it difficult to separate the contributions of the two paths.

[0243] Example 8 provides a specific application case in a space-ground integrated computing and network fusion scenario. This scenario utilizes LEO low-Earth orbit satellites as distributed computing power nodes, and is characterized by high dynamism, link uncertainty, and very limited resources. It is a typical scenario in which the various modules of this invention obtain the fullest application value.

[0244] Step 801: Obtain the real-time network slice status characteristics of each network slice and the global resource status of the resource pool.

[0245] In this embodiment, the real-time network slice status features, in addition to including conventional service QoS indicators, also specifically include status information related to satellite links. For example, the slice status feature vector s i (t) includes the current satellite elevation angle, link signal-to-noise ratio (SNR), and the remaining visible time window length. The global resource status specifically refers to the real-time capacity and occupancy of the onboard computing resource pool (such as onboard GPU computing power) and satellite-to-ground backhaul link bandwidth of the currently passing satellite node.

[0246] Step 802: Calculate the dynamic priority weight of each network slice based on the real-time network slice state characteristics.

[0247] In this embodiment, the calculation of dynamic priority weights can respond quickly to the highly dynamic changes in the satellite-ground environment's topology. Specifically, when a satellite node passes over a specific area (such as an ocean-going vessel or a remote base station), a slice of that area gains the physical possibility of accessing onboard computing power. At this time, based on the hierarchical weight calculation mechanism, potential connection opportunities and SLA fulfillment pressures are detected, such as a sharp drop in QoS caused by ground link interruption. The system automatically calculates a large dynamic offset g. i (t), rapidly increasing the dynamic priority weight α of the slice in that region. i (t). This mechanism enables agile scheduling with priority given to satellites passing over the area, ensuring maximum resource value within the short transit window.

[0248] Step 803: Based on the dynamic priority weights and the pre-maintained reference strategy, a first regularization term with an asymmetric geometric shape is constructed; wherein, the dynamic priority weights are mapped to the geometric shape parameters of the first regularization term to adjust the sensitivity of the first regularization term to the constraints of different regions of the strategy distribution.

[0249] In this embodiment, the Tsallis divergence order parameter q i The dynamic mapping mechanism is used to handle rapid priority reconstruction in satellite-to-ground scenarios. When a satellite's transit causes the priority of a slice to spike, its q... i The value increases rapidly, for example, from 0.8 to 1.8. This immediately changes the geometry of the regularization term from encouraging tail exploration to locking the head strategy, forcing the agent to anchor its strategy to an efficient mode of utilizing onboard resources and avoiding ineffective trial-and-error exploration during the precious transit window.

[0250] Step 804: Based on the real-time network slice state characteristics, predict the SLA default risk probability of each network slice, and combine the dynamic priority weight and global resource state to generate an adaptive regularization coefficient.

[0251] In this embodiment, the lightweight risk prediction network focuses on learning the uncertainty patterns unique to satellite-to-ground links. Specifically, satellite-to-ground links are susceptible to weather (rain attenuation), changes in antenna elevation angle, and solar interference, resulting in SLA default risk exhibiting both periodic (orbit-dependent) and sudden (weather-dependent) characteristics. Trained on historical data, the risk prediction network can anticipate link quality deterioration trends caused by reduced elevation angle or increased cloud cover, outputting an increased risk probability p. risk The parameterized Sigmoid mapping mechanism generates a high regularization coefficient to tighten policy constraints in advance, guiding slices to smoothly switch to backup paths or reduce transmission rates before link interruption.

[0252] Step 805: Calculate the policy gradient objective based on the historical interaction experience data of each network slice, couple the first regularization term with the adaptive regularization coefficient to obtain the total regularization loss, and construct the total policy loss function in combination with the policy gradient objective.

[0253] In this embodiment, the dual-channel multiplication coupling mechanism plays a crucial gating role in the allocation of satellite and ground resources. Onboard computing power is highly scarce, and backhaul bandwidth is the main bottleneck. For high-priority services, such as emergency communication at disaster sites and military reconnaissance data backhaul, which have large order parameters q and large coefficients λ, the dual strong constraints prompt them to prioritize the acquisition of satellite resources to ensure timeliness. For ordinary broadband services, although they may have resource requirements, under the dual-channel adjustment, their tendency to acquire satellite resources is suppressed, and they are guided to the ground network or edge nodes, achieving a three-dimensional diversion effect of uploading high-priority services to satellite and grounding ordinary services.

[0254] Step 806: Update the resource allocation strategy of each network slice using the total policy loss function, and perform resource allocation based on the updated resource allocation strategy.

[0255] In this embodiment, the specific actions involved offloading the computational tasks of the slice to the overpassing satellite node or keeping them for local processing, as well as allocating specific satellite-to-ground link time slot resources.

[0256] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A network slice resource allocation method based on Tsallis-MAAC, characterized in that, include: Obtain the real-time network slice status characteristics of each network slice and the global resource status of the resource pool; Based on the real-time network slice state characteristics, the dynamic priority weight of each network slice is calculated. Based on dynamic priority weights and pre-maintained reference strategies, a first regularization term with an asymmetric geometric shape is constructed; wherein, the dynamic priority weights are mapped to the geometric shape parameters of the first regularization term to regulate the constraint sensitivity of the first regularization term to different regions of the strategy distribution. Based on the real-time network slice state characteristics, the probability of SLA default risk for each network slice is predicted, and an adaptive regularization coefficient is generated by combining dynamic priority weights and global resource status. The policy gradient objective is calculated based on the historical interaction experience data of each network slice. The first regularization term is coupled with the adaptive regularization coefficient to obtain the total regularization loss. The total policy loss function is then constructed by combining the policy gradient objective. The resource allocation strategy of each network slice is updated using the overall policy loss function, and resource allocation is performed based on the updated resource allocation strategy. The first regularization term is the Tsallis divergence regularization term defined based on the Tsallis relative entropy; Mapping dynamic priority weights to geometric morphology parameters of the first regularization term includes mapping dynamic priority weights to the order parameters of Tsallis divergence. Among them, the order parameter determines the distribution of the penalty weights of the Tsallis divergence regularization term on the high-probability action region and the low-probability action region. The first regularization term is coupled with the adaptive regularization coefficients to obtain the total regularization loss, which includes: The total regularization loss is obtained by multiplying the first regularization term with the adaptive regularization coefficient using a slice-by-slice multiplication method. Dual-channel priority modulation is achieved through multiplication. The first regularization term provides priority-based geometric shape constraints, and the adaptive regularization coefficient provides priority-based amplitude intensity constraints. The two work together to apply differentiated game equilibrium guidance to network slices with different priorities. The overall policy loss function also includes a Shannon entropy regularization term; The Shannon entropy regularization term is constructed based on the Shannon entropy of the current policy distribution and is used to provide undirected policy exploration incentives to prevent the policy distribution from collapsing prematurely.

2. The method according to claim 1, characterized in that, Based on the real-time network slice state characteristics, the dynamic priority weight of each network slice is calculated, including: Obtain the preset slice category benchmark weight for the service category to which each network slice belongs; Based on the real-time network slice state characteristics, the dynamic offset representing the SLA performance pressure is calculated; The dynamic offset is superimposed on the slice category baseline weight, and the dynamic priority weight is obtained through preset boundary constraint processing.

3. The method according to claim 2, characterized in that, Calculate the dynamic offset characterizing SLA performance pressure, including: Obtain the preset SLA target parameters and corresponding direction identifiers; Extract real-time QoS indicators from real-time network slice status features and calculate their normalized deviation from SLA target parameters; The sign of the normalization bias is corrected using directional identifiers to unify the optimization direction of different types of indicators; The corrected normalized bias is weighted and aggregated to obtain the dynamic offset.

4. The method according to claim 1, characterized in that, The order parameters that map dynamic priority weights to Tsallis divergence include: Obtain the preset range of order parameter values ​​and singularity avoidance threshold; The dynamic priority weights are converted into initial order parameters through a linear mapping, so that the initial order parameters increase as the dynamic priority weights increase; Detect whether the initial order parameters fall within the singularity avoidance threshold range centered at 1; If it falls within the range, the initial order parameter is shifted away from 1 to obtain the final order parameter, in order to avoid numerical singularity in the Tsallis divergence calculation.

5. The method according to claim 1, characterized in that, The pre-maintained reference strategy is updated in the following ways: Obtain the real-time policy distribution of agents in other network slices besides the current network slice; Calculate the aggregated average policy of other network slice agents as the adversary group policy; Using the exponentially weighted moving average method, the reference strategy of the previous time step is smoothly updated based on the opponent group strategy to obtain the reference strategy of the current time step; Based on real-time network slice state characteristics, predict the SLA default risk probability of each network slice, including: Input real-time network slice state features into a pre-built lightweight risk prediction network; By using forward inference through a lightweight risk prediction network, a numerical value representing the probability of a Service Level Agreement (SLA) default under the current state is output as the SLA default risk probability.

6. The method according to claim 5, characterized in that, By combining dynamic priority weights and global resource status, adaptive regularization coefficients are generated, including: Construct a parameterized sigmoid mapping function, whose slope and center point parameters are functions of dynamic priority weights; The slope and center point parameters are adjusted using dynamic priority weights, so that higher priority weights correspond to steeper slopes and lower center point thresholds. By using the adjusted parameterized Sigmoid mapping function, the SLA default risk probability is mapped to the base value of the adaptive regularization coefficient, so as to achieve differentiated risk response for different priority slices.

7. The method according to claim 6, characterized in that, The generation of adaptive regularization coefficients also includes macroscopic adjustments to the base values: Calculate the current global resource utilization rate based on the global resource status; When the global resource utilization rate exceeds the preset resource stress threshold, the base value of the adaptive regularization coefficient is amplified and adjusted using the global resource utilization rate to obtain the adjusted adaptive regularization coefficient, thereby enhancing the game constraint on resource competition under resource stress conditions.

8. The method according to claim 7, characterized in that, The generation of adaptive regularization coefficients also includes constraints on the total amount of the adjusted adaptive regularization coefficients: Calculate the sum of the adjusted adaptive regularization coefficients of all network slices as the current total regularization strength, and check whether it exceeds the preset upper bound of the total regularization strength. If the upper bound is exceeded, the adjusted adaptive regularization coefficients are jointly pruned and redistributed based on the dynamic priority weights of each network slice, so that the proportion of coefficients retained by each network slice is positively correlated with its dynamic priority weights, thus obtaining the final adaptive regularization coefficients.

Images