LSNs-oriented green computing service aggregation strategy

Abstract

The invention discloses an LSNs-oriented green computing service aggregation strategy, which is characterized in that a control satellite, a service satellite and user equipment work cooperatively by constructing a three-layer network architecture, and service scheduling and satellite state management are optimized by utilizing a Markov Decision Process (MDP) and a double deep Q-Network (Double DQN). A simulation result verifies that 47.87% of energy consumption can be saved when the service volume is low, 4.36% of energy consumption can be saved when the service volume is high, and the effectiveness of a service aggregation strategy is reflected.

Description

Technical Field

[0001] The invention relates to network communication technology, in particular to a green computing service aggregation strategy for LSNs. Background Art

[0002] The sixth generation mobile communication system (6G) will achieve higher data transmission rate, wider coverage and lower latency to meet the huge demand of future society in information transmission and processing. Low Earth Orbit (LEO) satellite networks (LSNs) can achieve full coverage of remote areas and provide lower latency than other satellite systems, making them an important part of 6G.

[0003] Although LSNs have many advantages, their energy supply is limited because LEO satellites rely on solar cells for energy. In addition, LEO satellites periodically pass through the shadow area and the sunlit area, resulting in unstable energy supply. This energy limitation and instability affect the continuous operation capability and service reliability of LSNs.

[0004] To address the energy limitation challenge of LSNs, it can be addressed by efficiently obtaining solar radiation energy. Existing literature has studied the application of perovskite-based solar cells in LEO satellites. By simulating the thermal stimulation of solar panels in the space environment, it has been verified that it can significantly improve the conversion efficiency of solar cell arrays to radiation energy. In addition, this challenge can also be addressed through effective on-board energy management. Some literature has also suggested that satellite energy will be affected by periodic fluctuations. By optimizing the charging of satellites in the sunlit area and the discharging in the shadow area, the transmission efficiency and coverage performance of LSNs in high mobility and large-scale connection application scenarios are significantly improved.

[0005] Existing studies have shown that using service aggregation in LSNs can achieve significant green energy-saving effects. Some literature points out that due to differences in service demand in different regions, in areas with smaller service demand, an average of about 45% of LEO satellite nodes can be shut down, but it does not consider the increase in system latency caused by service aggregation; some literature points out that even when the link forwarding service is not used, the link must be kept in a working state, resulting in low energy efficiency, and network energy consumption can be saved by establishing dynamic inter-satellite links on demand, but it does not quantify the increased latency cost. Summary of the invention

[0006] In view of the problems of service aggregation and LEO satellite energy limitation in the above-mentioned LSNs, the present invention provides a green service aggregation strategy for Low Earth Orbit (LEO) satellite networks (LEO Satellite Networks, LSNs). By constructing a three-layer network architecture, control satellites, service satellites and user equipment are coordinated to work together, and the Markov Decision Process (MDP) and Double Deep Q-Network (Double DQN) are used to optimize service scheduling and satellite status management.

[0007] The technical solution for achieving the purpose of the present invention is:

[0008] A green computing service aggregation strategy for LSNs, including:

[0009] (1) Building a network model:

[0010] The control satellite layer consists of satellites S c It is responsible for making aggregation decisions on the services of the business satellite layer; the business satellite layer consists of M LEO satellites in the low orbit plane and is responsible for handling specific services; the business satellite set is defined as Where S i and S k Any business satellite; the user layer is composed of various types of terminals located on the ground, which generate services and transmit them to the business satellites covering their areas;

[0011] The service set received by the service satellite layer is Where T i With T k S i and S k The number of services received by the business satellite is Where N i With N k S i and S k The number of services received; define S i The receiving business sequence is in For S i The jth task received; task The property list is B ij For Business The data size, in bits, The maximum delay that can be tolerated, in seconds;

[0012] The LEO satellite orbit model is as follows:

[0013] r(t)=(R+h)[1-ecosα(t)] (1)

[0014] Where R is the radius of the Earth, h is the satellite altitude, e is the orbital eccentricity, and α(t) is the eccentric anomaly angle;

[0015] S i With S k The distance formula is:

[0016]

[0017] Among them, d ik YesS i With S k The spherical distance, h i 、h k S i and S k orbital altitude;

[0018] (2) Build a communication model:

[0019] Ground users and S i The uplink channel capacity between is:

[0020]

[0021] in For users and S i The channel bandwidth between tx and G tx is the transmit power and antenna gain of the user equipment, is the channel gain, is the noise power;

[0022] To S i The transmission delay is:

[0023]

[0024] S i To S k The link channel capacity is:

[0025]

[0026] in YesS i With S k The channel bandwidth between TX Yes Yes S i The transmission power, G TX YesSi The antenna gain is YesS i With S k The link channel gain between YesS i With S k Noise power between

[0027] From S i Transfer to S k The required delay is:

[0028]

[0029] (3) Constructing the computational model:

[0030] like In S i Calculation, considering the existence of a service queue, the calculation delay is:

[0031]

[0032] Where f is S i CPU processor frequency, in cycles / s, κ is S i The computing power is expressed in cycles / bit.

[0033] S i Calculate T i The required delay is:

[0034]

[0035] S i Calculate T i The energy consumption is:

[0036]

[0037] Where P c YesS i The computing power of

[0038] (4) According to the satellite's working mode, S i The operating states of the satellite are divided into working state, idle state and dormant state. In the working state, the satellite performs business transmission and computing operations, which consumes a lot of energy. The average power of the communication transmission and receiving modules is The average power of the entire module is calculated as In the idle state, the satellite has no task processing, and the communication and computing modules operate at low power, with power consumption of and In the dormant state, the satellite shuts down the computing function module, and its power is Its value is zero;

[0039] LSNs gather the overall business on some business satellites for processing, and other business satellites enter a dormant state; definition and is a collection of dormant satellites and working satellites, where S i and S k are any dormant satellite and working satellite respectively, U and V is the number of dormant satellites and working satellites, and satisfies U+V=M;

[0040] Dormant Satellite S i Will Transmit to working satellite S k , the energy consumption is:

[0041]

[0042] The system as a whole transforms business from Gather to The required communication energy consumption is:

[0043]

[0044] Dormant Satellite S i The energy saving is:

[0045]

[0046] τ is the length of the dormancy time;

[0047] Dormant Satellite Collection The energy saving is:

[0048]

[0049] The energy savings brought about by service aggregation are:

[0050] ΔE=E s -E T (14)

[0051] S k After receiving the services from the dormant satellites, they are inserted into the S k After the original business queue, Insert S k After that, its serial number is N a , then the updated queue can be expressed as Among them, N' k For S k Total number of services received from dormant satellites;

[0052] After business aggregation, S k calculate The required delay is:

[0053]

[0054] S k Calculate T k The required delay is:

[0055]

[0056] After adopting the aggregation strategy, the time delay required for all working satellites to complete the business calculation is:

[0057]

[0058] Assume that the delay required for all service satellites to complete the service calculation without adopting the aggregation strategy is:

[0059] D o =max(D1,...,D i ,....,D M ) (18)

[0060] The delay increment caused by service aggregation is:

[0061] ΔD=D LA -D o (19)

[0062] In order to comprehensively evaluate the impact of energy saving and delay increase, the Energy-Delay Ratio (EDR) is defined as a benefit evaluation indicator;

[0063]

[0064] The unit of this indicator is J / s, which indicates the energy saving corresponding to unit delay;

[0065] The optimization objectives of the business aggregation strategy are:

[0066] max EDR

[0067]

[0068] 1≤U≤M (c)

[0069] U+V=M (d)

[0070] Among them, constraint (a) means that the service processing time must be less than the maximum tolerable delay of the service; constraint (b) means that the service Only assigned once, x ijkThe value is 0 or 1, indicating whether the service is assigned to the service satellite S k ; Constraints (c) and represent the range of values ​​of the number of working satellites; Constraint (d) represents that the sum of the number of working satellites and dormant satellites must be equal to the total number of all operational satellites;

[0071] (5) Optimize the services of the business satellites through OCSBE, including:

[0072] 1) Initialization: Control satellite initialization and collect information related to service distribution and satellite parameters;

[0073] 2) Determine the dormant satellite set: Sort the list in ascending order according to the volume of traffic Then determine the number of working satellites and the working satellite set;

[0074] 3) Dormant satellite service transfer: Use the Markov Decision Process (MDP) to model the problem, and then use the Double Deep Q-Network (Double DQN) to transfer the service;

[0075] 4) Record the allocation plan;

[0076] 5) Repeat the above steps to make allocation decisions for all services that need to be transferred, and form the final service-working satellite mapping.

[0077] Further:

[0078] Step 2) The sleep satellite set is determined by using a capacity-driven sleep satellite quantity mechanism (CDSSQ), and the process is as follows:

[0079] First, sort the business satellites in ascending order according to the business volume to obtain a business satellite list

[0080] Then the number of working satellites is determined by the relationship between the current traffic volume and the maximum system capacity, and is solved using equation (21):

[0081]

[0082] Among them, B c represent Current business volume, B max for The maximum calculable business volume, in bits, represents rounding up;

[0083] Finally, according to the number of working satellites and Get dormant satellite set And a collection of working satellites

[0084] Step 3) The dormant satellite service transfer is solved by MDP-Double DQN joint solution, and the process is as follows:

[0085] The element group of MDP is designed as:

[0086] ·s t = {V,B1,....B i ,...,B k ,....,B M} is the state space, where B i Represents S i Current business volume;

[0087] ·a t ={a1,.....a k ,....a V} is the action space: action a k Indicates that Assign to S k ;

[0088] ·P(s t+1 ∣s t ,a t ) is the transition probability, Assign to S k The probability can be expressed as: P(B k +B ij ∣B k ,a k ), indicating S k From state B k Transition to state B k +B ij The probability of t+1 Is to perform action a k The new state after

[0089] ·R(s t ,a t ) is the reward function, Assign to S k The reward function can be expressed as: R(B k ,a k ), indicating that Transfer to S k The ratio of the energy difference to the delay difference;

[0090] γ is the discount factor: between 0 and 1, balancing the importance of current rewards and future rewards;

[0091] Double DQN is used to solve the above MDP problem; Double DQN uses an online network Q o (s t ,a t ;θ) and the target network Q T (s t ,a t θ - ) to reduce the overestimation problem of Q value, where θ and θ - Q o (s t ,a t ;θ) and Q T (s t ,a t θ - )’s neural network parameters; they are designed as follows:

[0092] First, according to the ε-greedy strategy, in a given state s t Select action a k :

[0093]

[0094] ε is a probability between 0 and 1, indicating the possibility of random exploration by the system. In each decision-making step, the system generates a random number. If this random number is less than ε, the system will choose a random action. Otherwise, the system will choose the action that maximizes the current Q value. o (s t ,a t ; θ) is the action value function, which represents the expected return; argmax a Indicates Q o (s t ,a t ;θ) maximizes the action;

[0095] Then calculate the state transition probability and reward:

[0096] Transfer to S k The probability of is calculated as follows:

[0097]

[0098] in, YesS k The maximum amount of business that can be received, B k YesS k Current business volume;

[0099] Take action a k The reward value is:

[0100]

[0101] Then use the Bellman equation to calculate Q o To update:

[0102]

[0103] Among them, Q p (s t ,a k ; θ) is the selection action a k When the Q value before update, max a′ Q T (s t+1 ,a k θ - ) is the maximum Q value of the next state estimated by the target network;

[0104] Finally, the convergence check of Q value is carried out, and the design is as follows:

[0105] ΔQ=|Q o (S t ,a k ;θ)-Q p (S t ,a k ;θ)∣<δ (26)

[0106] Where: δ is a small threshold. If the value of ΔQ is less than δ after multiple consecutive updates, it can be considered that the Q value has converged, that is, it has been found. The optimal allocation strategy.

[0107] The present invention uses the Double DQN strategy to optimize service allocation, aiming to improve the energy efficiency of satellite networks. Through simulation experiments, the results show that when processing a small amount of services, 47.87% of energy consumption can be saved at low service volume, and 4.36% of energy consumption can be saved at high service volume. The Double DQN strategy performs particularly well, reflecting the effectiveness of the service aggregation strategy. And it can still maintain efficient operation and reduce energy consumption under high service. These research findings have important practical guiding significance in energy conservation and emission reduction, and have important reference value for industries that require high energy efficiency management, such as data centers and smart grids. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] Figure 1 It is a diagram of the hierarchical LSNs network architecture;

[0109] Figure 2Aggregate graphs for commercial satellite payloads;

[0110] Figure 3 Aggregate flow chart for business;

[0111] Figure 4 Provide an overall performance picture of the proposed strategy;

[0112] Figure 5 It is a graph showing the number of enterprises with different EDR optimization strategies; DETAILED DESCRIPTION

[0113] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0114] Example:

[0115] The present invention proposes a green computing service aggregation strategy for LSNs. First, the Energy-Delay Ratio (EDR) index is proposed under the three-layer network architecture of the control satellite layer, the service satellite layer and the user layer to evaluate the service aggregation optimization effect; secondly, the Optimal Consolidation Strategy for Business Efficiency (OCSBE) is proposed; finally, the effectiveness of the aggregation strategy is verified through simulation analysis. The specific scheme is as follows:

[0116] 1. Building a system model

[0117] 1. Build a network model

[0118] The hierarchical LSNs network architecture is as follows: Figure 1 The control satellite layer consists of satellites S in the high orbit plane. c The business satellite layer is composed of M LEO satellites in the low orbit plane and is responsible for handling specific services. The business satellite set is defined as Where S i and S k The user layer is composed of various types of terminals located on the ground, which generate services and transmit them to the business satellites covering their areas.

[0119] The service set received by the service satellite layer is Where T i With T k S i and S k The number of services received by the business satellite is Where N i With N k S i and Sk The number of services received. Definition S i The receiving business sequence is in For S i The jth task received. Task The property list is B ij For Business The data size, in bits, The maximum delay that can be tolerated, in seconds.

[0120] The LEO satellite orbit model is as follows:

[0121] r(t)=(R+h)[1-ecosα(t)] (1)

[0122] Where R is the radius of the Earth, h is the satellite altitude, e is the orbit eccentricity, and α(t) is the eccentric anomaly angle.

[0123] S i With S k The distance formula is:

[0124]

[0125] Among them, d ik YesS i With S k The spherical distance, h i 、h k S i and S k orbital altitude.

[0126] 2. Build a communication model

[0127] Ground users and S i The uplink channel capacity between is:

[0128]

[0129] in For users and S i The channel bandwidth between tx and G tx is the transmit power and antenna gain of the user equipment, H GSi is the channel gain, is the noise power;

[0130] To S i The transmission delay is:

[0131]

[0132] S i To S k The link channel capacity is:

[0133]

[0134] in YesS i With S k The channel bandwidth between TX Yes Yes S i The transmission power, G TX YesS i The antenna gain is YesS i With S k The link channel gain between YesS i With S k The noise power between.

[0135] From S i Transfer to S k The required delay is:

[0136]

[0137] For the return of calculation results, since the amount of data is small, the impact of transmission energy consumption and delay can be ignored.

[0138] 3. Build a computational model:

[0139] like In S i Calculation, considering the existence of a service queue, the calculation delay is:

[0140]

[0141] Where f is S i CPU processor frequency, in cycles / s, κ is S i The computing power of the computer is measured in cycles / bit.

[0142] S i Calculate T i The required delay is:

[0143]

[0144] S i Calculate T i The energy consumption is:

[0145] E c i =Pc D i (9)

[0146] Where P c YesS i The computing power.

[0147] 2. Problem Modeling

[0148] According to the satellite working mode, S i The operating states of the satellite can be divided into working state, idle state and dormant state. In the working state, the satellite performs business transmission and computing operations, which consumes a lot of energy. The average power of the communication transmission and receiving modules is The average power of the entire module is calculated as In the idle state, the satellite has no task processing, but to ensure timely processing of tasks, the communication and computing modules operate at low power, and their power is respectively and In the dormant state, the satellite shuts down the computing function module, and its power is Its value is zero, but the communication module function is retained for timely response of the business. It is assumed that the power level of all business satellites remains consistent.

[0149] In certain specific situations, such as in sparsely populated areas or at night, the overall ground traffic volume is low, resulting in fewer tasks to be processed by the corresponding business satellite. In this scenario, the basic operation of the functional module circuit in the satellite idle state will lead to energy waste. To improve efficiency, centralized business aggregation optimization can be used.

[0150] Business aggregation Figure 2 As shown in the figure: LSNs aggregates the overall business on some business satellites for processing, while other business satellites enter a dormant state. Definition and is a collection of dormant satellites and working satellites, where S i and S k are any dormant satellite and working satellite respectively, U and V is the number of dormant satellites and working satellites, and satisfies U+V=M.

[0151] Dormant Satellite S i Will Transmit to working satellite S k , the energy consumption is:

[0152]

[0153] The system as a whole transforms business from Gather to The required communication energy consumption is:

[0154]

[0155] Dormant Satellite S i The energy saving is:

[0156]

[0157] τ is the length of the sleep time.

[0158] Dormant Satellite Collection The energy saving is:

[0159]

[0160] The energy savings brought about by service aggregation are:

[0161] ΔE=E s -E T (14)

[0162] S k After receiving services from dormant satellites, these services are inserted into S k After the original business queue, assuming Insert S k After that, its serial number is N a , then the updated queue can be expressed as Among them, N' k For S k Total number of services received from dormant satellites.

[0163] After business aggregation, S k calculate The required delay is:

[0164]

[0165] S k Calculate T k The required delay is:

[0166]

[0167] After adopting the aggregation strategy, the time delay required for all working satellites to complete the business calculation is:

[0168]

[0169] Assume that the delay required for all service satellites to complete the service calculation without adopting the aggregation strategy is:

[0170] D o =max(D1,...,D i ,....,D M) (18)

[0171] The delay increment caused by service aggregation is:

[0172] ΔD=D LA -D o (19)

[0173] In order to comprehensively evaluate the impact of energy savings and delay increase, this paper defines Energy-Delay Ratio (EDR) as a benefit evaluation indicator.

[0174]

[0175] The unit of this indicator is J / s, which represents the energy saving corresponding to unit delay.

[0176] The optimization objectives of the business aggregation strategy are:

[0177] max EDR

[0178]

[0179] 1≤U≤M (c)

[0180] U+V=M (d)

[0181] Among them, constraint (a) means that the service processing time must be less than the maximum tolerable delay of the service; constraint (b) means that the service Only assigned once, x ijk The value is 0 or 1, indicating whether the service is assigned to the service satellite S k ; Constraints (c) and represent the range of values ​​of the number of working satellites; Constraint (d) represents that the sum of the number of working satellites and dormant satellites must be equal to the total number of all operational satellites.

[0182] 3. Problem Solving

[0183] The optimization involves both continuous and discrete variables, which is a mixed integer programming problem. The constraints are nonlinear, so this problem can be classified as a mixed integer nonlinear programming problem. OCSBE is used to aggregate the business satellite services. The strategy flow is as follows Figure 3 As shown:

[0184] Step 1 Initialization: Control satellite initialization and collect information related to service distribution and satellite parameters. Step 2 Determine the set of dormant satellites: Sort in ascending order according to the size of the service volume to obtain a list Then determine the number of working satellites and the set of working satellites.

[0185] Step 3: Dormant satellite service transfer: Use Markov Decision Process (MDP) to model the problem, and then use Double Deep Q-Network (Double DQN) to transfer the service.

[0186] Step 4: Record the allocation plan.

[0187] in:

[0188] Determination of the set of sleeping satellites: The capacity-driven sleeping satellite quantity mechanism (CDSSQ) is used to determine the number of sleeping satellites, and its design is as follows:

[0189] First, sort the business satellites in ascending order according to the business volume to obtain a business satellite list Then the number of working satellites is determined by the relationship between the current traffic volume and the maximum system capacity, and is solved using equation (21):

[0190]

[0191] Among them, B c represent Current business volume, B max for The maximum calculable business volume, in bits, Represents round up.

[0192] Finally, according to the number of working satellites and Get dormant satellite set And a collection of working satellites

[0193] Dormant satellite service transfer: MDP-Double DQN joint solution is adopted. The process is as follows:

[0194] The element group of MDP is designed as:

[0195] ·s t = {V,B1,....B i ,...,B k ,....,B M} is the state space, where B i Represents S i The current business volume.

[0196] ·a t ={a1,.....a k ,....a V} is the action space: action ak Indicates that Assign to S k .

[0197] ·P(s t+1 ∣s t ,a t ) is the transition probability, Assign to S k The probability can be expressed as: P(B k +B ij ∣B k ,a k ), indicating S k From state B k Transition to state B k +B ij The probability of t+1 Is to perform action a k The new status after.

[0198] ·R(s t ,a t ) is the reward function, Assign to S k The reward function can be expressed as: R(B k ,a k ), indicating that Transfer to S k The ratio of the energy difference to the delay difference.

[0199] γ is the discount factor: between 0 and 1, balancing the importance of current rewards and future rewards.

[0200] Double DQN is then used to solve the above MDP problem. Double DQN uses two Q networks: the online network Q o (s t ,a t ;θ) and the target network Q T (s t ,a t θ - ) to reduce the overestimation problem of Q value, where θ and θ - Q o (s t ,a t ;θ) and Q T (s t ,a t θ - )’s neural network parameters. Its design is as follows:

[0201] First, according to the ε-greedy strategy, in a given state s t Select action a k :

[0202]

[0203] ε is a probability between 0 and 1, indicating the possibility of random exploration by the system. In each decision-making step, the system generates a random number. If this random number is less than ε, the system will choose a random action; otherwise, the system will choose the action that maximizes the current Q value. o (s t ,a t ; θ) is the action value function, which represents the expected return. argmax a Indicates Q o (s t ,a t ;θ) maximizes the action.

[0204] Then calculate the state transition probability and reward:

[0205] Transfer to S k The probability of is calculated as follows:

[0206]

[0207] in, YesS k The maximum amount of business that can be received, B k YesS k Current business volume.

[0208] Take action a k The reward value is:

[0209]

[0210] Then use the Bellman equation to calculate Q o To update:

[0211]

[0212] Among them, Q p (s t ,a k ; θ) is the selection action a k When the Q value before update, max a′ Q T (s t+1 ,a k θ - ) is the maximum Q value of the next state estimated by the target network.

[0213] Finally, the convergence check of Q value is carried out, and the design is as follows:

[0214] ΔQ=|Q o (S t ,a k ;θ)-Q p (S t ,a k ;θ)∣<δ (26)

[0215] Where δ is a small threshold. If the value of ΔQ is less than δ after multiple consecutive updates, then it can be considered that the Q value has converged, that is, it has found The optimal allocation strategy.

[0216] Repeat the above steps to make allocation decisions for all services that need to be transferred, and form the final service-working satellite mapping.

[0217] 4. Simulation Analysis

[0218] A. Simulation parameter design

[0219] By using STK to build a three-layer network architecture scenario, the control satellite is set at an altitude of 2000 kilometers, while the business satellites are distributed at an altitude of 500 to 1500 kilometers. The power range of the entire functional module of the business satellite for sending intersatellite signals is 10w to 50w, the power range of the entire functional module for receiving signals is 5w to 25w

[10] , and the power range of the entire functional module for computing functions is 60w to 415w. The clock frequency of the satellite CPU is 2*10^10cycle / s. The parameters of LSNs are shown in Table 1.

[0220] Table 1LSNs Parameters

[0221]

[0222]

[0223] The parameters of the Double DQN strategy are shown in Table 2. The learning rate is set to 0.001 to ensure a stable step size for parameter updates; the Epsilon decay rate is set to 0.995 to ensure sufficient exploration in the early stage and strategy optimization in the later stage; the discount factor (Gamma) is 0.99 to emphasize the importance of long-term returns. In addition, this paper selects 500 training rounds for training during the simulation process.

[0224] Table 2:Double DQN Strategy Parameters

[0225] Parameter Value Parameter Value Learning Rate(lr) 0.001 Epsilon Decay 0.995 Gamma 0.99 Episodes 500

[0226] B Strategy Simulation and Analysis

[0227] Figure 4 The performance of the recommended strategy in terms of business aggregation is demonstrated. First, as the business volume increases, the number of business satellites involved in processing the business also increases accordingly. Second, the recommended strategy tends to select satellites with large initial business volume as business aggregation points. This helps to reduce the transmission energy consumption generated during business aggregation. Finally, after the business is aggregated, the business load difference between the working satellites is small. This is due to the reward and punishment mechanism setting in the strategy. By reasonably allocating business, the overall benefit is maximized, thereby effectively optimizing the overall performance of the system.

[0228] In order to verify the performance advantages of the recommended strategy, this paper selects the following benchmark strategies for comparative analysis:

[0229] Ant Colony Strategy: This strategy simulates ant colony behavior to generate business aggregation decisions and resource allocation to achieve global optimization.

[0230] ●MDP-QL Strategy: This strategy generates business aggregation decisions and resource allocation by building a Q table, aiming to achieve global optimization.

[0231] Figure 5 The performance of different optimization strategies is analyzed. First, as the business volume increases, the optimization objectives of each strategy show an overall downward trend and eventually converge gradually. This is mainly because the increase in business volume leads to an increase in transmission energy consumption and processing delay, which reduces the EDR value. Second, the recommended strategy performs better than other strategies under different business volume conditions. This is because the recommended strategy uses a reinforcement learning method, which can effectively balance the exploration and utilization of the optimal solution in the continuous state space, thereby achieving better optimization results. Table 3: Delay Delta and energy reduction ratio for different business volumes

[0232] Number of businesses Delay Delta(s) Energy Reduction Ratio 1000 0.0161 47.87％ 3000 0.0227 22.01％ 5000 0.0203 11.34％ 7000 0.0304 7.83％ 9000 0.0299 4.36％

[0233] Through the service aggregation strategy recommended in this paper, the delay increase and energy reduction ratio under different service volumes are shown in Table 3: When the service volume is small, the service aggregation strategy can significantly save energy, while the delay increase is small. As the service volume increases, the energy saving ratio gradually decreases. Therefore, the service aggregation strategy recommended in this paper is more advantageous in scenarios with less service volume.

[0234] Table 4:Strategy space complexity analysis table

[0235] AC MDP-QL MDP-Double DQN Space complexity O(N×|S|×|A|) O(|S|×|A|) O(W+Z+B)

[0236] The space complexity analysis of the baseline strategy and Double DQN is shown in Table 4: where N is the number of ants, |S| is the number of states, |A| is the number of actions; W is the total number of network parameters, Z is the size of the experience replay pool, and B is the batch size. The space complexity of the ant colony strategy mainly depends on the number of ants and the size of the pheromone matrix; the space complexity of MDP-QL is mainly determined by the storage of the Q table; MDP-Double DQN has a higher space complexity due to the neural network training.

Claims

1. A green computing service aggregation strategy for LSNs, characterized by: include: (1) Building a network model: The control satellite layer consists of satellites S c It is responsible for making aggregation decisions on the services of the business satellite layer; the business satellite layer consists of M LEO satellites in the low orbit plane and is responsible for handling specific services; the business satellite set is defined as Where S i and S k Any business satellite; the user layer is composed of various types of terminals located on the ground, which generate services and transmit them to the business satellites covering their areas; The service set received by the service satellite layer is Where T i With T k S i and S k The number of services received by the business satellite is Where N i With N k S i and S k The number of services received; define S i The receiving business sequence is in For S i The jth task received; task The attribute list is B ij For Business The data size, in bits, The maximum delay that can be tolerated, in seconds; The LEO satellite orbit model is as follows: r(t)=(R+h)[1-ecosα(t)] (1) Where R is the radius of the Earth, h is the satellite altitude, e is the orbital eccentricity, and α(t) is the eccentric anomaly angle; S i With S k The distance formula is: Among them, d ik YesS i With S k The spherical distance, h i 、h k S i and S k orbital altitude; (2) Build a communication model: Ground users and S i The uplink channel capacity between is: in For users and S i The channel bandwidth between tx and G tx is the transmit power and antenna gain of the user equipment, is the channel gain, is the noise power; To S i The transmission delay is: S i To S k The link channel capacity is: in YesS i With S k The channel bandwidth between TX Yes Yes S i The transmission power, G TX YesS i The antenna gain is YesS i With S k The link channel gain between YesS i With S k Noise power between From S i Transfer to S k The required delay is: (3) Constructing the computational model: like In S i Calculation, considering the existence of a service queue, the calculation delay is: Where f is S i CPU processor frequency, in cycles / s, κ is S i The computing power is expressed in cycles / bit. S i Calculate T i The required delay is: S i Calculate T i The energy consumption is: Where P c YesS i The computing power of (4) According to the satellite's working mode, S i The operating states of the satellite are divided into working state, idle state and dormant state. In the working state, the satellite performs business transmission and computing operations, which consumes a lot of energy. The average power of the communication transmission and receiving modules is The average power of the entire module is calculated as In the idle state, the satellite has no task processing, and the communication and computing modules operate at low power, with power consumption of and In the dormant state, the satellite shuts down the computing function module, and its power is Its value is zero; LSNs gather the overall business on some business satellites for processing, and other business satellites enter a dormant state; definition and is a collection of dormant satellites and working satellites, where S i and S k are any dormant satellite and working satellite respectively, U and V is the number of dormant satellites and working satellites, and satisfies U+V=M; Dormant Satellite S i Will Transmit to working satellite S k , the energy consumption is: The system as a whole transforms business from Gather to The required communication energy consumption is: Dormant Satellite S i The energy saving is: τ is the length of the dormancy time; Dormant Satellite Collection The energy saving is: The energy savings brought about by service aggregation are: ΔE=E s -E T (14) S k After receiving the services from the dormant satellites, they are inserted into the S k After the original business queue, Insert S k After that, its serial number is N a , then the updated queue can be expressed as Among them, N' k For S k Total number of services received from dormant satellites; After business aggregation, S k calculate The required delay is: S k Calculate T' k The required delay is: After adopting the aggregation strategy, the time delay required for all working satellites to complete the business calculation is: Assume that the delay required for all service satellites to complete the service calculation without adopting the aggregation strategy is: D o =max(D1,...,D i ,....,D M ) (18) The delay increment caused by service aggregation is: ΔD=D LA -D o (19) In order to comprehensively evaluate the impact of energy saving and delay increase, the energy-delay ratio is defined as the benefit evaluation index; The unit of this indicator is J / s, which indicates the energy saving corresponding to unit delay; The optimization objectives of the business aggregation strategy are: max EDR 1≤U≤M (c) U+V=M (d) Among them, constraint (a) means that the service processing time must be less than the maximum tolerable delay of the service; constraint (b) means that the service Only assigned once, x ijk The value is 0 or 1, indicating whether the service is assigned to the service satellite S k ; Constraints (c) and represent the range of values ​​of the number of working satellites; Constraint (d) represents that the sum of the number of working satellites and dormant satellites must be equal to the total number of all operational satellites; (5) Optimize the services of the business satellites through OCSBE, including: 1) Initialization: Control satellite initialization and collect information related to service distribution and satellite parameters; 2) Determine the dormant satellite set: Sort the list in ascending order according to the volume of traffic Then determine the number of working satellites and the working satellite set; 3) Dormant satellite service transfer: Use the Markov decision process to model the problem, and then use the dual-depth Q network to transfer the service; 4) Record the allocation plan; 5) Repeat the above steps to make allocation decisions for all services that need to be transferred, and form the final service-working satellite mapping.

2. The green computing service aggregation strategy for LSNs according to claim 1 is characterized by: Step 2) The dormant satellite set is determined by using a capacity-driven dormant satellite quantity mechanism, and the process is as follows: First, sort the business satellites in ascending order according to the business volume to obtain a business satellite list Then the number of working satellites is determined by the relationship between the current traffic volume and the maximum system capacity, and is solved using equation (21): Among them, B c represent Current business volume, B max for The maximum calculable business volume, in bits, represents rounding up; Finally, according to the number of working satellites and Get dormant satellite set And a collection of working satellites 3. The green computing service aggregation strategy for LSNs according to claim 1 is characterized by: Step 3) The dormant satellite service transfer is solved by MDP-Double DQN joint solution, and the process is as follows: The element group of MDP is designed as: ·s t = {V,B1,....B i ,...,B k ,....,B M } is the state space, where B i Represents S i Current business volume; ·a t ={a1,.....a k ,....a V } is the action space: action a k Indicates that Assign to S k ; ·P(s t+1 ∣s t ,a t ) is the transition probability, Assign to S k The probability can be expressed as: P(B k +B ij ∣B k ,a k ), indicating S k From state B k Transition to state B k +B ij The probability of t+1 Is to perform action a k The new state after ·R(s t ,a t ) is the reward function, Assign to S k The reward function can be expressed as: R(B k ,a k ), indicating that Transfer to S k The ratio of the energy difference to the delay difference; γ is the discount factor: between 0 and 1, balancing the importance of current rewards and future rewards; Double DQN is used to solve the above MDP problem; Double DQN uses an online network Q o (s t ,a t ;θ) and the target network Q T (s t ,a t θ - ) to reduce the overestimation problem of Q value, where θ and θ - Q o (s t ,a t ;θ) and Q T (s t ,a t θ - )’s neural network parameters; they are designed as follows: First, according to the ε-greedy strategy, in a given state s t Select action a k : ε is a probability between 0 and 1, indicating the possibility of random exploration by the system. In each decision-making step, the system generates a random number. If this random number is less than ε, the system will choose a random action. Otherwise, the system will choose the action that maximizes the current Q value. o (s t ,a t ; θ) is the action value function, which represents the expected return; argmax a Indicates Q o (s t ,a t ;θ) maximizes the action; Then calculate the state transition probability and reward: Transfer to S k The probability of is calculated as follows: in, YesS k The maximum amount of business that can be received, B k YesS k Current business volume; Take action a k The reward value is: Then use the Bellman equation to calculate Q o To update: Among them, Q p (s t ,a k ; θ) is the selection action a k When the Q value before update, max a′ Q T (s t+1 ,a k θ - ) is the maximum Q value of the next state estimated by the target network; Finally, the convergence check of the Q value is performed, and the design is as follows: ΔQ=∣Q o (S t ,a k ;θ)-Q p (S t ,a k ;θ)∣<δ (26) Where: δ is a small threshold. If the value of ΔQ is less than δ after multiple consecutive updates, it can be considered that the Q value has converged, that is, it has been found. The optimal allocation strategy.

Images