Resource allocation method, allocation system, equipment and medium for satellite communication system

Abstract

This application relates to the field of satellite communication technology and discloses a resource allocation method, allocation system, device, and medium for a satellite communication system. The method includes establishing a resource allocation model, wherein the optimization objective of the resource allocation model is to maximize the total capacity of the satellite communication system, and the optimization variables are the beam allocation factor for allocating satellite beams to wave positions, the subchannel allocation factor for allocating satellite beam subchannels to terminals, and the transmit power transmitted from the satellite to the terminal. Based on the resource allocation model, terminal power and beam subchannels are allocated to each terminal; the beam subchannels are the beam allocation factor and the subchannel allocation factor. This application achieves flexible allocation and management of resources, improving spectrum efficiency and resource utilization.

Description

Technical Field

[0001] The present application relates to the field of satellite communication technology, and in particular to a resource allocation method, allocation system, equipment and medium for a satellite communication system. Background Art

[0002] Currently, the scheduling of users within a beam is not fully considered, resulting in inefficient resource utilization. Furthermore, resource allocation schemes fail to account for the coupling between beams and multi-dimensional resources such as frequency and power, limiting optimal system performance. While beam-hopping technology in satellite systems has improved the positioning performance of communication networks, the impact of resource allocation on system capacity has not been thoroughly studied. Summary of the Invention

[0003] In view of this, the present application provides a resource allocation method, allocation system, device and medium for a satellite communication system, which aims to achieve a higher total system capacity and combines beam allocation, sub-channel allocation and power allocation to allocate resources for the satellite communication system, thereby realizing flexible allocation and management of resources and improving spectrum efficiency and resource utilization.

[0004] The present application discloses a resource allocation method for a satellite communication system, which includes:

[0005] Establishing a resource allocation model, wherein the optimization objective of the resource allocation model is to maximize the total capacity of the satellite communication system, and the optimization variables are a beam allocation factor for allocating satellite beams to wave positions, a subchannel allocation factor for allocating satellite beam subchannels to terminals, and a transmit power transmitted from the satellite to the terminal;

[0006] Based on the resource allocation model, terminal power and beam sub-channel are allocated to each terminal; the beam sub-channel is the beam allocation factor and the sub-channel allocation factor.

[0007] Furthermore, allocating terminal power and beam sub-channel to each terminal based on the resource allocation model includes:

[0008] Converting the resource allocation model into a power allocation problem, approximating the power allocation problem as a geometric programming problem, substituting the known beam subchannel allocation result into the geometric programming problem, and obtaining a terminal power allocation result;

[0009] Converting the resource allocation model into a beam subchannel allocation problem, modeling the beam subchannel allocation problem as a Markov decision process, executing the Markov decision process, and obtaining a beam subchannel allocation result;

[0010] According to the terminal power allocation result and the beam sub-channel allocation result, an optimal terminal power allocation result and beam sub-channel allocation result are obtained.

[0011] Furthermore, converting the resource allocation model into a power allocation problem includes:

[0012] Assuming that the beam subchannel allocation is known, the expression of the total capacity of the satellite communication system is updated, that is, the power allocation problem is obtained; the total capacity of the satellite communication system before the update is determined according to the beam allocation factor of the satellite beam allocated to the wave position, the subchannel allocation factor of the satellite beam subchannel allocated to the terminal, the bandwidth of a single subchannel, and the signal-to-interference-and-noise ratio of the terminal; the total capacity of the satellite communication system after the update is determined according to the beam allocation factor of the satellite beam allocated to the wave position, the subchannel allocation factor of the satellite beam subchannel allocated to the terminal, the bandwidth of a single subchannel, the signal-to-interference-and-noise ratio of the terminal, the interference of the terminal within the beam, the interference of the terminal between intra-satellite beams, and the interference of the terminal between inter-satellite beams.

[0013] Furthermore, approximating the power allocation problem as a geometric programming problem includes:

[0014] The transmission rate and the minimum transmission rate in the third constraint of the resource allocation model are converted into a new third constraint, where the new third constraint is related to interference of a terminal within a beam, interference of a terminal between beams within a satellite, interference of a terminal between beams between satellites, and bandwidth of a single subchannel;

[0015] The optimization objective of the resource allocation model is replaced by minimizing the specified variable, and the optimization variable is replaced by the transmission power transmitted from the satellite to the terminal; the constraint condition is replaced by the first constraint condition, the second constraint condition, the new third constraint condition and the eleventh constraint condition, that is, the power allocation problem is approximated as a geometric programming problem; the eleventh constraint condition is related to the interference of the terminal within the beam, the interference of the terminal between the intra-satellite beams, the interference of the terminal between the inter-satellite beams and the specified variable.

[0016] Furthermore, the constraints of the resource allocation model include:

[0017] The first constraint is that the transmission power of each satellite does not exceed the maximum transmission power; the second constraint is that the power allocated to each terminal is not less than zero; the third constraint is that the transmission rate on each subchannel is not less than the minimum transmission rate; the fourth constraint is that the beam allocation factor of the satellite's beam allocated to the wave position is used to indicate unallocated or allocated; the fifth constraint is that the subchannel allocation factor of the satellite's beam allocated to the terminal is used to indicate unallocated or allocated; the sixth constraint is that only terminals within the wave position illuminated by the beam can be scheduled; the seventh constraint is that each beam of each satellite can be allocated to at most one wave position; the eighth constraint is that each wave position can be allocated to at most one beam on each satellite; the ninth constraint is that each terminal can be allocated at mostN sub-channels; the tenth constraint: each sub-channel can be superimposed on N max terminals; N and N max All are positive integers.

[0018] Furthermore, converting the resource allocation model into a beam sub-channel allocation problem and modeling the beam sub-channel allocation problem as a Markov decision process includes:

[0019] Setting possible values ​​of the beam allocation factor and the subchannel allocation factor that satisfy the constraints in the resource allocation model into a state space;

[0020] Setting possible changes of the beam allocation factor and the subchannel allocation factor that satisfy the constraints in the resource allocation model as an action space;

[0021] According to the terminal power allocation result and the given beam subchannel allocation result, a reward function for selecting an action to be executed is set at the state of the current time slot.

[0022] Furthermore, setting a reward function for selecting an action to be executed based on the state of the current time slot according to the terminal power allocation result and the given beam subchannel allocation result includes:

[0023] Calculating the total capacity of the satellite communication system in the current time slot according to the terminal power allocation result and the given beam subchannel allocation result;

[0024] Based on the total capacity of the satellite communication system, a reward function for selecting an action to be executed in the state of the current time slot is set; the reward function is related to the total capacity of the satellite communication system in the current time slot.

[0025] Furthermore, executing the Markov decision process to obtain a beam subchannel allocation result includes:

[0026] Cycle through time steps and update the strategy. For each time step, the agent uses the current values ​​of the beam allocation factor and the subchannel allocation factor as the state, selects an action (i.e., a beam subchannel allocation result) according to the strategy, calculates a reward value according to the reward function, executes the action, and transitions to the next state. The state transition trajectory is recorded in the experience pool. One time step corresponds to one time slot.

[0027] Copy the parameters of the policy network to the old policy network to maintain the old policy, update the parameters of the old policy network in the proximal policy optimization algorithm, update the parameters of the policy network and the value network, and obtain the updated policy;

[0028] Empty the experience pool;

[0029] Until the action that maximizes the cumulative reward is found, that is, a set of beam allocation factors and sub-channel allocation factor values, then the values ​​of the beam allocation factors and sub-channel allocation factors, that is, the beam-sub-channel allocation result, are output.

[0030] Furthermore, the parameters of the policy network are copied to the old policy network to maintain the old policy, the parameters of the old policy network in the proximal policy optimization algorithm are updated, the parameters of the policy network and the parameters of the value network are updated, and the updated policy is obtained, including:

[0031] Fitting the policy of the agent using a policy network, copying the parameters of the policy network to the old policy network to maintain the old policy, and updating the parameters of the policy network training process each time the agent collects the state transition trajectory of the specified number of time steps;

[0032] The state value is estimated using the value network, and the training goal is to update the parameters of the value network based on minimizing the mean square error loss function;

[0033] The updated strategy is obtained according to the updated parameters of the strategy network and the updated parameters of the value network.

[0034] Furthermore, obtaining an optimal terminal power allocation result and beam subchannel allocation result based on the terminal power allocation result and the beam subchannel allocation result includes:

[0035] Obtaining the total capacity of the satellite communication system according to the terminal power allocation result and the beam sub-channel allocation result;

[0036] According to the total capacity of the satellite communication system, the optimal terminal power allocation result and beam sub-channel allocation result are obtained.

[0037] Furthermore, obtaining the total capacity of the satellite communication system according to the terminal power allocation result and the beam sub-channel allocation result includes:

[0038] For each of the terminal power allocation results and each of the beam sub-channel allocation results, calculating the total system capacity based on the terminal power allocation result and the beam sub-channel allocation result;

[0039] Substituting the beam subchannel allocation result into the geometric programming problem to obtain an updated terminal power allocation result; and recalculating the total system capacity based on the beam subchannel allocation result and the updated terminal power allocation result.

[0040] Furthermore, obtaining an optimal terminal power allocation result and beam sub-channel allocation result based on the total capacity of the satellite communication system includes:

[0041] If the difference between the calculated total capacity of the satellite communication system and the recalculated total capacity of the satellite communication system is not equal, substituting the updated terminal power allocation result into a Markov decision process, executing the Markov decision process, and obtaining an updated beam subchannel allocation result;

[0042] The updated beam subchannel allocation result replaces the beam subchannel allocation result, and the step of obtaining the total capacity of the satellite communication system based on the terminal power allocation result and the beam subchannel allocation result is executed cyclically until the difference between the most recently calculated total capacity of the satellite communication system and the last calculated total capacity of the satellite communication system is equal. At this time, the optimal terminal power allocation result and the optimal beam subchannel allocation result are output.

[0043] Furthermore, obtaining an optimal terminal power allocation result and beam sub-channel allocation result based on the total capacity of the satellite communication system includes:

[0044] If the difference between the calculated total capacity of the satellite communication system and the recalculated total capacity of the satellite communication system is equal, the beam sub-channel allocation result and the updated terminal power allocation result are output as the optimal allocation result.

[0045] The present application also discloses a resource allocation system for a satellite communication system, which includes:

[0046] A model building module is used to establish a resource allocation model, wherein the optimization goal of the resource allocation model is to maximize the total capacity of the satellite communication system, and the optimization variables are the beam allocation factor of the satellite beam allocated to the wave position, the subchannel allocation factor of the satellite beam allocated to the terminal, and the transmission power of the satellite to the terminal;

[0047] The resource allocation result acquisition module is used to allocate terminal power and beam sub-channel to each terminal based on the resource allocation model; the beam sub-channel is the beam allocation factor and the sub-channel allocation factor.

[0048] The present application also discloses an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the resource allocation method for the satellite communication system described in any one of the above items is implemented.

[0049] The present application also discloses a computer-readable storage medium, which includes a computer program or instructions. When the computer program or instructions are executed on a computer, the computer executes any one of the above-mentioned resource allocation methods for a satellite communication system.

[0050] Due to the adoption of the above technical solution, this application has the following advantages: this application realizes more accurate allocation and more efficient utilization of resources, and improves the performance of the satellite communication system and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0052] Figure 1 This is a flow chart of a resource allocation method for a satellite communication system provided by an embodiment of the present application;

[0053] Figure 2 is a model diagram of a satellite communication system provided in an embodiment of the present application;

[0054] Figure 3 This is an architectural diagram of proximal strategy optimization and geometric planning provided by an embodiment of the present application;

[0055] Figure 4 This is a flowchart of an iterative algorithm for proximal strategy optimization and geometric programming provided by an embodiment of the present application;

[0056] Figure 5 This is a graph of accumulated reward training for the three algorithms provided in the embodiments of this application;

[0057] Figure 6 This is a convergence diagram of the geometric programming power allocation algorithm provided in an embodiment of the present application under the minimum rate requirements of three sub-channels;

[0058] Figure 7 is a histogram of subchannel capacities calculated by the geometric programming power allocation algorithm provided in an embodiment of the present application;

[0059] Figure 8 This is a graph showing the variation trend of the total system capacity with the transmit power of a single satellite under four different schemes provided in the embodiments of the present application;

[0060] Figure 9 This is a relationship diagram between the total system capacity and the minimum sub-channel rate requirements under four different schemes provided in the embodiments of the present application. DETAILED DESCRIPTION

[0061] The present application is further described with reference to the accompanying drawings and embodiments. The embodiments described are only a part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of the embodiments of the present application.

[0062] See also Figure 1 The present application provides an embodiment of a resource allocation method for a satellite communication system, which includes:

[0063] Step 101: Establish a resource allocation model. The optimization objective of the resource allocation model is to maximize the total capacity of the satellite communication system. The optimization variables are the beam allocation factor of the satellite beam allocated to the wave position, the subchannel allocation factor of the satellite beam subchannel allocated to the terminal, and the transmission power of the satellite to the terminal.

[0064] Optionally, each beam of each satellite is allocated to at most one terminal; each beam of each satellite is allocated to at most one beam position.

[0065] The comprehensive performance optimization of this application: integrates beam hopping and non-orthogonal multiple access technology, and achieves comprehensive optimization of overall performance by jointly optimizing beam allocation, sub-channel allocation and power allocation, with the goal of maximizing the total capacity of the satellite communication system.

[0066] In one embodiment of the present application, see Figure 2 , also includes:

[0067] Satellite communication systems include S Satellites and In the wave terminals; S Satellites with a collection Represents a collection The s Satellite s , s= 1,2,…, S ; Terminal collection Represents a collection No. m Terminal m , m= 1,2,…, M ;satellite s have K beams, satellite s of K The beams are represented by a set ,gather The k Satellite s Beam k , k= 1,2,…, K ;satellite s Beam k Total N sub-channels, using the set Represents a collection The n Satellite s Beam k Subchannel n , n= 1,2,…, N ;satellite s Shared Services Z Wavelength, satellite s Service Z Wavelength Set Represents a collection The z Satellite s Wave position z , z= 1,2,…, Z .

[0068] satellite s Beam k Subchannel n The transmitted superposition signal is expressed as:

[0069]

[0070] in, Indicates satellite s Beam k Subchannel n The terminal collection in and Represents satellites s Transfer to Collection Terminal in m Right now The transmission power and transmission signal.

[0071] terminal The received signal is represented as:

[0072]

[0073] in, For terminal With satellite Beam The channel gain between and Represents satellites s Transfer to Collection Terminal m Different terminals Right now The transmission power and transmission signal, ; Indicates satellite s Beam j Subchanneln The terminal collection in j ≠ k ; For collection mid-terminal Right now With satellite s Beam j The channel gain between and Represents satellites s Transfer to Collection mid-terminal Right now The transmission power and transmission signal; Indicates satellite of K A collection of beams; Indicates satellite Beam j Subchannel n The terminal collection in i ≠ s ; For collection mid-terminal Right now With satellite Beam j The channel gain between and Represents satellites Transfer to Collection mid-terminal Right now The transmission power and transmission signal; Indicates terminal With satellite s Beam k Subchannel n The mean between them is 0 and the variance is Additive white Gaussian noise.

[0074] Defining a terminal With satellite s Beam k The noise normalized channel gain between , then the terminal The signal-to-interference-noise ratio is expressed as:

[0075]

[0076] in, Indicates the interference within the beam (terminal In satellite s Beam k The specific expression is:

[0077]

[0078] Boolean variables Indicates satellite s Beam k Subchannel n Assign to terminal l The subchannel allocation factor of the satellite s Beam k Subchannel n Assign to terminal but ,otherwise ;terminal With satellite s Beam k The noise normalized channel gain between ;

[0079] in, Indicates the interference between intra-satellite beams (terminal In satellite The specific expression is:

[0080]

[0081] Among them, Boolean variables Indicates satellite s Beam j Assigned to wave position z The beam allocation factor of the satellite s Beam j Assigned to wave position z but ,otherwise , j ≠ k ; Indicates satellite s Beam j Subchannel n Assign to terminal l The subchannel allocation factor of the satellite s Beam j Subchannel n Assign to terminal l but ,otherwise ;terminal With satellite s Beam k The noise normalized channel gain between ;

[0082] in, Indicates the interference between inter-satellite beams (terminal Interference between different satellites), specifically expressed as:

[0083]

[0084] Among them, Boolean variables Indicates satellite Beam j Assigned to wave position z The beam allocation factor of the satellite Beam j Assigned to wave position z but ,otherwise .

[0085] in, Indicates satellite s Beam k Subchannel n Assign to terminal The subchannel allocation factor, Indicates satellite s Transfer to Collection Terminal m Different terminals Right now The transmission power, Indicates terminal With satellite s Beam k The noise normalized channel gain between ; Indicates satellite s Beam j Assigned to wave position z The beam allocation factor, Indicates satellite s Beam j Subchannel n Assign to terminal The subchannel allocation factor, ; Indicates satellite s Transfer to Collection mid-terminal Right now The transmission power, Indicates satellite s Beam j Subchannel n The terminal collection in Indicates terminal With satellite s Beam j The noise normalized channel gain between ; Indicates satellite of K A collection of beams ; Indicates satellite Transfer to Collection mid-terminal Right now The transmission power, Indicates satellite Beam j Subchannel n The terminal collection in; Indicates satellite Beam j Assigned to wave position z The beam allocation factor, Indicates satellite Beam j Subchannel n Assign to terminal The subchannel allocation factor, Indicates terminal With satellite Beam j The noise normalized channel gain between .

[0086] In one embodiment of the present application, a resource allocation model is established. The optimization objective of the resource allocation model is to maximize the total capacity of the satellite communication system. The optimization variables are the beam allocation factor of the satellite beam allocated to the beam position, the subchannel allocation factor of the satellite beam allocated to the terminal, and the transmission power of the satellite to the terminal, including:

[0087] To maximize the total capacity of the satellite communication system To optimize the target, where the objective function is the total capacity of the satellite communication system, the established resource allocation model is:

[0088]

[0089] st Constraints C1 to C10:

[0090] Constraint C1: The transmission power of each satellite does not exceed the maximum transmission power P max , which can be expressed as:

[0091]

[0092] Constraint C2: The power allocated to each terminal is not less than zero, which can be expressed as:

[0093]

[0094] Constraint C3: The transmission rate on each subchannel is not less than the minimum transmission rate Rmin , which can be expressed as:

[0095]

[0096] Constraint C4: Satellite s Beam k Assigned to wave position z Beam allocation factor It can only be 0 or 1, 0 means unallocated, 1 means allocated, and the formula is:

[0097]

[0098] Constraint C5: Satellite s Beam k Subchannel n Assign to terminal m Subchannel allocation factor It can only be 0 or 1, 0 means unallocated, 1 means allocated, and the formula is:

[0099]

[0100] Constraint C6: Only terminals within the beam position illuminated by the beam can be scheduled, which can be expressed as:

[0101]

[0102] in, Indicates wave position z The terminal collection within;

[0103] Constraint C7: Each beam of each satellite can be assigned to at most one beam position, which can be expressed as:

[0104]

[0105] Constraint C8: Each beam position can be assigned at most one beam on each satellite, which can be expressed as:

[0106]

[0107] Constraint C9: Each terminal is allocated at most N sub-channels, which can be expressed as:

[0108]

[0109] Constraint C10: Each subchannel can be superimposed at most N max terminals, expressed as:

[0110]

[0111] in, represents maximization, st Indicates the need to satisfy, Indicates any Indicates satellite s Beam k Assigned to wave position z The beam allocation factor, Indicates satellite s Beam k Subchannel n Assign to terminal m The subchannel allocation factor, Indicates satellite s Transfer to Collection Terminal in m Right now The transmission power of Indicates satellite s Beam k Subchannel n The terminal collection in represents the maximum transmit power of each satellite, R min Indicates the preset minimum transmission rate, N max Indicates the maximum number of terminals that can be superimposed on each sub-channel.

[0112] The constraints that need to be met during resource allocation (constraints C1 to C10) are requirements that need to be met when the satellite communication system operates.

[0113] The resource allocation model of joint beam subchannel allocation and power allocation can be expressed as:

[0114]

[0115] st Indicates that something needs to be satisfied.

[0116] Step 102: Based on the resource allocation model, allocate terminal power and beam subchannel to each terminal; the beam subchannel is a beam allocation factor and a subchannel allocation factor.

[0117] In one embodiment of the present application, step 102 includes:

[0118] Step 1021: Convert the resource allocation model into a power allocation problem, approximate the power allocation problem as a geometric programming problem, substitute the known beam subchannel allocation result into the geometric programming problem, and obtain the terminal power allocation result;

[0119] In one embodiment of the present application, assuming that the beam subchannel allocation is known, the expression for the total capacity of the satellite communication system is updated, that is, the power allocation problem is obtained. The total capacity of the satellite communication system before the update is determined based on the beam allocation factor of the satellite beam allocated to the wave position, the subchannel allocation factor of the satellite beam subchannel allocated to the terminal, the bandwidth of a single subchannel, and the signal-to-interference-and-noise ratio of the terminal. The total capacity of the satellite communication system after the update is determined based on the beam allocation factor of the satellite beam allocated to the wave position, the subchannel allocation factor of the satellite beam subchannel allocated to the terminal, the bandwidth of a single subchannel, the signal-to-interference-and-noise ratio of the terminal, the interference of the terminal within the beam, the interference of the terminal between intra-satellite beams, and the interference of the terminal between inter-satellite beams.

[0120] The transmission rate and minimum transmission rate in the third constraint of the resource allocation model are converted into a new third constraint. The new third constraint is related to the interference of terminals within a beam, the interference of terminals between beams within a satellite, the interference of terminals between beams between satellites, and the bandwidth of a single subchannel.

[0121] The optimization objective of the resource allocation model is replaced by minimizing a specified variable, and the optimization variable is replaced by the transmit power transmitted from the satellite to the terminal. The constraints are replaced by the first constraint, the second constraint, the new third constraint, and the eleventh constraint, that is, the power allocation problem is approximated as a geometric programming problem. The eleventh constraint is related to the interference of the terminal within the beam, the interference of the terminal between the intra-satellite beams, the interference of the terminal between the inter-satellite beams, and the specified variable.

[0122] Solve the geometric programming problem (using the CVX toolbox) to obtain the terminal power allocation when the beam subchannel allocation is known.

[0123] In one possible implementation, assuming that the beam sub-channel allocation is known (i.e. Given), the total capacity of the satellite communication system The expression is transformed into:

[0124]

[0125] The rest of the resource allocation model remains unchanged, which means the power allocation problem is obtained;

[0126] in, B is the bandwidth of a single subchannel, Indicates satellite s Beam k Assigned to wave position z The beam allocation factor, Indicates satellite s Beam k Subchannel n Assign to terminal m The subchannel allocation factor, Indicates terminal With satellite s Beam k The noise normalized channel gain between Indicates terminal Interference within the beam, Indicates terminal Interference between intra-satellite beams, Indicates terminal Interference between intersatellite beams, terminals With satellite s Beam k The noise normalized channel gain between , Indicates satellite s Transfer to Collection Terminal in m Right now The transmission power of Indicates satellite s Beam k Subchannel n The terminal collection in; Represents a collection of terminals, Indicates satellite s Service Z A collection of wave positions, Indicates satellite s of K A collection of beams, express S A collection of satellites;

[0127] Using parameters Total capacity of alternative satellite communication systems In the expression ,calculate The geometric mean approximation of :

[0128]

[0129] Among them, π represents the cumulative multiplication symbol, 、 、 、 、 are parameters used for approximation, Indicates satellite s Beam k Subchannel n Assign to terminal The subchannel allocation factor, Indicates satellite s Beam j Subchanneln Assign to terminal The subchannel allocation factor, Indicates satellite Beam j Subchannel n Assign to terminal The subchannel allocation factor, Indicates terminal With satellite s Beam j The noise normalized channel gain between Indicates terminal With satellite Beam j The noise normalized channel gain between Represents a collection of terminals, Indicates satellite s Service Z Wave positions are used as a set, Indicates satellite Service Z Wave positions are used as a set, Indicates satellite s of K A collection of beams, express S A collection of satellites, Indicates satellite of K A collection of beams; Indicates satellite s Transfer to Collection Terminal m Different terminals Right now The transmission power, Indicates satellite s Beam k Subchannel n The terminal collection in Indicates satellite s Transfer to Collection mid-terminal Right now The transmission power, Indicates satellite s Beam j Subchannel n The terminal collection in Indicates satellite Transfer to Collection mid-terminal Right now The transmission power, Indicates satellite Beam j Subchanneln The terminal collection in;

[0130] Based on the geometric mean approximation , constraint C3 is reformulated as constraint :

[0131] when hour,

[0132]

[0133] Introducing auxiliary variables , converting the maximum total capacity of the satellite communication system into the minimum , and construct the constraint condition C11 of the resource allocation model, approximating the power allocation problem as a geometric programming problem:

[0134]

[0135] st C1 to ,

[0136] C11:

[0137]

[0138] The expressions for the parameters used for the approximation are:

[0139]

[0140] Step 1022: Assume that the terminal power allocation result (ie ) It is known that the resource allocation model is converted into a beam subchannel allocation problem, and the beam subchannel allocation problem is modeled as a Markov decision process. The Markov decision process is executed to obtain the beam subchannel allocation result;

[0141] In one embodiment of the present application, possible values ​​of the beam allocation factor and the subchannel allocation factor that satisfy the constraints in the resource allocation model are set as a state space; possible changes of the beam allocation factor and the subchannel allocation factor that satisfy the constraints in the resource allocation model are set as an action space; the total capacity of the satellite communication system in the current time slot is calculated based on the terminal power allocation result and the given beam subchannel allocation result; based on the total capacity of the satellite communication system, a reward function for selecting an action to be executed in the state of the current time slot is set; the reward function is related to the total capacity of the satellite communication system in the current time slot; the time step is looped and the strategy is updated. For each time step, the agent selects the current beam allocation factor and the subchannel allocation factor. The value of the sub-sub-channel is used as the state, an action is selected according to the strategy, that is, a beam sub-channel allocation result, the reward value is calculated according to the reward function, the agent implements the action and transfers to the next state, and the state transition trajectory is recorded in the experience pool; one time step corresponds to one time slot; the parameters of the policy network are copied to the old policy network to maintain the old policy, the parameters of the old policy network in the proximal policy optimization algorithm are updated, the parameters of the policy network and the parameters of the value network are updated to obtain the updated policy; the experience pool is cleared; until the action that maximizes the cumulative reward is found, that is, a set of beam allocation factors and sub-channel allocation factor values, at this time, the values ​​of the beam allocation factors and sub-channel allocation factors are output, that is, the beam sub-channel allocation result.

[0142] Optionally, based on the total capacity of the satellite communication system, a reward function for selecting an action to be executed based on the state of the current time slot is set, including:

[0143] Based on the total capacity of the satellite communication system, the current time slot t Status Select an action The reward function for execution is set to:

[0144]

[0145] in, represents the reward function; is a penalty term. The more sub-channels that do not meet the rate constraint, the greater the penalty value. and are different positive constants used to adjust the rewards to the appropriate order of magnitude; Indicates the current time slot t The total capacity of the satellite communication system.

[0146] Figure 3 This is the architecture diagram of proximal strategy optimization and geometric programming, and the iterative algorithm flow chart of proximal strategy optimization and geometric programming is as follows Figure 4As shown in Figure 2, near-end policy optimization uses the policy network, old policy network, and value network to obtain the beam subchannel allocation results. The power allocation problem is approximated as a geometric programming problem through geometric mean approximation, resulting in the terminal power allocation result.

[0147] In one achievable approach, Figure 3 、 Figure 4 As shown, step 1022 includes the following steps:

[0148] Step 10221: At each time step t The agent uses the current beam allocation factor and subchannel allocation factor as the state , according to the strategy Select an action That is, a beam sub-channel allocation result, the reward value is calculated according to the reward function, and the agent performs the action and move to the next state;

[0149] Step 10222: Use the policy network to fit the agent's policy , copy the parameters of the policy network to the old policy network to keep the old policy, the agent collects The state transition trajectory of the time step updates the parameters of the policy network training process :

[0150] in, Indicates status Next action The advantage function value of are the parameters of the old policy network, is the cropping factor, Indicates status Next action The policy network value of Indicates status Next action The old policy network value of Represents a clipping function. The first input parameter is the value to be clipped, the second is the lower limit of clipping, and the third is the upper limit of clipping. Represents the parameter when the function achieves the maximum value in a given input set;

[0151] Step 10223: Estimate state value using the value network The training goal is to update the parameters of the value network based on minimizing the mean square error loss function :

[0152]

[0153] in, Status The status value of is the accumulated reward calculated based on the state transition trajectory data, Represents the parameter when the function obtains the minimum value in a given input set, is the mean square error loss function;

[0154] Step 10224: Obtain a new strategy based on the updated parameters of the strategy network and the value network.

[0155] Step 10225: Repeat steps 10221 to 10224 until an action that maximizes the cumulative reward is found, that is, a set of values ​​of the beam allocation factor and the subchannel allocation factor.

[0156] Step 1023: Obtain the optimal terminal power allocation result and beam sub-channel allocation result according to the terminal power allocation result and the beam sub-channel allocation result.

[0157] In one embodiment of the present application, for each terminal power allocation result and each beam subchannel allocation result, the total system capacity is calculated based on the terminal power allocation result and the beam subchannel allocation result; the beam subchannel allocation result is substituted into the geometric programming problem to obtain an updated terminal power allocation result; and the total system capacity is recalculated based on the beam subchannel allocation result and the updated terminal power allocation result.

[0158] If the difference between the calculated total capacity of the satellite communication system and the recalculated total capacity of the satellite communication system is not equal, the updated terminal power allocation result is substituted into the Markov decision process, and the Markov decision process is executed to obtain an updated beam subchannel allocation result; the updated beam subchannel allocation result replaces the beam subchannel allocation result, and the step of obtaining the total capacity of the satellite communication system according to the terminal power allocation result and the beam subchannel allocation result is repeatedly executed until the difference between the latest calculated total capacity of the satellite communication system and the last calculated total capacity of the satellite communication system is equal, at which time the optimal terminal power allocation result and the optimal beam subchannel allocation result are output.

[0159] In one embodiment of the present application, for each terminal power allocation result and each beam subchannel allocation result, the total system capacity is calculated based on the terminal power allocation result and the beam subchannel allocation result; the beam subchannel allocation result is substituted into the geometric programming problem to obtain an updated terminal power allocation result; and the total system capacity is recalculated based on the beam subchannel allocation result and the updated terminal power allocation result.

[0160] If the difference between the calculated total capacity of the satellite communication system and the recalculated total capacity of the satellite communication system is equal, the beam subchannel allocation result and the updated terminal power allocation result are output as the optimal allocation result.

[0161] The satellite communication system provided in the embodiment of the present application integrates beam hopping and non-orthogonal multiple access technology, can freely schedule beams and allocate sub-channels, and at the same time allow terminals to transmit information on the same sub-channel, thereby improving the flexibility of multi-dimensional resource utilization in the frequency domain and spatial domain.

[0162] In summary, the resource allocation method for a satellite communication system provided in an embodiment of the present application aims to maximize the total system capacity of the satellite communication system by establishing a resource allocation model that combines beam subchannel allocation and power allocation. The power allocation problem in the resource allocation model is approximated as a geometric programming problem, and the beam subchannel allocation problem in the resource allocation model is modeled as a Markov decision problem. The resource allocation model is solved by iteratively solving the geometric programming problem and the Markov decision problem to obtain beam subchannel allocation and power allocation strategies. This application integrates beam allocation, subchannel allocation, and power allocation to achieve a higher total system capacity than traditional heuristic algorithms.

[0163] The following simulation verification is carried out. Figures 5 to 9 The corresponding simulation results.

[0164] Figure 5 The cumulative reward training curves for proximal policy optimization scheduling combined with geometric programming for power allocation and the other two algorithms (proximal policy optimization scheduling combined with average power allocation, and proximal policy optimization scheduling combined with power allocation according to channel gain ratio). As the number of training rounds increases, the three algorithms gradually converge and reach a stable state after about 40 training rounds. After this point, the cumulative reward no longer fluctuates significantly or improves significantly. This shows that the three algorithms have learned a better strategy and are able to obtain relatively stable rewards in terminal scheduling tasks. However, it is not difficult to see from the comparison that the resource allocation scheme of proximal policy optimization scheduling combined with geometric programming for power allocation provided by the present application has a much higher cumulative reward than the other two schemes, which shows that the resource allocation scheme provided by the present application is superior.

[0165] Figure 6 The convergence of the geometric programming power allocation algorithm under the three sub-channel minimum rate requirements is shown. The three sub-channel minimum rate requirements are R min =5000bit / s, R min =8000bit / s, R min =10000bit / s. Figure 6 As can be seen, after the number of iterations exceeds 80, the algorithm's convergence performance reaches a stable level and no longer shows significant improvement, indicating that the algorithm has converged to a near-optimal solution. Furthermore, as the minimum rate requirement for the subchannel increases, the power allocation solution space is compressed, resulting in a smaller total capacity after convergence.

[0166] Figure 7 Histogram of subchannel capacities calculated for the geometric programming power allocation algorithm. Figure 7 It can be seen that the subchannel capacity of each terminal in each terminal group (including one strong terminal and one weak terminal) is above the minimum rate requirement line, which means that the algorithm fully considers the minimum rate requirement during the optimization process and ensures that each subchannel can meet the minimum communication needs of the terminal.

[0167] Figure 8 The paper shows the variation trend of total system capacity with the transmission power of a single satellite under four different schemes, namely, proximal strategy optimization scheduling + geometric planning power allocation (this application), scheduling according to channel gain + geometric planning power allocation, proximal strategy optimization scheduling + power allocation according to channel gain ratio, and proximal strategy optimization scheduling + average power allocation. Figure 8 It can be seen that as the transmit power increases, the total system capacity improves, and the iterative algorithm of proximal strategy optimization and geometric programming (this application) performs better than other comparison algorithms.

[0168] Figure 9 The relationship between the total system capacity and the minimum sub-channel rate requirement under four different schemes is shown. Figure 8 The four schemes shown are identical. Figure 9 As can be seen, as the minimum subchannel rate requirement increases, the total system capacity decreases. As the minimum subchannel rate requirement increases, the power allocation solution space becomes compressed, resulting in a decrease in total capacity. Compared to scheduling based on channel gain, the resource scheduling scheme using proximal policy optimization scheduling and geometric programming power allocation (this application) achieves higher total capacity.

[0169] The simulation results prove that the resource allocation method and system for satellite communication systems provided in this application, which combines proximal strategy optimization scheduling and geometric planning to allocate power, can achieve higher system capacity compared with the comparison algorithm.

[0170] The present application also provides a resource allocation system for a satellite communication system, which includes:

[0171] A model building module is used to establish a resource allocation model, wherein the optimization goal of the resource allocation model is to maximize the total capacity of the satellite communication system, and the optimization variables are the beam allocation factor of the satellite beam allocated to the wave position, the subchannel allocation factor of the satellite beam allocated to the terminal, and the transmission power of the satellite to the terminal;

[0172] The resource allocation result acquisition module is used to allocate terminal power and beam sub-channel to each terminal based on the resource allocation model; the beam sub-channel is the beam allocation factor and the sub-channel allocation factor.

[0173] An embodiment of the present application further provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the resource allocation method for the satellite communication system described in the above embodiment is implemented.

[0174] An embodiment of the present application further provides a computer-readable storage medium, which includes a computer program or instructions. When the computer program or instructions are executed on a computer, the computer executes the resource allocation method for the satellite communication system described in the above embodiment.

[0175] It should be noted that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0176] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them. Although the present application has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present application can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present application should be included in the scope of protection of the claims of the present application.

Claims

1. A resource allocation method for a satellite communication system, characterized in that: include: Establishing a resource allocation model, wherein the optimization objective of the resource allocation model is to maximize the total capacity of the satellite communication system, and the optimization variables are a beam allocation factor for allocating satellite beams to wave positions, a subchannel allocation factor for allocating satellite beam subchannels to terminals, and a transmit power transmitted from the satellite to the terminal; Allocate terminal power and beam subchannel to each terminal based on the resource allocation model; the beam subchannel is the beam allocation factor and the subchannel allocation factor; The allocating terminal power and beam sub-channel to each terminal based on the resource allocation model includes: Converting the resource allocation model into a power allocation problem, converting the power allocation problem into a geometric programming problem, substituting the known beam subchannel allocation result into the geometric programming problem, and obtaining a terminal power allocation result; Converting the resource allocation model into a beam subchannel allocation problem, modeling the beam subchannel allocation problem as a Markov decision process, executing the Markov decision process, and obtaining a beam subchannel allocation result; Obtaining an optimal terminal power allocation result and beam subchannel allocation result according to the terminal power allocation result and the beam subchannel allocation result; Converting the resource allocation model into a power allocation problem includes: Assuming that the beam subchannel allocation is known, the expression for the total capacity of the satellite communication system is updated, that is, the power allocation problem is obtained; the updated total capacity of the satellite communication system is determined based on the beam allocation factor of the satellite beam allocated to the wave position, the subchannel allocation factor of the satellite beam subchannel allocated to the terminal, the bandwidth of a single subchannel, the signal-to-interference-and-noise ratio of the terminal, the interference of the terminal within the beam, the interference of the terminal between the intra-satellite beams, and the interference of the terminal between the inter-satellite beams; Converting the power allocation problem into a geometric programming problem includes: The transmission rate and the minimum transmission rate in the third constraint of the resource allocation model are converted into a new third constraint, where the new third constraint is related to interference of a terminal within a beam, interference of a terminal between beams within a satellite, interference of a terminal between beams between satellites, and bandwidth of a single subchannel; The optimization objective of the resource allocation model is replaced by minimizing the specified variable, and the optimization variable is replaced by the transmission power transmitted from the satellite to the terminal; the constraints are replaced by the first constraint, the second constraint, the new third constraint and the eleventh constraint, that is, the power allocation problem is converted into a geometric programming problem; the eleventh constraint is related to the interference of the terminal within the beam, the interference of the terminal between the beams within the satellite, the interference of the terminal between the beams between satellites and the specified variable; the first constraint is that the transmission power of each satellite does not exceed the maximum transmission power; the second constraint is that the power allocated to each terminal is not less than zero.

2. The resource allocation method for a satellite communication system according to claim 1, wherein: The total capacity of the satellite communication system before the update is determined based on the beam allocation factor of the satellite's beam allocated to the wave position, the subchannel allocation factor of the satellite's beam allocated to the terminal, the bandwidth of a single subchannel, and the signal-to-interference-and-noise ratio of the terminal.

3. The resource allocation method for a satellite communication system according to claim 1 or 2, characterized in that: The constraints of the resource allocation model include: The first constraint is that the transmission power of each satellite does not exceed the maximum transmission power; the second constraint is that the power allocated to each terminal is not less than zero; the third constraint is that the transmission rate on each subchannel is not less than the minimum transmission rate; the fourth constraint is that the beam allocation factor of the satellite's beam allocated to the wave position is used to indicate unallocated or allocated; the fifth constraint is that the subchannel allocation factor of the satellite's beam allocated to the terminal is used to indicate unallocated or allocated; the sixth constraint is that only terminals within the wave position illuminated by the beam can be scheduled; the seventh constraint is that each beam of each satellite can be allocated to at most one wave position; the eighth constraint is that each wave position can be allocated to at most one beam on each satellite; the ninth constraint is that each terminal can be allocated at most N sub-channels; the tenth constraint: each sub-channel can be superimposed on N max terminals; N and N max All are positive integers.

4. The resource allocation method for a satellite communication system according to claim 1 or 2, characterized in that: Converting the resource allocation model into a beam sub-channel allocation problem, and modeling the beam sub-channel allocation problem as a Markov decision process, includes: Setting possible values ​​of the beam allocation factor and the subchannel allocation factor that satisfy the constraints in the resource allocation model into a state space; Setting possible changes of the beam allocation factor and the subchannel allocation factor that satisfy the constraints in the resource allocation model as an action space; According to the terminal power allocation result and the given beam subchannel allocation result, a reward function for selecting an action to be executed is set at the state of the current time slot.

5. The resource allocation method for a satellite communication system according to claim 4, characterized in that: The step of setting a reward function for selecting an action to be executed based on the state of the current time slot according to the terminal power allocation result and the given beam subchannel allocation result includes: Calculating the total capacity of the satellite communication system in the current time slot according to the terminal power allocation result and the given beam subchannel allocation result; Based on the total capacity of the satellite communication system, a reward function for selecting an action to be executed in the state of the current time slot is set; the reward function is related to the total capacity of the satellite communication system in the current time slot.

6. The resource allocation method for a satellite communication system according to claim 1, wherein: The performing of the Markov decision process to obtain a beam sub-channel allocation result includes: Cycle through time steps and update the strategy. For each time step, the agent uses the current values ​​of the beam allocation factor and the subchannel allocation factor as the state, selects an action (i.e., a beam subchannel allocation result) according to the strategy, calculates a reward value according to the reward function, executes the action, and transitions to the next state. The state transition trajectory is recorded in the experience pool. One time step corresponds to one time slot. Copy the parameters of the policy network to the old policy network to maintain the old policy, update the parameters of the old policy network in the proximal policy optimization algorithm, update the parameters of the policy network and the value network, and obtain the updated policy; Empty the experience pool; Until the action that maximizes the cumulative reward is found, that is, a set of beam allocation factors and sub-channel allocation factor values, then the values ​​of the beam allocation factors and sub-channel allocation factors, that is, the beam-sub-channel allocation result, are output.

7. The resource allocation method for a satellite communication system according to claim 6, characterized in that: The parameters of the policy network are copied to the old policy network to maintain the old policy, update the parameters of the old policy network in the proximal policy optimization algorithm, update the parameters of the policy network and the parameters of the value network, and obtain the updated policy, including: Fitting the policy of the agent using a policy network, copying the parameters of the policy network to the old policy network to maintain the old policy, and updating the parameters of the policy network training process each time the agent collects the state transition trajectory of the specified number of time steps; The state value is estimated using the value network, and the training goal is to update the parameters of the value network based on minimizing the mean square error loss function; The updated strategy is obtained according to the updated parameters of the strategy network and the updated parameters of the value network.

8. The resource allocation method for a satellite communication system according to claim 1, wherein: The obtaining, according to the terminal power allocation result and the beam sub-channel allocation result, an optimal terminal power allocation result and a beam sub-channel allocation result, includes: Obtaining the total capacity of the satellite communication system according to the terminal power allocation result and the beam sub-channel allocation result; According to the total capacity of the satellite communication system, the optimal terminal power allocation result and beam sub-channel allocation result are obtained.

9. The resource allocation method for a satellite communication system according to claim 8, characterized in that: Obtaining the total capacity of the satellite communication system according to the terminal power allocation result and the beam sub-channel allocation result includes: For each of the terminal power allocation results and each of the beam sub-channel allocation results, calculating the total system capacity based on the terminal power allocation result and the beam sub-channel allocation result; Substituting the beam subchannel allocation result into the geometric programming problem to obtain an updated terminal power allocation result; and recalculating the total system capacity based on the beam subchannel allocation result and the updated terminal power allocation result.

10. The resource allocation method for a satellite communication system according to claim 8 or 9, characterized in that: Obtaining an optimal terminal power allocation result and a beam sub-channel allocation result based on the total capacity of the satellite communication system includes: If the difference between the calculated total capacity of the satellite communication system and the recalculated total capacity of the satellite communication system is not equal, substituting the updated terminal power allocation result into a Markov decision process, executing the Markov decision process, and obtaining an updated beam subchannel allocation result; The updated beam subchannel allocation result replaces the beam subchannel allocation result, and the step of obtaining the total capacity of the satellite communication system based on the terminal power allocation result and the beam subchannel allocation result is executed cyclically until the difference between the most recently calculated total capacity of the satellite communication system and the last calculated total capacity of the satellite communication system is equal. At this time, the optimal terminal power allocation result and the optimal beam subchannel allocation result are output.

11. The resource allocation method of a satellite communication system according to claim 8, characterized in that: Obtaining an optimal terminal power allocation result and a beam sub-channel allocation result based on the total capacity of the satellite communication system includes: If the difference between the calculated total capacity of the satellite communication system and the recalculated total capacity of the satellite communication system is equal, the beam sub-channel allocation result and the updated terminal power allocation result are output as the optimal allocation result.

12. A resource allocation system for a satellite communication system, characterized in that: include: A model building module is used to establish a resource allocation model, wherein the optimization goal of the resource allocation model is to maximize the total capacity of the satellite communication system, and the optimization variables are the beam allocation factor of the satellite beam allocated to the wave position, the subchannel allocation factor of the satellite beam allocated to the terminal, and the transmission power of the satellite to the terminal; A resource allocation result acquisition module, configured to allocate terminal power and beam subchannels to each terminal based on the resource allocation model; the beam subchannels are the beam allocation factor and the subchannel allocation factor; The allocating terminal power and beam sub-channel to each terminal based on the resource allocation model includes: Converting the resource allocation model into a power allocation problem, converting the power allocation problem into a geometric programming problem, substituting the known beam subchannel allocation result into the geometric programming problem, and obtaining a terminal power allocation result; Converting the resource allocation model into a beam subchannel allocation problem, modeling the beam subchannel allocation problem as a Markov decision process, executing the Markov decision process, and obtaining a beam subchannel allocation result; Obtaining an optimal terminal power allocation result and beam subchannel allocation result according to the terminal power allocation result and the beam subchannel allocation result; Converting the resource allocation model into a power allocation problem includes: Assuming that the beam subchannel allocation is known, the expression for the total capacity of the satellite communication system is updated, that is, the power allocation problem is obtained; the updated total capacity of the satellite communication system is determined based on the beam allocation factor of the satellite beam allocated to the wave position, the subchannel allocation factor of the satellite beam subchannel allocated to the terminal, the bandwidth of a single subchannel, the signal-to-interference-and-noise ratio of the terminal, the interference of the terminal within the beam, the interference of the terminal between the intra-satellite beams, and the interference of the terminal between the inter-satellite beams; Converting the power allocation problem into a geometric programming problem includes: The transmission rate and the minimum transmission rate in the third constraint of the resource allocation model are converted into a new third constraint, where the new third constraint is related to interference of a terminal within a beam, interference of a terminal between beams within a satellite, interference of a terminal between beams between satellites, and bandwidth of a single subchannel; The optimization objective of the resource allocation model is replaced by minimizing the specified variable, and the optimization variable is replaced by the transmission power transmitted from the satellite to the terminal; the constraints are replaced by the first constraint, the second constraint, the new third constraint and the eleventh constraint, that is, the power allocation problem is converted into a geometric programming problem; the eleventh constraint is related to the interference of the terminal within the beam, the interference of the terminal between the beams within the satellite, the interference of the terminal between the beams between satellites and the specified variable; the first constraint is that the transmission power of each satellite does not exceed the maximum transmission power; the second constraint is that the power allocated to each terminal is not less than zero.

13. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the resource allocation method for a satellite communication system according to any one of claims 1 to 11 is implemented.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a computer program or instructions. When the computer program or instructions are executed on a computer, the computer is caused to execute the resource allocation method for a satellite communication system according to any one of claims 1 to 11.

Images