Full-flexible satellite frequency plan and beam power allocation design method
By applying reinforcement learning algorithms and neural networks on fully flexible satellites to design frequency planning and beam power distribution, the complexity of satellite resource allocation and NP difficulties are solved, and efficient beam capacity matching rate and user service capabilities are achieved.
Patent Information
- Application Number
- CN202411831593.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-12
- Publication Date
- 2025-04-08
AI Technical Summary
The expansion of fully flexible satellite resource dimensions and the increase in the scale of adjustable parameters have significantly improved the complexity of resource allocation problems. The traditional manual resource management and control method is not applicable. At the same time, the joint optimization of beam bandwidth and power is a problem of NP, which leads to the algorithm running time too long and it is easy to converge to the suboptimal solution.
A fully flexible satellite frequency planning and beam power distribution design method is designed, and a reinforcement learning algorithm framework is used to train the output frequency planning arrangement and beam power distribution results through neural networks, and secondary optimization is carried out to improve the power distribution effect.
On the premise of minimizing synchronous interference, the average capacity matching rate of all beams is maximized, and the ability of satellite service users is improved, solving the shortcomings of traditional methods in complex resource allocation and NP difficulties.
Smart Images
Figure CN119788154A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of satellite resource management and control, and in particular relates to a fully flexible satellite frequency plan and beam power allocation design method. Background Art
[0002] With the continuous development of high-performance phased array antennas, digital beamforming technology, digital transparent processors, and multi-port amplifier technology, future communication satellites will be a new type of fully flexible satellite. Flexibility means that with the support of flexible payloads, the satellite's coverage area, beam power, frequency plan, and hinge relationship between beams can all be reconstructed on-orbit on demand, which will significantly improve the service capabilities for users. At the same time, it also puts forward more stringent requirements on the satellite's resource management algorithm. Different from the fixed resource management strategy of traditional high-throughput satellites, the resource management strategy of fully flexible satellites requires the ability to accept user needs in real time and generate on-board payload resource configuration plans. When the coverage area is given, the satellite's frequency plan and beam power basically determine the capacity of the beam. Therefore, the joint optimization design of beam bandwidth and power is the core of the resource management strategy. To achieve this goal, there are two main problems that need to be solved. First, the expansion of the resource dimension of fully flexible satellites and the increase in the scale of adjustable parameters have significantly increased the complexity of resource allocation problems for such systems. The total number of resource combinations can reach billions, making traditional manual resource management methods no longer applicable. Second, from a theoretical point of view, the joint optimization problem of beam bandwidth and power is an NP-hard problem that cannot be modeled and solved using mathematical optimization theory. Methods based on iterative optimization will lead to excessive complexity in the solution, long algorithm running time, and easy convergence to a suboptimal solution. Therefore, it is necessary to design an effective algorithm to control the beam bandwidth and power of fully flexible satellites. Summary of the invention
[0003] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art and provide a fully flexible satellite frequency plan and beam power allocation design method, which maximizes the average capacity matching rate of all beams while minimizing co-channel interference, and further improves the ability of satellite to serve users.
[0004] The purpose of the present invention is achieved through the following technical solutions: a fully flexible satellite frequency plan and beam power allocation design method, comprising: obtaining beam requirements; establishing a system model according to the beam requirements; calculating the signal-to-interference-plus-noise ratio during each beam communication according to the system model, and obtaining an expression of the communication capacity according to the signal-to-interference-plus-noise ratio during each beam communication; establishing an optimization problem according to the system model and the expression of the communication capacity; building a reinforcement learning algorithm framework according to the optimization problem; according to the reinforcement learning algorithm framework, inputting the beam requirements into a neural network corresponding to the beam for training, and outputting frequency plan arrangement and beam power allocation results; performing secondary optimization on the frequency plan arrangement and the beam power allocation results to obtain a secondary optimized power allocation result.
[0005] In the above-mentioned fully flexible satellite frequency plan and beam power allocation design method, the beam requirements include beam capacity requirements, beam number and beam pointing.
[0006] In the above-mentioned fully flexible satellite frequency plan and beam power allocation design method, the system model is a two-dimensional network, wherein the total number of rows of the two-dimensional network is 2*N r , N r It is the number of times the whole frequency band is reused.
[0007] In the above-mentioned fully flexible satellite frequency plan and beam power allocation design method, the signal-to-interference-to-noise ratio during each beam communication is obtained by the following formula:
[0008]
[0009] Among them, γ b,n (t) is the signal-to-interference-noise ratio for each beam communication, P b (t) is the output power of the traveling wave tube, α j is the interference degree factor, N0 is the white noise power spectrum density, B b is the bandwidth of the bth beam, t is the time, h b,n (t) is the channel gain from the b-th beam to the n-th user, b is the beam number, n is the user number, A(j) is the antenna gain of the j-th beam to the b-th beam, and j is the number of the beam that interferes with the b-th beam.
[0010] In the above-mentioned fully flexible satellite frequency plan and beam power allocation design method, the expression of communication capacity is:
[0011] R b (t) = B b log22(1+γ b,n (t));
[0012] Among them, R b (t) is the downlink communication capacity that the bth beam can provide at time t, Bb is the bandwidth of the bth beam, t is the time, γ b,n (t) is the signal-to-interference-noise ratio for each beam communication, b is the beam number, and n is the user number.
[0013] In the above-mentioned fully flexible satellite frequency plan and beam power allocation design method, the optimization problem is obtained by the following formula:
[0014]
[0015] Among them, P1 is the optimization problem established, F is the set of multiplexed frequency bands that can be selected by the beam, K is the set of starting frequencies that can be selected by the beam, O is the set of available bandwidths of the beam, P is the set of available powers of the beam, T is the length of the time period considered, B is the total number of beams, C is the number of available powers of the beam, and b (t) is the beam capacity requirement, δ is the weighting factor, Γ(t) is the number of co-channel interference bands, P tot is the total power available to the satellite, f b (t) is the reuse frequency band of beam b, k b (t) is the starting frequency of beam b, o b (t) is the specific choice of the occupied bandwidth of beam b, N r is the number of times the full frequency band is reused, N B is the maximum available bandwidth, P b (t) is the transmission power of beam b, R b (t) is the downlink communication capacity that the b-th beam can provide at time t, and b is the sequence number of the beam.
[0016] In the above-mentioned fully flexible satellite frequency plan and beam power allocation design method, the reinforcement learning algorithm framework includes state space, action space and reward function; wherein; the state space is the beam capacity requirement, the number of beams and the beam pointing; the reward function is a weighted function of the capacity matching rate and the co-channel interference; the action space is a set of selectable multiplexing frequency bands, starting frequencies, available bandwidths and transmission powers of the beam.
[0017] In the above-mentioned fully flexible satellite frequency plan and beam power allocation design method, the frequency plan arrangement and beam power allocation results are subjected to secondary optimization to obtain the secondary optimized power allocation results, including: extracting the beams that do not meet the capacity requirements in the frequency plan arrangement and beam power allocation results, using the frequency plan arrangement and beam power allocation results as initial values, and using a continuous convex approximation algorithm to obtain the secondary optimized power allocation results.
[0018] A fully flexible satellite frequency plan and beam power allocation design system comprises: a first module for obtaining beam requirements; a second module for establishing a system model according to the beam requirements; a third module for calculating the signal-to-interference-noise ratio during each beam communication according to the system model, and obtaining an expression of the communication capacity according to the signal-to-interference-noise ratio during each beam communication; a fourth module for establishing an optimization problem according to the system model and the expression of the communication capacity; a fifth module for building a reinforcement learning algorithm framework according to the optimization problem; a sixth module for inputting the beam requirements into a neural network corresponding to the beam for training according to the reinforcement learning algorithm framework, and outputting frequency plan arrangement and beam power allocation results; and a seventh module for performing secondary optimization on the frequency plan arrangement and the beam power allocation results to obtain a secondary optimized power allocation result.
[0019] An electronic device comprises: a memory for storing computer-readable instructions; and a processor for running the computer-readable instructions to execute a fully flexible satellite frequency plan and beam power allocation design method.
[0020] Compared with the prior art, the present invention has the following beneficial effects:
[0021] The goal of the present invention is to maximize the average capacity matching rate of all beams while minimizing co-channel interference, further enhance the satellite's ability to serve users, and manage the beam bandwidth and power of fully flexible satellites by designing effective algorithms. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0023] Figure 1 is a schematic diagram of a basic architecture of reinforcement learning provided by an embodiment of the present invention;
[0024] Figure 2 is a schematic diagram of the results obtained by multi-agent reinforcement learning provided by an embodiment of the present invention;
[0025] Figure 3 is a flow chart of a fully flexible satellite frequency plan and beam power allocation design method provided by an embodiment of the present invention;
[0026] Figure 4 Schematic diagram of the capacity matching rate of the proposed algorithm provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0027] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to be able to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0028] This embodiment provides a fully flexible satellite frequency plan and beam power allocation design method, the method comprising:
[0029] Obtain beam requirements;
[0030] Build system model based on beam requirements;
[0031] According to the system model, the signal-to-interference-to-noise ratio of each beam during communication is calculated, and the expression of the communication capacity is obtained according to the signal-to-interference-to-noise ratio of each beam during communication;
[0032] According to the system model and the expression of communication capacity, the optimization problem is established;
[0033] Build a reinforcement learning algorithm framework based on the optimization problem;
[0034] According to the reinforcement learning algorithm framework, the beam requirements are input into the neural network corresponding to the beam for training, and the frequency plan arrangement and beam power allocation results are output;
[0035] The frequency plan arrangement and the beam power allocation results are optimized twice to obtain the second-optimized power allocation result.
[0036] Beam requirements include beam capacity requirements, number of beams, and beam pointing.
[0037] The system model is a two-dimensional network, where the total number of rows in the two-dimensional network is 2*N r , N r It is the number of times the whole frequency band is reused.
[0038] The signal-to-interference-to-noise ratio of each beam communication is obtained by the following formula:
[0039]
[0040] Among them, γ b,n (t) is the signal-to-interference-noise ratio for each beam communication, P b (t) is the output power of the traveling wave tube, α j is the interference degree factor, N0 is the white noise power spectrum density, B bis the bandwidth of the bth beam, t is the time, h b,n (t) is the channel gain from the b-th beam to the n-th user, b is the beam number, n is the user number, A(j) is the antenna gain of the j-th beam to the b-th beam, and j is the number of the beam that interferes with the b-th beam.
[0041] The expression of communication capacity is:
[0042] R b (t) = B b log22(1+γ b,n (t));
[0043] Among them, R b (t) is the downlink communication capacity that the bth beam can provide at time t, B b is the bandwidth of the bth beam, t is the time, γ b,n (t) is the signal-to-interference-noise ratio for each beam communication, b is the beam number, and n is the user number.
[0044] The optimization problem is obtained by the following formula:
[0045]
[0046] Among them, P1 is the optimization problem established, F is the set of multiplexed frequency bands that can be selected by the beam, K is the set of starting frequencies that can be selected by the beam, O is the set of available bandwidths of the beam, P is the set of available powers of the beam, T is the length of the time period considered, B is the total number of beams, C is the number of available powers of the beam, and b (t) is the beam capacity requirement, δ is the weighting factor, Γ(t) is the number of co-channel interference bands, P tot is the total power available to the satellite, f b (t) is the reuse frequency band of beam b, k b (t) is the starting frequency of beam b, o b (t) is the specific choice of the occupied bandwidth of beam b, N r is the number of times the full frequency band is reused, N B is the maximum available bandwidth, P b (t) is the transmission power of beam b, R b (t) is the downlink communication capacity that the b-th beam can provide at time t, and b is the sequence number of the beam.
[0047] The reinforcement learning algorithm framework includes state space, action space and reward function; wherein; the state space is the beam capacity requirement, the number of beams and the beam pointing; the reward function is a weighted function of the capacity matching rate and the co-channel interference; the action space is a set of beam selectable reuse frequency bands, starting frequencies, available bandwidths and transmission powers.
[0048] The frequency plan arrangement and beam power allocation results are subjected to secondary optimization to obtain the secondary optimized power allocation results, including: extracting the beams that do not meet the capacity requirements in the frequency plan arrangement and beam power allocation results, using the frequency plan arrangement and beam power allocation results as initial values, and adopting a continuous convex approximation algorithm to obtain the secondary optimized power allocation results.
[0049] This embodiment is a joint optimization of the frequency plan arrangement and beam power of a fully flexible satellite. It mainly includes seven steps: demand acquisition, system model establishment, communication capacity calculation, optimization problem establishment, reinforcement learning algorithm construction, neural network training, and secondary optimization of power allocation scheme. Finally, the frequency plan arrangement and power allocation scheme that can meet the capacity requirements of each beam are output. The main steps are introduced as follows:
[0050] Step 1: Obtain requirements. According to the input from the upper level, obtain the capacity requirements of each beam, the outline of the coverage area, the number of beams used, the beam center width and beam center coordinates, available frequency bands, EIRP values and other parameters.
[0051] Step 2: Establish a system model. According to the requirements, the beam set is defined as B = {1, 2, ..., B}. In order to quantitatively describe the user's traffic requirements, a time-sharing discrete time system is considered, denoted as T = {1, 2, ..., T}. A two-dimensional network is used to describe the frequency plan arrangement and beam power allocation. In addition, let N tot is the total bandwidth (the current user downlink bandwidth is generally 17.7GHz-20.2GHz), N b is the number of blocks that the total bandwidth can be divided into. In practical applications, fully flexible satellites can also use left-hand polarization and right-hand polarization to further improve spectrum efficiency. Therefore, the total number of rows in the entire grid is 2*N. r , the total number of columns is N b It should be noted that two beams that use the same polarization mode and have overlapping spectrum resources will interfere with each other. Therefore, the frequency plan arrangement strategy needs to be optimized according to the actual goal. After the frequency plan is arranged, power allocation is to assign a specific power value to the frequency band occupied by each beam, that is, the overall resource allocation strategy can be fully represented in a two-dimensional grid.
[0052] Step 3: Calculate the communication capacity. According to the established system model, calculate the signal-to-interference-to-noise ratio of each beam communication. Assume that the satellite is fixed at In ECEF coordinates, the three-dimensional coordinates (X, Y, Z) of the satellite can be calculated based on the radius of the earth and the altitude of the satellite orbit. Assume that the longitude and latitude of the user at each time t can be expressed as The three-dimensional coordinates (xb,n (t),y b,n (t),z b,n (t)). From this, the distance d between the user and the satellite at each time t can be calculated b,n (t). Therefore, the free space loss between the user and the satellite can be expressed as:
[0053]
[0054] Where f represents the carrier frequency. Then the channel gain from the bth beam to the nth user can be expressed as:
[0055] h b,n (t) = G b,t -L b,t -L b,n (t)+G b,r -L b,o ;
[0056] Among them, G b,t is the transmit antenna gain, L b,t is the transmission loss of the transmitting antenna, G b,r is the receiving antenna gain, L b,o is other losses including atmospheric attenuation and rain attenuation. Therefore, the signal-to-interference-noise ratio of each user can be expressed as:
[0057]
[0058] Where P b represents the output power of the traveling wave tube, α j represents the interference factor used to measure the overlap between two beams, N0 represents the white noise power spectrum density, B b represents the bandwidth of the bth beam. Then the downlink communication capacity that the bth beam can provide at time t can be expressed as:
[0059] R b (t) = B b log22(1+γ b,n (t));
[0060] Step 4: Establish the optimization problem. Based on the system model and the expression of the communication capacity obtained, the problem to be studied is modeled as an optimization problem. The main goal is to reduce the inter-beam interference as much as possible while ensuring that the downlink capacity of the satellite can meet the needs of users. First, the main objective function to be considered is clearly defined - the capacity matching rate, which is defined as follows:
[0061]
[0062] where c b(t) is the capacity requirement of the bth beam. The reason for considering the absolute value is that we are considering capacity matching, that is, if the actual capacity of the beam is much higher than the user demand, it is also considered a bad decision because this allocation method wastes resources and increases interference to other beams. From an optimization perspective, the problem under study can be modeled as P1. The second term in the objective function is the inter-beam interference term, which is defined as the total number of overlapping resource blocks between beams, and δ is the weight coefficient. The first two constraints give constraints on the total transmission power. The last three constraints give integer variable constraints on the frequency plan arrangement, which represent the reused frequency band, starting frequency and occupied frequency bandwidth selected by any beam. These three integer variables can fully describe the position of a beam in the entire grid. The meaning of the last constraint is that the allocation of beams cannot exceed the available frequency band.
[0063]
[0064] P1 contains integer variables and continuous variables, and the optimization objective function is non-convex. Therefore, this problem cannot be solved by traditional convex optimization methods. Therefore, we consider using reinforcement learning to solve it.
[0065] Step 5: Build a reinforcement learning algorithm framework. The basic principle of using reinforcement learning to solve this optimization problem is as follows: Figure 1 As shown. The basic idea of reinforcement learning is exploration and utilization. By modeling each beam as an intelligent agent, the intelligent agent continuously learns in the interaction with the environment, and increases the probability of making actions that meet the requirements under the guidance of the reward function. In the problem studied, each beam must make three discrete actions and one continuous action. The quality of the solution can only be evaluated after all beams have completed the action. Therefore, we use cooperative multi-agent reinforcement learning. All beams share the state space and reward function, and use the same reward to update the neural network parameters of all beams. The state space refers to the set of all influencing factors when the agent makes a decision. In the problem studied, the state space can be modeled as a set of beam number, beam pointing, beam capacity requirements, and current resource usage. The reward function is defined as a weighted function of capacity matching rate and co-channel interference.
[0066] Step 6: Training of the neural network. According to the reinforcement algorithm framework that has been built, the proximal strategy optimization method is used to train the neural network. This method can ensure the stability of the training process without problems such as overfitting and gradient disappearance. In the initial stage, each agent interacts with the environment, generates samples for training and stores them in the experience pool, and then updates the current state. When the experience pool is full, a round of training begins until 10,000 rounds of training. When the network converges, it has the ability to output frequency plan arrangement and power allocation schemes in real time according to different beam capacity requirements, that is, given the input number of beams, beam center pointing and beam capacity requirements, it can output a frequency plan arrangement and beam power allocation scheme. Figure 2 The output results after the network converges are given.
[0067] Step 7: Secondary optimization of power allocation scheme: Since neural networks have inherent randomness, from a statistical point of view, it is impossible to guarantee the output of the optimal solution in each state. Therefore, it is necessary to further optimize the frequency plan arrangement and beam power allocation scheme output by the reinforcement learning algorithm. Specifically, first extract the beams that do not meet the capacity requirements under the current scheme, then convert the non-convex objective function into the form of two concave functions subtracted, and use the first-order Taylor expansion method to convert the solution of the original problem into the solution of a series of convex optimization problems. Use the output of the reinforcement learning algorithm as the initial value, and iterate continuously until the difference between the optimization results of two adjacent times is less than a threshold, and finally output a secondary optimized power allocation scheme. Figure 4 The capacity matching ratio results after further power optimization are given, and the results show that the proposed method can meet the beam capacity requirements. Figure 3 The complete flowchart of the proposed method is given.
[0068] The center pointing of the beam and the capacity requirement provided by the proposed algorithm are shown in Table 1.
[0069] Table 1 Center, longitude and latitude of all beams and flow demand information
[0070]
[0071]
[0072] Satellite antenna EIRP and G / T are 61dBW and 15dB / K respectively; terminal antenna EIRP and G / T are 51.6dBW and 25.2dB / K respectively; user downlink available starting frequency: 17700MHz; user downlink available ending frequency: 20200MHz; minimum interval of co-frequency beams: 1000km; total power envelope of all beams: 1000W; maximum multiplexing times of the whole band: 4; minimum bandwidth unit: 250MHz; total number of training times: 10000; neural network architecture: three hidden layers, 128*128 neurons in each layer; learning rate: 3*1e -4 ; Long-term return decay rate: 0.1; Threshold for exiting the second stage algorithm optimization: 0.001.
[0073] With the number of beams, the beam center pointing beam capacity requirements as input, this embodiment can be programmed and implemented in the order of step 1 to step 7, and run by a computer. The running result is within 1 minute, and the algorithm performance can meet the actual needs of users.
[0074] This embodiment also provides a fully flexible satellite frequency plan and beam power allocation design system, which includes: a first module for obtaining beam requirements; a second module for establishing a system model according to the beam requirements; a third module for calculating the signal-to-interference-noise ratio during each beam communication according to the system model, and obtaining an expression of the communication capacity according to the signal-to-interference-noise ratio during each beam communication; a fourth module for establishing an optimization problem according to the system model and the expression of the communication capacity; a fifth module for building a reinforcement learning algorithm framework according to the optimization problem; a sixth module for inputting the beam requirements into the neural network corresponding to the beam for training according to the reinforcement learning algorithm framework, and outputting the frequency plan arrangement and beam power allocation results; a seventh module for performing secondary optimization on the frequency plan arrangement and the beam power allocation results to obtain a secondary optimized power allocation result.
[0075] This embodiment also provides an electronic device, including: a memory: for storing computer-readable instructions; and a processor: for running the computer-readable instructions to execute a fully flexible satellite frequency plan and beam power allocation design method.
[0076] This embodiment adopts a multi-agent based reinforcement learning method. By modeling the frequency plan arrangement and beam power allocation as three discrete variables and one continuous variable respectively, the proximal strategy optimization method is used to jointly optimize the two. After the network converges, the speed of making decisions is significantly increased compared with the existing algorithm.
[0077] This embodiment further proposes a power optimization method based on continuous convex approximation to overcome the inherent randomness of neural networks. The output results of multi-agent reinforcement learning are used as the initial solution. The Taylor expansion method is used to transform the non-convex objective function into a convex function. The capacity matching rate of the proposed method is further improved by solving a series of convex optimization problems. The operation results show that the capacity matching rate of the proposed method is significantly improved compared with the traditional method.
[0078] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.
Claims
1. A fully flexible satellite frequency plan and beam power allocation design method, characterized in that include: Obtain beam requirements; Build system model based on beam requirements; According to the system model, the signal-to-interference-to-noise ratio of each beam during communication is calculated, and the expression of the communication capacity is obtained according to the signal-to-interference-to-noise ratio of each beam during communication; According to the system model and the expression of communication capacity, the optimization problem is established; Build a reinforcement learning algorithm framework based on the optimization problem; According to the reinforcement learning algorithm framework, the beam requirements are input into the neural network corresponding to the beam for training, and the frequency plan arrangement and beam power allocation results are output; The frequency plan arrangement and the beam power allocation results are optimized twice to obtain the second-optimized power allocation result.
2. The method for designing a fully flexible satellite frequency plan and beam power allocation according to claim 1, characterized in that: The beam requirements include beam capacity requirements, beam number and beam pointing.
3. The method for designing a fully flexible satellite frequency plan and beam power allocation according to claim 1, characterized in that: The system model is a two-dimensional network, where the total number of rows in the two-dimensional network is 2*N r , N r It is the number of times the whole frequency band is reused.
4. The method for designing a fully flexible satellite frequency plan and beam power allocation according to claim 1, characterized in that: The signal-to-interference-to-noise ratio of each beam communication is obtained by the following formula: Among them, γ b,n (t) is the signal-to-interference-noise ratio for each beam communication, P b (t) is the output power of the traveling wave tube, α j is the interference degree factor, N0 is the white noise power spectrum density, B b is the bandwidth of the bth beam, t is the time, h b,n (t) is the channel gain from the b-th beam to the n-th user, b is the beam number, n is the user number, A(j) is the antenna gain of the j-th beam to the b-th beam, and j is the number of the beam that interferes with the b-th beam.
5. The method for designing a fully flexible satellite frequency plan and beam power allocation according to claim 1, characterized in that: The expression of communication capacity is: R b (t)=B b log22(1+γ b,n (t)); Among them, R b (t) is the downlink communication capacity that the bth beam can provide at time t, B b is the bandwidth of the bth beam, t is the time, γ b,n (t) is the signal-to-interference-noise ratio for each beam communication, b is the beam number, and n is the user number.
6. The method for designing a fully flexible satellite frequency plan and beam power allocation according to claim 1, characterized in that: The optimization problem is obtained by the following formula: Among them, P1 is the optimization problem established, F is the set of multiplexed frequency bands that can be selected by the beam, K is the set of starting frequencies that can be selected by the beam, O is the set of available bandwidths of the beam, P is the set of available powers of the beam, T is the length of the time period considered, B is the total number of beams, C is the number of available powers of the beam, and b (t) is the beam capacity requirement, δ is the weighting factor, Γ(t) is the number of co-channel interference bands, P tot is the total power available to the satellite, f b (t) is the reuse frequency band of beam b, k b (t) is the starting frequency of beam b, o b (t) is the specific choice of the occupied bandwidth of beam b, N r is the number of times the full frequency band is reused, N B is the maximum available bandwidth, P b (t) is the transmission power of beam b, R b (t) is the downlink communication capacity that the b-th beam can provide at time t, and b is the sequence number of the beam.
7. The method for designing a fully flexible satellite frequency plan and beam power allocation according to claim 1, characterized in that: The reinforcement learning algorithm framework includes state space, action space and reward function; in; The state space is beam capacity requirement, number of beams and beam pointing; The reward function is a weighted function of capacity matching rate and co-channel interference; The action space is a set of selectable multiplexing frequency bands, starting frequencies, available bandwidths, and transmission powers for the beam.
8. The method for designing a fully flexible satellite frequency plan and beam power allocation according to claim 1, characterized in that: The frequency plan arrangement and beam power allocation results are optimized twice to obtain the power allocation results of the secondary optimization, including: The beams that do not meet the capacity requirements in the frequency plan arrangement and beam power allocation results are extracted, and the frequency plan arrangement and beam power allocation results are used as initial values. A continuous convex approximation algorithm is used to obtain the quadratic optimized power allocation result.
9. A fully flexible satellite frequency plan and beam power allocation design system, characterized in that include: The first module is used to obtain beam requirements; The second module is used to build a system model based on beam requirements; The third module is used to calculate the signal-to-interference-plus-noise ratio during each beam communication according to the system model, and obtain an expression of the communication capacity according to the signal-to-interference-plus-noise ratio during each beam communication; The fourth module is used to establish an optimization problem based on the system model and the expression of the communication capacity; The fifth module is used to build a reinforcement learning algorithm framework based on optimization problems; The sixth module is used to input the beam requirements into the neural network corresponding to the beam for training according to the reinforcement learning algorithm framework, and output the frequency plan arrangement and beam power allocation results; The seventh module is used to perform secondary optimization on the frequency plan arrangement and the beam power allocation result to obtain the secondary optimized power allocation result.
10. An electronic device, characterized in that: include: Memory: used to store computer-readable instructions; and Processor: used to run the computer-readable instructions to perform the method according to any one of claims 1 to 8.