Stackelberg game-based satellite-ground network anti-interference channel selection method

Abstract

The invention relates to the technical field of satellite communication, and particularly discloses a satellite-ground network anti-interference channel selection method based on the Steinberg game. S01, establishing a system scene to simulate communication between a user and a satellite, including interference suffered by the user when the user selects a channel to communicate with the satellite; s02, constructing a system model according to the system scene, obtaining the throughput of the user according to the transmitting antenna gain of the satellite, the receiving antenna gain of the user and the interference suffered by the user, and performing satisfaction evaluation on the throughput to obtain a satisfaction result; step S03, optimizing the channel selected by the user according to the satisfaction result, and constructing an optimization problem; and S04, constructing a Steinberg game model according to the optimization problem, and solving an equilibrium solution for the Steinberg game model by adopting a layered anti-interference learning algorithm to complete the anti-interference channel selection of the satellite-ground network based on the Steinberg game.

Description

Technical Field

[0001] The present invention relates to the field of satellite communication technology, and in particular to a satellite-to-ground network anti-interference channel selection method based on Steinberg game. Background Art

[0002] Low earth orbit (LEO) satellites have the advantages of low latency, high data transmission rate and good coverage. In recent years, they have attracted widespread attention from all walks of life. Their deployment speed and scale have been increasing, and they have become an important part of the next generation of mobile communications. However, there are two problems in the process of satellite-ground communication: on the one hand, with the development of ground information technology, the frequency band is crowded, and multiple users accessing one channel cause interference to each other. The conflict between the shortage of spectrum resources and the increase in the number of users is becoming increasingly obvious; on the other hand, due to the exposure of satellites and the open wireless channels, satellite communication signals are easily intercepted and interfered by the enemy, which brings huge challenges to the security protection of satellite communication systems. Therefore, studying the anti-interference channel selection technology of satellite-ground networks in complex electromagnetic environments can effectively avoid external malicious interference and improve spectrum utilization efficiency, which is of great significance for enhancing the information service capabilities of satellite-ground networks. Summary of the invention

[0003] In view of the above problems, the purpose of the present invention is to provide a satellite-to-ground network anti-interference channel selection method based on Steinberg game. In the current context of shortage of wireless open channels and spectrum resources, this method can effectively avoid external malicious interference and improve spectrum utilization efficiency, which is of great significance for enhancing the information service capabilities of the satellite-to-ground network.

[0004] The technical solution adopted by the present invention is: a satellite-to-ground network anti-interference channel selection method based on Steinberg game, comprising: Step S01: Establishing a system scenario to simulate the communication between a user and a satellite, including the interference to which the user is subjected when the user selects a channel to communicate with the satellite; Step S02: constructing a system model according to the system scenario, obtaining the user's throughput according to the satellite's transmitting antenna gain, the user's receiving antenna gain and the interference suffered by the user, and performing a satisfaction evaluation on the throughput to obtain a satisfaction result; Step S03: Optimizing the channel selected by the user according to the satisfaction result to construct an optimization problem; Step S04: constructing a Steinberg game model according to the optimization problem, using a layered anti-interference learning algorithm to obtain an equilibrium solution for the Steinberg game model, and completing the anti-interference channel selection of the satellite-to-ground network based on the Steinberg game.

[0005] Preferably, the interference suffered by the user in step S01 includes: co-frequency interference between different users in a beam, co-frequency interference between beams, and malicious interference.

[0006] Preferably, the throughput of the user is: ; ; in, is the channel bandwidth, is the signal power received by the user, is the co-channel interference from other users in the same beam of the LEO satellite in the downlink, is the co-channel interference from other beams of the LEO satellite in the downlink, For malicious interference from jammers, For Link The channel noise, is the transmit power of the antenna beam, is the maximum gain of the satellite transmitting antenna, is the maximum gain of the user receiving antenna, is the wavelength of communication transmission, For Link distance.

[0007] Preferably, the transmitting antenna gain of the satellite is: , , The receiving antenna gain of the user is: , ; in, is the satellite antenna off-axis angle, is the antenna half-power beamwidth, is the level at which the gain of the antenna main beam and the near-field sidelobe mask crosses below the peak gain, is the far-field sidelobe level, is the user antenna off-axis angle, is the maximum gain of the satellite transmitting antenna, is the maximum gain of the user receiving antenna, is the wavelength of communication transmission, is the circular equivalent diameter of the antenna.

[0008] Preferably, ; ; ; ; in, , is the transmission power of the satellite beam, is the transmit power of the jammer, , For users Satellite antenna interference gain in direction, is the user antenna receive gain in the direction of the jammer, , is the distance of the interfering link, is the maximum gain of the user receiving antenna, is the wavelength of communication transmission, , For users k , Select the channel. A function to determine whether the jammer and the user select the same channel.

[0009] Preferably, the method of performing satisfaction evaluation on the throughput to obtain a satisfaction result includes: calculating an average estimated score, using the average estimated score to characterize the subjective QoE feeling, and obtaining a satisfaction result; The calculation formula of the average estimated score MOS is: MOS = c 1 + exp [ d ( R − h ) ] ; The user's satisfaction is expressed as: ; in, is the user throughput, , and is a constant, For user satisfaction, For users The channel selection strategy, To remove the user The channel selection strategy set of other users, is the channel selection of the jammer, The throughput is The score value at that time.

[0010] Preferably, the Steinberg game model is expressed as: ; in, is the ground user set, For jammers, and are the policy sets for users and interference respectively, and are the effect functions of user and interference respectively.

[0011] Preferably, the Steinberg game model includes finding the best strategy , expressed as the maximum leader effect function: ; ; , ; in, For users Select the channel. Select a policy set for a user channel, is the channel selection strategy of the jammer, To meet the satisfaction threshold of basic communication, The throughput is Satisfaction at the time, To determine whether the jammer and the user select the same channel, To determine the attack channel Whether to corrupt the data transfer function.

[0012] Preferably, the Steinberg game model includes finding the best strategy , expressed as the maximum follower effect function: ; ; ; in, For users The channel selection strategy, To remove the user The channel selection strategy set of other users, is the channel selection of the jammer, For users satisfaction, For users Neighbors, For The channel selection sequence of adjacent users, For k The channel selection sequence of adjacent users, To remove a user Other users of For users and k The distance between The protection radius of the user.

[0013] Preferably, the equilibrium solution is: ; ; in, In order to maximize the channel allocation strategy for user benefits, In order to maximize the channel allocation strategy of the jammer, is the user's effect function, is the effect function of the jammer.

[0014] Beneficial effects of the above technical solution: The present invention solves the channel selection and anti-interference problem of multiple users in the satellite communication downlink, taking into account the users being simultaneously subjected to malicious interference and co-frequency interference. At the same time, in order to avoid the waste of resources caused by blindly increasing the throughput, a user experience quality (QoE) model based on the mean opinion score (MOS) value centered on user needs is cited to improve system performance. A Steinberg game model is established to describe the adversarial relationship between satellite users and jammers. At the same time, based on the characteristics of mutual interference between users, a local altruistic game model is established to describe the cooperative and competitive relationship between users. The proposed game model is proved to have an equilibrium solution. In order to obtain the equilibrium solution of the game model, an MSAP hierarchical anti-interference learning algorithm is proposed. The upper-layer interference uses a multi-armed tiger algorithm to update its own actions, and the lower-layer users use a spatial adaptive algorithm to obtain the optimal strategy. Finally, the simulation results verify the convergence and effectiveness of the proposed anti-interference scheme. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flow chart of the satellite-to-ground network anti-interference channel selection method based on Steinberg game provided by the present invention; Figure 2 A system scenario diagram provided for an embodiment of the present invention; Figure 3 A schematic diagram of strategy update iterations of a leader and a follower provided for one embodiment of the present invention; Figure 4 A schematic diagram of random distribution of users provided by an embodiment of the present invention; Figure 5 A curve diagram of network satisfaction rate variation under different numbers of channels provided by an embodiment of the present invention; Figure 6 A graph showing expected change in the regret value of a jammer under different numbers of channels provided by an embodiment of the present invention; Figure 7 A comparison chart of network satisfaction rates of different algorithms provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0016] The following is a further detailed description of the implementation methods of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0017] The terms "first", "second", etc. (if any) in the specification and claims are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0018] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, which can exist in three relationships. For example, A and / or B can be: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally means that the associated objects before and after are in an "or" relationship.

[0019] Embodiment 1

[0020] like Figure 1 The process of the satellite-to-ground network anti-interference channel selection method based on Steinberg game of the present invention aims to alleviate the problems of low utilization rate of existing resources and interference from multiple aspects. Simulation results show that the convergence and effectiveness of the method provided by the present invention can make the network have a better satisfaction rate.

[0021] 1. Establishment of system scenario The present invention considers the channel selection problem of multiple users under a single beam of a low-orbit multi-beam satellite in the downlink. Each user selects a channel in a time slot to communicate with the satellite. At the same time, there is a jammer J carried by a drone in the satellite beam, which is equipped with a wide-beam antenna and works in time division multiple access mode. In each time slot, the jammer selects a channel to transmit an interference attack signal. Therefore, the user will be interfered with in three aspects: co-frequency interference between different users in the beam, co-frequency interference between beams, and malicious interference, such as Figure 2 As shown in the figure, both users and jammers have the functions of environmental perception adaptation and strategy optimization and update. For the communication between satellite and users, in order to ensure reliable transmission, it is necessary to optimize channel selection to minimize malicious interference and co-frequency interference; for the jammer, it is necessary to dynamically adjust the attacked channel to maximize the interference and destruction effect.

[0022] The set of satellite beams is ; The collection of users is ; The available channel sets are ; in, , the jammer and the user have the same selectable channel set, For the beams, For the Users, for arrive The downlink for right The co-channel interference link, For Link and The angle of For Link With jammer Angle.

[0023] 2. Establishment of system model 1) Propagation model: In the present invention, only free space loss is considered, and its expression is: (1) in, is the wavelength of communication transmission, The distance for communication transmission.

[0024] For LEO satellites, according to ITU-R S.1528, the transmit antenna gain of LEO satellites is: (2) (3) in, is the maximum gain of the satellite antenna, is the satellite antenna off-axis angle, is the antenna half-power beamwidth, is the level at which the gain of the antenna main beam and the near-field sidelobe mask crosses below the peak gain, is the far-field sidelobe level, preferably, , .

[0025] For users, according to ITU-R S.465, the receiving antenna gain of terrestrial users is: (4) in, is the maximum gain of the user antenna, is the off-axis angle of the user antenna.

[0026] (5) in, is the circular equivalent diameter of the antenna, The wavelength used for communication transmission.

[0027] 2) Transmission rate model: such as Figure 2 , assuming the user Access Channel , jammer Access Channel , then there are three potential interferences: i) : is the co-channel interference from other users in the same beam of the LEO satellite in the downlink; (6) ii) : is the co-channel interference from other beams of the LEO satellite in the downlink; (7) iii) : Malicious interference from jammers; (8) in, , is the transmission power of the satellite beam, is the transmit power of the jammer, , For users Satellite antenna interference gain in direction, is the user antenna receive gain in the direction of the jammer, , is the distance of the interfering link.

[0028] is a binary indicator function, which is used to determine whether the two select the same channel at the current moment. Its expression is: (9) Based on the above propagation model, users are obtained The transmission rate or throughput is: (10) (11) in, is the channel bandwidth, For Link The channel noise, is the signal power received by the user, is the transmit power of the antenna beam, For Link distance.

[0029] 3) User satisfaction evaluation indicators In many current studies on wireless network channel resource allocation, the main goal is usually to find a channel allocation strategy that can maximize the throughput of the entire network. However, this optimization cannot directly and accurately reflect the actual satisfaction of users with their services. From the user's perspective, if the increase in throughput is not enough to significantly improve the user experience, then this increase is actually meaningless. Therefore, if the actual needs of users can be taken into account in the resource allocation process, resource utilization can be improved more effectively. The present invention uses an optimization approach oriented to user needs, converting user utility from quality of service (QoS) to subjective QoE level, that is, average estimated score MOS. This design will effectively alleviate user competition and optimize resource utilization, that is, when a certain user's satisfaction level no longer increases, resources will be allocated to other users.

[0030] The average evaluation score MOS is used to characterize the user's QoE. The present invention adopts the user satisfaction evaluation method proposed by the International Telecommunication Union (ITU) to divide the subjective QoE experience into six levels: "poor", "poor", "average", "better", "good" and "very good". An ordinal scale method is used from low to high to represent the user's subjective feelings. The larger the MOS value, the higher the user satisfaction. On the contrary, the smaller the MOS value, the lower the user satisfaction. It can be seen that this is a process in which quantitative change leads to qualitative change. What users intuitively feel is the qualitative change. When the quantitative change is not enough to cause qualitative change, this improvement does not make much sense to the user. In addition, when the user's throughput can already fully meet their needs, pursuing higher throughput will only bring waste.

[0031] The present invention defines the MOS function as: MOS = c 1 + exp [ d ( R − h ) ] (12) in, is the user throughput, , and is a constant. The specific value range and meaning of MOS are shown in Table 1.

[0032] Table 1 Specific range and meaning of MOS MOS value QoE Satisfaction Score S(R) NOT<2.6 Difference Dissatisfied 1 2.6≤MOS<3.1 Poor Less satisfied 2 3.1≤MOS<3.6 generally generally 3 3.6≤MOS<4 better Fairly satisfied 4 4≤MOS<4.3 good satisfy 5 MOS ≥ 4.3 very good Very satisfied 6 3. Construction of optimization problem Using functions Indicates user satisfaction, (13) in, Represents the transmission rate The score value at , is given according to Table 1 and formula (12). The total satisfaction of all users can be expressed as: (14) Based on the above analysis, the user's goal is to dynamically and adaptively adjust the channel selection by perceiving the environment to maximize the satisfaction of all users, that is: (15) To ensure the reliability of data transmission between satellites and users, Right now The above threshold can meet the basic communication requirements, and the threshold is recorded as For users, if , it indicates that the data transmission is successful, otherwise, the data transmission fails.

[0033] Based on this, the effect value C of the jammer is defined as the attack channel The number of users that cause data transmission to be disrupted, that is: (16) in, is a binary indicator function, and its expression is: .

[0034] Based on the above analysis, the goal of the jammer is to dynamically and adaptively adjust the attack channel by sensing the environment to maximize the attack effectiveness, namely: (17) 4. Channel selection game First, the optimization problem is modeled using a game model, and then the equilibrium solution of the game is given.

[0035] 4.1 Constructing the Steinberg Game Model The Steinberg game, also known as a hierarchical game, includes two participants with different attributes, namely leaders and followers, which can well describe the adversarial relationship. The adversarial relationship between the jammer and the ground user is a hierarchical Steinberg game, expressed as ,in, is the ground user set, For jammers, and are the policy sets for users and interference respectively, and They are the effect functions of the user and the jammer, respectively. With the jammer as the leader and the ground user as the follower, the jammer prioritizes environmental perception and strategy update to maximize the corresponding effect function. After the jammer completes the strategy update, the ground user perceives and adjusts to optimize the corresponding effect function.

[0036] 4.1.1 Leader Subgame For the jammer, the goal is to disrupt as much data transmission between the satellite and the user as possible, so its corresponding effect function is Defined as: (18) Therefore, the leader subgame can be expressed mathematically as: (19) Then the leader subgame problem can be described as finding the best strategy , in order to maximize its effect function, that is: (20) 4.1.2 Follower Subgame In the traditional game model, users are "selfish" and the utility function only considers the benefits they obtain. The consequence of this "selfishness" is that the global optimum cannot be achieved, because the best benefits of users are affected by their co-channel users. Therefore, in order to achieve the global optimum, a local cooperation model is introduced. Each user considers the benefits of himself and the adjacent users affected by the co-channel to seek greater benefits. Possibly with radius There is potential co-channel interference with other users within the range, and the users with potential interference are considered as users Neighbors of , denoted as: (twenty one) in, To remove a user Other users of For users and k The distance between The protection radius of the user.

[0037] Therefore, for the user, the corresponding effect function Defined as: (twenty two) in, For The channel selection sequence of other users, For The channel selection sequence of adjacent users, Fork Channel selection sequence of adjacent users.

[0038] Therefore, the follower subgame can be mathematically expressed as: (twenty three) Then the follower subgame can be described as finding the best strategy , in order to maximize its effect function, that is: (twenty four) 4.2 Game Equilibrium Analysis Definition 1 (Nash equilibrium NE): Given a non-cooperative game , strategy set is a Nash equilibrium if and only if hour, , satisfying the conditions: (25) That is, no participant can increase the value of the effect function by unilaterally changing its strategy, so the situation is called a pure strategy Nash equilibrium.

[0039] Definition 2 (Potential Game): Given a finite strategy game , for any game participant and two different strategies , , if the game is an exact potential game, then the conditions are met: (26) in, Function is the potential energy function of the game.

[0040] In the exact potential energy game, the change in the utility function and the change in the potential energy function caused by any user unilaterally changing their strategy are the same.

[0041] Potential energy games have two important properties: (1) Any exact potential game has at least one pure strategy Nash equilibrium.

[0042] (2) The global or local optimal solution of the potential energy function is a Nash equilibrium.

[0043] Theorem 1: When the jammer's jamming strategy is given Then, the follower subgame It is an exact potential game and there exists at least one pure strategy Nash equilibrium.

[0044] Proof: First, construct the potential energy function as follows: (27) When the user Unilaterally change the channel strategy from becomes , the effect function changes to: u i ( a i ′ , α − i , c J ) − u i ( a i , α − i , c J ) = q i ( a i ′ , α I i , c J ) − q i ( a i , α I i , c J ) + ∑ k ∈ I i [ q k ( a k , α I k ′ , c J ) − q k ( a k , α I k , c J ) ] (28) At this time, the change of potential energy function is: (29) Because the user Only the utility of neighbor users is considered, so: (30) Therefore, the change in potential energy function is further written as: Φ ( a i ′ , α − i , c J ) − Φ ( a i , α − i , c J ) = q i ( a i ′ , α I i , c J ) − q i ( a i , α I i , c J ) + ∑ k ∈ I i [ q k ( a k , α I k ′ , c J ) − q k ( a k , α I k , c J ) ] (31) In summary, (32) It can be seen that the follower subgame is an exact potential energy game, that is, the change in the user's utility function caused by unilaterally changing the strategy is equal to the change in the potential energy function. According to the properties of the exact potential energy game, there exists a pure strategy Nash equilibrium, that is, the maximum value of the potential energy function is a pure strategy Nash equilibrium of the follower subgame.

[0045] The constructed Steinberg game has an equilibrium solution, and the proof process is as follows: Definition 3 (Steinberg equilibrium SE): If the strategy combination satisfy: (33) Then this strategy combination is the Steinberg equilibrium solution of the proposed game, that is, no participant can improve the overall utility by unilaterally changing the strategy. In order to maximize the channel allocation strategy for user benefits, Channel allocation strategy to maximize the jammer's effectiveness.

[0046] Theorem 2: The Steinberg game constructed by the present invention has an SE solution consisting of the jammer's stable strategy and the user's Nash equilibrium strategy.

[0047] Proof: Given a jamming strategy , the Steinberg game becomes a local cooperative game. Based on Theorem 1, the follower subgame is an exact potential game, and there is at least one pure strategy NE solution Similarly, when a user policy is given , there is at least one NE solution in the leader subgame The stable strategies of the leader game and the follower game constitute a SE, namely: (34) Because every finite strategy game has a mixed strategy equilibrium, due to the limited number of users, the available channel solutions are also limited, and the interference strategy is also restricted. This constitutes the SE in the sense of stationarity for this problem.

[0048] 5. Equilibrium solution of the Steinberg game model The hierarchical anti-interference learning algorithm is used to obtain the equilibrium solution of the Steinberg game model.

[0049] 5.1 Algorithm Description In order to obtain the Steinberg game equilibrium solution of the constructed hierarchical game, a MSAP hierarchical anti-interference learning algorithm is proposed based on the multi-armed bandit (MAB) learning mechanism and the spatial adaptive algorithm (SAP). In the proposed algorithm, the jammer and satellite users update their strategy choices at different time scales, and the leader updates the strategy at a periodic Update the channel selection once, and the follower uses the time slot Update the channel selection once every cycle Include Time slots ,Right now The strategy update iterations of the leader and follower during the execution of the proposed hierarchical channel selection learning algorithm are as follows: Figure 3 shown.

[0050] In the lower-level subgame, SAP, as a learning algorithm, can converge to the pure strategy Nash equilibrium that maximizes the potential function with arbitrarily large probability. In order to characterize SAP, the game is expanded into a mixed strategy form. The mixed strategy in the kth iteration is expressed as a probability distribution: ,in, For the action set In SAP, a user is randomly selected according to the hybrid strategy to update its channel selection, while other users remain unchanged. This process is repeated until the stopping rule is met.

[0051] In the upper-level sub-game, the channel state and user information are unknown. In order to obtain an equilibrium solution, a channel selection algorithm based on MAB is proposed. Each channel is considered as an arm. The jammer balances exploration and utilization by selecting the channel with the highest UCB (UpperConfidence Bound) value through interaction with the environment. The UCB algorithm can effectively approach the optimal channel while continuously learning the channel performance.

[0052] In the present invention The cumulative return in the period is: (35) Based on this, the expected cumulative return is: (36) use Indicates channel in front The number of times selected in a moment is expressed as: (37) in, represents the indicator function, defined as: (38) In the MAB problem, the strategy does not always play the best role. Regret is an important metric that determines the performance of the MAB algorithm.

[0053] The regret value at this moment is: E [ Β ( t ) ] = E [ r * ( t ) ] − E [ r ( t ) ] = ∑ j ≠ j ∗ ( u J j ∗ − u J j ) E [ χ j ( t ) ] (39) ; in, yes The upper bound of , To select a channel in front The number of times a time slot is selected. In this problem, the goal is to maximize the long-term reward, that is, to minimize the regret function.

[0054] The detailed steps of the strategy update iteration of the leader and follower during the execution of the proposed hierarchical anti-interference learning algorithm are shown in Algorithm 1. First, the jammer selects the channel to be attacked in the current cycle; then, the satellite user learns the environment until its subgame converges to the Nash equilibrium solution; then, the jammer learns the environment and selects the optimal channel; the process is repeated until the maximum number of iterations is reached or the channel selection strategies of adjacent time slots are the same. After multiple iterations, both the leader subgame and the follower subgame converge to the Nash equilibrium solution, which is the Steinberg equilibrium solution of the hierarchical game.

[0055] Algorithm 1: MSAP hierarchical anti-interference learning algorithm Step 1: Setup , , , the user randomly generates a channel access strategy.

[0056] Step 2: In a period, the interference selects a channel according to formula (41) .

[0057] Step 3: The user’s learning process is as follows, loop k=0,1,2,….

[0058] ① In the In each time slot, all users exchange information with their neighbors; ② Randomly select a user ; ③ Based on the information of neighbor users, calculate the rewards of all actions of the user , The channel selection strategies of other users remain unchanged, i.e. ; ④ The Boltzmann update formula (40) is used to determine the action selected this time. The user Randomly select an action; (40) in, is the learning parameter, and . Effect function Calculated according to formula (22).

[0059] End of loop Step 4: Interference to obtain utility based on user strategy ,in .

[0060] Step 5: Interference selects the channel with the highest UCB value and updates the number of channel selections and the total channel return.

[0061] (41) in, , For at the moment Select Channel The total return obtained, For Channel The number of times selected.

[0062] Step 6: Update , execute Step 2 until the maximum number of cycles is reached.

[0063] 5.2 Convergence Analysis Analyze the convergence of the SAP algorithm used in the follower subgame. The set of all user channel selection combinations is .

[0064] Theorem 3: Given a jamming strategy After that, when learning parameters When is large enough, the SAP learning algorithm can converge to the global optimal solution of the follower subgame with a probability close to 1. .

[0065] Proof: All users in The channel selection combination of the time slot is expressed as ,in for Time slot user Selected channel. Since the algorithm SAP selects only one user to randomly detect a channel each time, is a non-periodic irreducible discrete Markov chain, so has a unique steady-state distribution.

[0066] Let the channel combinations of the users in two consecutive iterations be a and b respectively, then the transition probability between the two is P a b = P [ a ( k + 1 ) = b | a ( k ) = a ] .

[0067] The Markov equilibrium equation is: (42) The distribution that satisfies the Markov equilibrium equation is the only steady-state distribution. Since in the proposed algorithm only one user is selected to update its choice, at most one element in the network state can be changed between any two consecutive iterations, which can be verified Satisfies the Markov process equilibrium condition, where is the potential energy function. Therefore It is the only equilibrium stable distribution.

[0068] The only action that maximizes the potential energy function is (43) Right now ,when As it approaches infinity, the following inequality holds: (44)

[0069] Then, based on equations (43) and (44), we can obtain: (45) The lower-level SAP algorithm will converge to the maximum value of the potential energy function of the follower subgame with a probability close to 1, which is the global optimal solution.

[0070] The convergence of the MAB algorithm used in the leader subgame is proved as follows: Theorem 4: The proposed regret function grows at a logarithmic rate, i.e. E [ Β ( t ) ] ∼ O ( l o g t ) , as time goes by, the selected strategy gets closer and closer to the optimal strategy, and the upper bound of the overall regret value function can be expressed as follows: E [ Β ( t ) ] ≤ 8 ∑ j : u J j < u J * l n ( t ) Δ j + ( 1 + π 2 3 ) ( ∑ j = 1 M Δ j ) (46) prove: E [ χ j ( t ) ] ≤ 8 l n ( t ) Δ j 2 + 1 + π 2 3 (47) in, .

[0071] Furthermore, the upper bound of the expected regret function is obtained as follows: E [ Β ( t ) ] = ∑ j ≠ j ∗ ( u J j ∗ − u J j ) E [ χ j ( t ) ] ≤ ∑ j ≠ j ∗ ( u J j ∗ − u J j ) ( 8 l n ( t ) Δ j 2 + 1 + π 2 3 ) = ∑ j ≠ j ∗ Δ j ( 8 l n ( t ) Δ j 2 + 1 + π 2 3 ) = ∑ j : u J j < u J * Δ j ( 8 l n ( t ) Δ j 2 + 1 + π 2 3 ) = 8 ∑ j : u J j < u J * l n ( t ) Δ j + ( 1 + π 2 3 ) ( ∑ j = 1 M Δ j ) (48) In summary, we can finally get E [ Β ( t ) ] ≤ 8 ∑ j : u J j < u J * l n ( t ) Δ j + ( 1 + π 2 3 ) ( ∑ j = 1 M Δ j ) , the regret function converges.

[0072] Simulation results of anti-interference channel selection method for satellite-to-ground network based on Steinberg game The present invention uses the performance of the layered anti-interference learning algorithm to perform simulation analysis. Users are randomly distributed in the beam, and the schematic diagram of user random distribution is as follows: Figure 4 As shown, the learning parameters It increases with the number of iterations. The specific parameter settings are shown in Table 2: Table 2

[0073] The present invention uses the network-wide satisfaction rate to measure the overall QoE of the network, which is defined as follows: (49) in, For users QoE value, N is the number of users in the whole network.

[0074] Figure 5 The curves of the network satisfaction rate change when the number of channels is 3, 4, and 5 are given respectively. It can be seen that the network satisfaction rate eventually converges to the optimal solution as the number of iterations increases. At the same time, the network satisfaction rate increases with the increase in the number of available channels, which shows that sufficient channel resources can make the user's QoE value higher, reduce the probability of users and jammers or users and neighboring users using the same channel, bring better user experience, and improve the overall network satisfaction.

[0075] Figure 6 The expected change curves of the jammer's regret value are given when the number of channels is 3, 4, and 5. It can be seen that the jammer's regret value expectation increases with the increase in the number of channels, which means that the increase in the number of channels makes the user experience better, which reduces the probability that the jammer successfully interferes with the user, thus increasing the regret value and causing the expected regret value to increase.

[0076] Figure 7A comparison chart of the network satisfaction rate under the algorithm proposed in the present invention, the optimal NE and the random selection algorithm is given. It can be seen that the performance of the proposed hierarchical learning algorithm is better than the random selection algorithm and is close to the optimal NE, which verifies the effectiveness of the algorithm proposed in the present invention and ultimately achieves a higher network satisfaction rate.

[0077] The present invention considers the channel selection optimization problem under malicious interference scenarios in satellite-to-ground networks, and proposes an anti-interference channel selection mechanism based on the Steinberg game. First, the adversarial relationship between the user and the jammer is modeled as a Steinberg game, with the jammer as the leader, whose goal is to maximize the attack and destruction effectiveness, and the user as the follower, whose goal is to maximize the satisfaction rate of the entire network. Secondly, considering the influence of neighboring users, the follower sub-game is expressed as a local altruistic game, and it is proved to be an exact potential energy game, and the QoE evaluation index is introduced to increase the user experience. For the constructed game model, the existence of the game equilibrium solution is analyzed. Finally, in order to obtain the game equilibrium solution, an MSAP hierarchical anti-interference learning algorithm is proposed. The simulation results show the convergence and effectiveness of the proposed algorithm, and it has good optimization performance in anti-interference channel selection.

[0078] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made on the basis of the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. A satellite-to-ground network anti-interference channel selection method based on Steinberg game, characterized in that: include: Step S01: Establishing a system scenario to simulate the communication between a user and a satellite, including the interference to which the user is subjected when the user selects a channel to communicate with the satellite; Step S02: constructing a system model according to the system scenario, obtaining the user's throughput according to the satellite's transmitting antenna gain, the user's receiving antenna gain and the interference suffered by the user, and performing a satisfaction evaluation on the throughput to obtain a satisfaction result; Step S03: Optimizing the channel selected by the user according to the satisfaction result to construct an optimization problem; Step S04: constructing a Steinberg game model according to the optimization problem, using a hierarchical anti-interference learning algorithm to obtain an equilibrium solution for the Steinberg game model, and completing the satellite-to-ground network anti-interference channel selection based on the Steinberg game.

2. The satellite-to-ground network anti-interference channel selection method based on Steinberg game according to claim 1 is characterized in that: The interference to the user in step S01 includes: co-frequency interference between different users in a beam, co-frequency interference between beams, and malicious interference.

3. The satellite-to-ground network anti-interference channel selection method based on Steinberg game according to claim 1 is characterized in that: The throughput of the user is: ； ； in, is the channel bandwidth, is the signal power received by the user, is the co-channel interference from other users in the same beam of the LEO satellite in the downlink, is the co-channel interference from other beams of the LEO satellite in the downlink, For malicious interference from jammers, For Link The channel noise, is the transmit power of the antenna beam, is the maximum gain of the satellite transmitting antenna, is the maximum gain of the user receiving antenna, is the wavelength of communication transmission, For Link distance.

4. The satellite-to-ground network anti-interference channel selection method based on Steinberg game according to claim 3 is characterized in that: The transmitting antenna gain of the satellite is: ， ， The receiving antenna gain of the user is: ， ； in, is the satellite antenna off-axis angle, is the antenna half-power beamwidth, is the level at which the gain of the antenna main beam and the near-field sidelobe mask crosses below the peak gain, is the far-field sidelobe level, is the user antenna off-axis angle, is the maximum gain of the satellite transmitting antenna, is the maximum gain of the user receiving antenna, is the wavelength of communication transmission, is the circular equivalent diameter of the antenna.

5. The satellite-to-ground network anti-interference channel selection method based on Steinberg game according to claim 3 is characterized in that: ； ； ； ； in, , is the transmission power of the satellite beam, is the transmit power of the jammer, , For users Satellite antenna interference gain in direction, is the user antenna receive gain in the direction of the jammer, , is the distance of the interfering link, is the maximum gain of the user receiving antenna, is the wavelength of communication transmission, , For users k , Select the channel, A function to determine whether the jammer and the user select the same channel.

6. The satellite-to-ground network anti-interference channel selection method based on Steinberg game according to claim 1, characterized in that: The method for evaluating the throughput for satisfaction and obtaining a satisfaction result includes: calculating an average estimated score, using the average estimated score to describe the subjective QoE feeling, and obtaining a satisfaction result; The calculation formula of the average estimated score MOS is: ； The user's satisfaction is expressed as: ； in, is the user throughput, , and is a constant, For user satisfaction, For users The channel selection strategy, To remove the user The channel selection strategy set of other users, is the channel selection of the jammer, The throughput is The score value at that time.

7. The satellite-to-ground network anti-interference channel selection method based on Steinberg game according to claim 1, characterized in that: The Steinberg game model is expressed as: ； in, is the ground user set, For jammers, and are the policy sets for users and interference respectively, and are the effect functions of user and interference respectively.

8. The satellite-to-ground network anti-interference channel selection method based on Steinberg game according to claim 7 is characterized in that: The Steinberg game model involves finding the best strategy , expressed as the maximum leader effect function: ； ； ， ； in, For users Select the channel, Select a policy set for a user channel, is the channel selection strategy of the jammer, To meet the satisfaction threshold of basic communication, The throughput is Satisfaction at the time, To determine whether the jammer and the user select the same channel, To determine the attack channel Whether to corrupt the data transfer function.

9. The satellite-to-ground network anti-interference channel selection method based on Steinberg game according to claim 7, characterized in that: The Steinberg game model involves finding the best strategy , expressed as the maximum follower effect function: ； ； ； in, For users The channel selection strategy, To remove the user The channel selection strategy set of other users, is the channel selection of the jammer, For users satisfaction, For users Neighbors, For The channel selection sequence of adjacent users, For k The channel selection sequence of adjacent users, To remove a user Other users of For users and k The distance between The protection radius of the user.

10. The satellite-to-ground network anti-interference channel selection method based on Steinberg game according to claim 1, characterized in that: The equilibrium solution is: ； ； in, In order to maximize the channel allocation strategy for user benefits, In order to maximize the channel allocation strategy of the jammer, is the user's effect function, is the effect function of the jammer.

Images