Trusted spectrum trading method based on Stackelberg game in cognitive satellite-ground network

Abstract

The present invention discloses a trusted spectrum trading method based on Stackelberg game under cognitive satellite-ground network, including: constructing a reputation value model based on cognitive satellite-ground network spectrum trading, and obtaining the reputation value of each ground relay base station; constructing a cognitive satellite-ground network spectrum trading game model; solving the revenue optimization model of the ground relay base station, and obtaining the optimal service power of the ground relay base station; solving the revenue optimization model of satellite users, and obtaining the optimal satellite spectrum time allocation factor; transmitting signals according to the optimal satellite spectrum time allocation factor and the service power of the ground relay base station; updating the historical transaction reputation value. The present invention establishes a one-to-many Stackelberg game model to allow more ground relay base stations to participate in the transaction and solve the problem of spectrum shortage; establishes a reputation value model, and considers the recommended reputation values ​​of other satellites and the historical transaction reputation values ​​of satellites at the same time, to solve the influence of malicious ground relay base stations on satellites.

Description

Technical Field

[0001] The present invention belongs to the field of communication technology, and in particular relates to a trusted spectrum trading method based on Stackelberg game in a cognitive satellite-to-ground network. Background Art

[0002] In the cognitive satellite-to-ground network scenario, the direct link between the satellite and the satellite user is blocked, and the ground relay base station near the satellite user needs to assist in transmission. With the increase in the number of ground relay base station users, spectrum resources are in short supply, and the ground relay base station needs to use the satellite's spectrum to complete the transmission of its own users. PKSharma et al. first proposed that secondary ground users assist the main satellite transmission through cooperative relay technology in exchange for spectrum access. In order to solve the problem of resource competition between selfish users, game theory has been widely used in the cognitive satellite-to-ground network spectrum trading scenario to solve the problem of maximizing utilization under scarce resources. X. Zhang et al. proposed a relay selection based on the Vickrey auction mechanism, allowing secondary networks to compete for a spectrum sharing channel. X. Zhang, D. Guo et al. studied the use of auction algorithms for auction-based multi-channel cooperative spectrum sharing in cognitive satellite-to-ground networks, and proposed an auction-based optimization problem to maximize the total transmission rate of satellite users while meeting the minimum transmission rate of each secondary network. Liu Shasha used the Stackelberg game model to analyze the game between primary and secondary users in the scenario of single-user cooperative spectrum leasing.

[0003] Although the above scheme introduces game theory into the spectrum trading scenario of cognitive satellite-to-ground network, it does not take into account that one-to-one trading only solves the spectrum shortage problem of one ground relay base station, and still cannot solve the spectrum shortage problem of other ground relay base stations. In order to allow more users to participate in the transaction, Li Linling et al. modeled the spectrum allocation problem between operators as a generalized Stackelberg game, proved the superiority of Stackelberg game in solving spectrum game problems, and established a one-to-many Stackelberg game model. The ground primary user rents the frequency band to multiple ground secondary users and provides spectrum usage time in exchange for multiple secondary users to provide power to assist the primary user in transmission. Although the one-to-many Stackelberg game is used to allow more ground relay base stations to participate in the transaction, the impact of the actual service situation of the secondary network on the benefits of satellite users is not considered. This transaction is based on the honest service of the ground relay base station. If the ground relay base station conducts malicious transactions, it will affect the benefits of satellite users. In addition, the one-to-many Stackelberg game is mostly solved by iteration and reverse induction. The time complexity of the iteration method is high, and it is easy to have privacy leakage problems that affect the fairness of the transaction. The backward induction method is difficult to solve in multi-dimensional problems and is not suitable for solving them.

[0004] As mentioned above, on the one hand, the existing cognitive satellite-to-ground network spectrum resource transactions only conduct one-to-one games. In actual trading scenarios, there is a problem of spectrum shortage in multiple ground relay base stations. Therefore, in this case, it is necessary to establish a one-to-many Stackelberg game for trading. On the other hand, the existing cognitive satellite-to-ground network spectrum resource transactions do not consider the reputation value, cannot screen malicious ground relay base stations, guarantee the benefits of satellites, and have no way to motivate ground relay base stations to actively participate in transactions. At the same time, the existing technology uses traditional solution optimization algorithms when solving one-to-many Stackelberg games, which requires participants to publish bids, which is not conducive to protecting the privacy of bidders, affects the fairness of the game, and is difficult to solve when the game model is complex. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a cognitive satellite-to-ground network trusted spectrum trading method based on Stackelberg game. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] The present invention provides a trusted spectrum trading method based on Stackelberg game in a cognitive satellite-to-ground network, comprising:

[0007] S1: construct a reputation value model based on cognitive satellite-to-ground network spectrum trading, and obtain the reputation value of each ground relay base station according to the reputation value model;

[0008] S2: constructing a cognitive satellite-to-ground network spectrum trading game model according to the reputation value, wherein the game model includes a revenue optimization model for satellite users and a revenue optimization model for ground relay base stations;

[0009] S3: solving the revenue optimization model of the ground relay base station by using the reverse induction method to obtain the optimal service power of the satellite users of the ground relay base station;

[0010] S4: solving the revenue optimization model of the satellite user by using a deep reinforcement learning method to obtain an optimal satellite spectrum time allocation factor;

[0011] S5: performing signal transmission according to the optimal satellite spectrum time allocation factor and the optimal service power of the satellite user of the ground relay base station;

[0012] S6: After the current transaction is completed, the historical transaction reputation value is updated.

[0013] In one embodiment of the present invention, the S1 includes:

[0014] S1.1: Obtain the recommended reputation value of each ground relay base station;

[0015] S1.2: Obtain the historical transaction reputation value of each ground relay base station;

[0016] S1.3: Obtaining the overall reputation value of each ground relay base station according to the recommended reputation value and the historical transaction reputation value:

[0017] Rep si (t) = βRept si (t)+(1-β)Rep rec-i (t),

[0018] Among them, Rep. si (t) represents the overall reputation value of the ground relay base station i obtained by trading satellite s before the current transaction, Rept si (t) represents the historical transaction reputation value of the transaction satellite s to the ground relay base station i, Rep rec-i (t) represents the recommended reputation value of other satellites to the ground relay base station i, β is the weighting factor of the historical transaction reputation value of the transaction satellite s, and the expression is:

[0019]

[0020] Among them, t N represents the historical transaction times between the transaction satellite s and the ground relay base station i, and μ represents the weighting factor coefficient.

[0021] In one embodiment of the present invention, the expression of the recommendation reputation value is:

[0022]

[0023] Among them, η ji (t) represents the recommendation weight of the jth recommended satellite to the ith ground relay base station, M is the number of recommended satellites, Rep ji (t) represents the historical transaction reputation value between the j-th recommended satellite and the i-th ground relay base station;

[0024] The expression of the recommendation weight is:

[0025]

[0026] Among them, ρ ji represents the historical transaction times between the jth recommended satellite and the ith ground relay base station, ω ji (t) represents the historical recommendation accuracy of the jth recommended satellite to the ith ground relay base station, γ is the transaction time coefficient, t x is the current time, t ji Indicates the most recent transaction time between the j-th recommended satellite and the j-th ground relay base station.

[0027] In one embodiment of the present invention, the calculation process of the accuracy of the historical recommendations includes:

[0028] Calculate the accuracy of recommendations in the current transaction:

[0029]

[0030] in, Rep represents the accuracy of the recommendation opinion of satellite j for the i-th ground relay base station in the current transaction, si (t) is the transaction reputation value of the transaction satellite s to the i-th ground relay base station in the current transaction, Rep ji (t) is the historical transaction reputation value between the jth recommended satellite and the ith ground relay base station, and m is the coefficient for evaluating the accuracy of the recommendation opinion;

[0031] The accuracy of the recommendations in the current transaction is used to update the accuracy of the historical recommendations:

[0032]

[0033] Among them, λ is the exponential moving average update parameter, ω ji (t-1) is the accuracy of the recommendation opinion of the historical transaction recommendation satellite to the i-th ground relay base station, t represents the number of historical transactions, and t=0 represents no historical transaction.

[0034] In one embodiment of the present invention, the S1.2 includes:

[0035] S1.21: Calculate the average transmission power of the ground relay base station:

[0036]

[0037] in, is the average power transmitted by the ground relay base station i to the ground user of the trading satellite s, L is the number of times the transmission power of the ground relay base station is monitored, and P si (l) is the transmission power of the i-th ground relay base station serving the ground users of the trading satellite s obtained by the l-th monitoring;

[0038] S1.22: Calculate the variance of the transmission power of the ground relay base station:

[0039]

[0040] S1.23: Calculate the probability that the ground relay base station transmits power according to the transaction agreement:

[0041] in, represents the transmission power of the i-th ground relay base station;

[0042] S1.24: Obtain the credit value of this transaction

[0043] S1.25: Update the reputation value of this transaction to obtain the historical transaction reputation value:

[0044]

[0045] Wherein, t represents the number of historical transactions, t=0 represents no historical transactions, and λ represents the exponential moving average update coefficient.

[0046] In one embodiment of the present invention, the expression of the revenue optimization model of the satellite user is:

[0047]

[0048] in, represents the transmission rate obtained by the satellite user, α i is the spectrum usage time factor allocated by the trading satellite to the i-th ground relay base station, represents the service power of the ground relay base station to the satellite user, B is the satellite spectrum bandwidth, T is the idle time of the satellite spectrum, N is the number of ground relay base stations participating in the transaction, SNR S is the signal-to-noise ratio of the signal received by the satellite user, Indicates the transmission rate obtained by the relay base station user;

[0049] The expression of the revenue optimization model of the ground relay base station is:

[0050]

[0051] in, is the transmission rate obtained by the user of the i-th ground relay base station, Indicates the power of the ground relay base station to provide services to base station users. represents the channel coefficient from the i-th ground relay base station to the base station user, n0 is Gaussian white noise, C1 is the energy pricing of the ground relay base station assisting satellite users in transmission, C2 is the energy pricing of the ground relay base station serving its own users, C3 is the parameter of the reputation value affecting the transaction, Rep si (t) is the overall reputation value of the i-th ground relay base station calculated by the transaction satellite, P max It is the maximum service power of the ground relay base station.

[0052] In one embodiment of the present invention, S3 includes:

[0053] S3.1: Reverse the expression of the revenue optimization model of the ground relay base station to obtain a modified revenue optimization model:

[0054]

[0055] S3.2: Introduce the slack variable υ and transform the constraints into equality constraints:

[0056]

[0057] S3.3: Constructing Lagrangian function using the modified revenue optimization model

[0058]

[0059] S3.4: Set the partial differential equations of the Lagrangian function to 0, and calculate the optimal service power for satellite users of the ground relay base station.

[0060] In one embodiment of the present invention, the S4 includes:

[0061] S4.1: Optimal service power for satellite users at ground relay base stations Substituting the constraints into the satellite user's revenue optimization model, we get:

[0062]

[0063] S4.2: The satellite user is modeled as a single agent, and a centralized control center collects global information and controls the satellite and each ground relay base station, wherein the global information includes the calculated overall reputation value and the optimal service power of each ground relay base station to the satellite user;

[0064] S4.3: Build a single-agent based deep reinforcement learning model;

[0065] S4.4: Use the single-agent based deep reinforcement learning model to solve the satellite user's revenue optimization model and obtain the optimal satellite spectrum time allocation factor.

[0066] In one embodiment of the present invention, the single-agent based deep reinforcement learning model includes state, action and reward functions, wherein:

[0067] The status t ={α1,α2,...,α i ,...,α N} is the spectrum usage time allocated by the trading satellite to the ground relay base station. The user is in the current state s t The vector composed of the Q values ​​of all actions is:

[0068] y=[Q(s t ,a 0 ),Q(st ,a 1 ),...,Q(s t ,a i ),...,Q(s t ,a N )],

[0069] Among them, a i is an action selected;

[0070] The actions are:

[0071]

[0072] Among them, random is the set random number, eplison is the decay greed factor, and random(a) is the randomly selected action;

[0073] The reward function is:

[0074]

[0075] in, β is the penalty factor.

[0076] In one embodiment of the present invention, the S4.3 includes:

[0077] Take the initially set time allocation factor as the initial state, and output the Q values ​​of all actions in the current state action space; select the action with the largest Q value or a random action according to the random number and greed factor, change the current state, update the Q network, record the reward of the selected action, accumulate and sum the rewards, and save the experience into the experience replay pool;

[0078] Continue to select actions in the current state until the number of steps reaches the set number of steps per round, and update the target network;

[0079] Reset the environment state and repeat the steps until the set number of training rounds is reached, and the optimal satellite spectrum time allocation factor is obtained.

[0080] Compared with the prior art, the present invention has the following beneficial effects:

[0081] 1. The present invention provides a trusted spectrum trading method based on Stackelberg game in a cognitive satellite-to-ground network. First, a one-to-many Stackelberg game model is established in the spectrum trading scenario of the cognitive satellite-to-ground network to allow more ground relay base stations to participate in the transaction and solve the spectrum shortage problem; secondly, a ground relay base station reputation model is established according to the transaction model, taking into account the recommended reputation values ​​of other satellites and the reputation values ​​of satellite historical transactions, to solve the impact of malicious ground relay base stations on satellite revenues, and at the same time to encourage ground relay base stations to actively participate in transactions; finally, the reverse induction method and deep reinforcement learning are used to solve the game model to avoid privacy leakage and reduce the difficulty of solving.

[0082] 2. The reputation model constructed by the present invention modifies the weighting method of the recommendation reputation value and the transaction reputation value, modifies the calculation method of the recommendation reputation value, adds the recommendation accuracy and the recommender's recent transaction time, accurately describes the behavior of malicious ground relay base stations, and gives a more accurate evaluation. The present invention uses deep reinforcement learning to solve the problem of unfair transactions caused by the difficulty of solving traditional optimization algorithms and privacy leakage, protects the privacy of ground relay base stations, and ensures fair transactions.

[0083] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 It is a flow chart of a trusted spectrum trading method based on Stackelberg game in a cognitive satellite-to-ground network provided by an embodiment of the present invention;

[0085] Figure 2 It is a scene diagram of a cognitive satellite-to-ground network spectrum trading signal model provided by an embodiment of the present invention;

[0086] Figure 3 This is a schematic diagram of transaction time allocation provided by an embodiment of the present invention;

[0087] Figure 4 This is a schematic diagram of a method for calculating an overall reputation value provided by an embodiment of the present invention;

[0088] Figure 5 is a schematic diagram of a deep reinforcement learning network provided by an embodiment of the present invention;

[0089] Figure 6 It is the curve of the recommendation weight of each recommended satellite for different ground relay base stations as the number of recommendations changes;

[0090] Figure 7 It is a schematic diagram showing the change of the recommendation reputation value of the recommendation satellite to the ground relay base stations with different behaviors as the number of recommendations increases;

[0091] Figure 8 It is a schematic diagram of transaction reputation values ​​of ground relay base stations with different behaviors;

[0092] Fig. 9 Schematic diagram of comparison of transaction reputation values ​​of ground relay base station D obtained by different methods;

[0093] Fig.10 is a schematic diagram comparing the overall reputation values ​​of various ground relay base stations obtained by different methods;

[0094] Fig.11 It is a curve diagram showing the impact of the change in the reputation value of the ground relay base station A on the benefits of satellite users;

[0095] Fig.12 It is a schematic diagram of the time allocation factor of each ground relay base station changing with the reputation value of the ground relay base station A;

[0096] Fig.13 This is a comparison chart of the impact of considering and ignoring the reputation value when a satellite conducts transactions with four ground relay base stations;

[0097] Fig.14 It is a comparison chart of the benefits of satellite users obtained by using the method of the embodiment of the present invention and the existing iterative method as the number of training times increases. DETAILED DESCRIPTION

[0098] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following is a detailed description of a trusted spectrum trading method based on Stackelberg game in a cognitive satellite-to-ground network proposed by the present invention in combination with the accompanying drawings and specific implementation methods.

[0099] The above and other technical contents, features and effects of the present invention are clearly presented in the following detailed description of the specific implementation modes in conjunction with the accompanying drawings. Through the description of the specific implementation modes, the technical means and effects adopted by the present invention to achieve the predetermined purpose can be more deeply and specifically understood. However, the attached drawings are only for reference and explanation purposes and are not used to limit the technical solutions of the present invention.

[0100] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants are intended to cover non-exclusive inclusion, so that an article or device including a series of elements includes not only those elements, but also other elements that are not explicitly listed. In the absence of more restrictions, the elements defined by the statement "including one..." do not exclude the existence of other identical elements in the article or device including the elements.

[0101] The present invention first designs a reputation model based on cognitive satellite-to-ground network spectrum trading. The satellite calculates the reputation of the ground relay base station, considers the recommended reputation value and the transaction reputation value calculation, affects the time allocated by the ground relay base station in the transaction, encourages honest ground relay base stations to actively participate in the transaction, and guarantees the satellite's income. On this basis, a one-to-many Stackelberg game model is constructed, and multiple ground relay base stations participate in the transaction at the same time to solve the spectrum shortage problem. Finally, the deep reinforcement learning method is used to solve the Stackelberg game model, simplify the solution complexity, and avoid privacy leakage problems affecting transaction fairness.

[0102] See also Figure 1 , Figure 1 The flowchart of a trusted spectrum trading method based on Stackelberg game in a cognitive satellite-to-ground network provided by an embodiment of the present invention. The trusted spectrum trading method includes:

[0103] S1: Construct a reputation value model based on cognitive satellite-to-ground network spectrum trading, and obtain the reputation value of each ground relay base station according to the reputation value model.

[0104] See also Figure 2 , Figure 2 This is a scenario diagram of a cognitive satellite-to-ground network spectrum trading signal model provided by an embodiment of the present invention. There is a multi-antenna satellite S. The direct link between the multi-antenna satellite S and its own ground user SU is blocked, and the ground communication is poor. The ground relay base station R has its own service user RU, but the ground relay base station is short of spectrum resources and cannot meet the user's needs. In this case, consider trading between the satellite and multiple ground relay base stations. The satellite needs to select a ground relay base station with good channel conditions near the satellite user to assist in transmission. The ground relay base station needs satellite spectrum usage time to serve its own users. The specific process is as follows: In the first stage, the satellite broadcasts and transmits to the ground relay base station. In the second stage, the ground relay base station precodes, amplifies and forwards the received signal to the satellite user. In the third stage, the ground relay base station transmits to its own user according to the time allocated by the transaction. Please refer to Figure 3 , Figure 3 is a schematic diagram of transaction time allocation provided by an embodiment of the present invention, wherein N is the number of ground relay base stations participating in the transaction, α i is the spectrum usage time factor allocated by the trading satellite to the i-th ground relay base station, and T is the idle time of the satellite spectrum.

[0105] Specifically, in the first stage, the multi-antenna low-orbit satellite broadcast is sent to the ground relay base station, and the transmission signal is precoded by conjugate beamforming (CB). For the ground relay base station, the signal of each beam is superimposed and received. The i-th ground relay base station R i The received signal is:

[0106]

[0107] Among them, P s represents the satellite transmission power, n0 is Gaussian white noise, represents the channel coefficient between the satellite and the i-th ground relay base station, in, The channel coefficient between the i-th satellite antenna and the i-th ground relay base station is expressed as:

[0108]

[0109] Among them, G S is the transmitting antenna gain of the low-orbit satellite, G R is the ground relay base station antenna receiving gain, is the channel coefficient of the shadow Rice channel between the i-th beam of the satellite and the i-th ground relay base station, is the path loss of free space propagation between the satellite and the ground relay base station, and the calculation formula is:

[0110]

[0111] Where H is the height of the low-orbit satellite and f is the center frequency of the low-orbit satellite.

[0112] In the second stage, the ground relay base station first normalizes the received signal. The satellite signal received by the i-th ground relay base station is The calculation formula is as follows:

[0113]

[0114] Here, x represents the signal sent by satellite broadcasting.

[0115] Then, the ground relay base station receives the normalized signal Precoding is performed to eliminate interference between signals received by satellite users. The signal sent by the i-th ground relay base station received by the satellite user is:

[0116]

[0117] in, is the transmission power of the i-th ground relay base station, which is also the transmission power agreed in the transaction. is the channel coefficient of the channel between the i-th ground relay base station and the satellite user, and the calculation formula is as follows:

[0118]

[0119] Where η is the small-scale path loss coefficient, usually 2. is the distance between the i-th ground relay base station and the satellite user, is the Rayleigh channel coefficient of the channel between the i-th ground relay base station and the satellite user. is the normalized weight coefficient of precoding, and the calculation formula is as follows:

[0120]

[0121] Among them, || means taking the absolute value, and * means conjugation.

[0122] Further, the signals sent by N ground relay base stations and y received by the satellite user are obtained. D , the calculation formula is as follows:

[0123]

[0124] The signal-to-noise ratio of the signal received by the satellite user is SNR S , the calculation formula is as follows:

[0125]

[0126] In the second stage, the total transmission rate obtained by the satellite user is R D , the calculation formula is as follows:

[0127] R D =Blog(1+SNR s ) (10)

[0128] Where B represents the spectrum bandwidth of the satellite.

[0129] In the third stage, the signal sent by the ground relay base station to its own base station user is:

[0130]

[0131] in, is the power of the i-th ground relay base station serving the base station user, is the channel coefficient from the i-th ground relay base station to its own user, and the calculation method is the same as formula (6).

[0132] The transmission rate obtained by the ground relay base station user in the third stage is The calculation formula is as follows:

[0133]

[0134] On this basis, this embodiment provides a reputation value model suitable for the cognitive satellite-to-ground network spectrum resource transaction scenario. Before each transaction with the i-th ground relay base station, the satellite calculates the reputation value of the ground relay base station. The score is obtained by weighting the historical transaction experience between the satellite and the i-th ground relay base station and the recommendations of other satellites on the ground relay base station. That is, the expression of the reputation value model is:

[0135] Rep si (t) = βRept si (t)+(1-β)Rep rec-i (t) (13)

[0136] Among them, Rep. si (t) represents the overall reputation value of the ground relay base station i obtained by trading satellite s before the current transaction, Rept si (t) represents the reputation evaluation of the transaction satellite s on the ground relay base station i in the historical transactions, that is, the historical transaction reputation value, Rep rec-i (t) represents the recommended reputation value of other satellites to the ground relay base station i, and β is the weighting factor of the historical transaction reputation value of the transaction satellite s in the final reputation value, as shown below:

[0137]

[0138] Among them, t N represents the historical transaction times between the transaction satellite s and the ground relay base station i, and μ represents the weighting factor coefficient.

[0139] See also Figure 4 , Figure 4 It is a schematic diagram of an overall reputation value calculation method provided by an embodiment of the present invention. When a satellite conducts a transaction with an unfamiliar ground relay base station, since there is no historical transaction information as a reference, it is impossible to accurately judge the reliability of the ground relay base station, and other satellites that have traded with the ground relay base station are required to make recommendations and provide recommended reputation values. In order to avoid malicious evaluations of the ground relay base station in transactions by other satellites, which affects the revenue of the transaction satellite, and considering that when the number of transactions between the satellite and the current ground relay base station increases, the satellite has more knowledge about the ground relay base station in transactions, and is more willing to evaluate the ground relay base station according to its own understanding, so after the satellite has traded with a ground relay base station for many times, the satellite evaluates the ground relay base station without considering the recommendations of other satellites, but only considering the historical transactions with it.

[0140] The recommended reputation value is to solve the problem that there is no historical transaction information between the transaction satellite and the unfamiliar ground relay base station as a reference, and the satellite cannot accurately evaluate the credibility of the ground relay base station. Therefore, other satellites are introduced to evaluate the reputation value of the ground relay base station according to the historical transaction evaluation with this ground relay base station. The recommended reputation value of other satellites for this ground relay base station is Rep rec-i (t), the calculation formula is as follows:

[0141]

[0142] Among them, η ji (t) represents the recommendation weight of the jth recommended satellite to the ith ground relay base station, M is the number of recommended satellites, Rep ji (t) represents the historical transaction reputation value between the j-th recommended satellite and the i-th ground relay base station.

[0143] In order to make accurate recommendations, it is necessary to judge the reliability of the recommendation opinions of the recommended satellites. In this embodiment, the recommendation weight of the recommended satellite is calculated according to the number of historical transactions between the recommended satellite and the recommended ground relay base station, the accuracy of the historical recommendation opinions of the recommended satellite to the ground relay base station, and the timeliness of the transaction between the most recent recommended satellite and the ground relay base station. Through this calculation method, the weight of the recommendation opinion of each recommended satellite to the i-th ground relay base station is counted, and then the recommendation reputation value is calculated in combination with the historical transaction reputation value of the recommended satellite.

[0144] The recommendation weight takes into account both the historical recommendation accuracy and transaction timeliness of the recommended satellite. The transaction timeliness is the time interval between the current transaction and the last transaction between the recommended satellite and the recommended ground relay base station. ji The calculation formula of (t) is:

[0145]

[0146] Among them, ρ ji represents the historical transaction times between the jth recommended satellite and the ith ground relay base station, ω ji (t) represents the historical recommendation accuracy of the jth recommended satellite to the ith ground relay base station, γ is the transaction time coefficient, t x Indicates the current time, t ji Indicates the most recent transaction time between the j-th recommended satellite and the i-th ground relay base station.

[0147] Furthermore, in order to show the accuracy of the historical recommendations of the recommending satellite on the ground relay base station, the accuracy of the recommendation of the recommending satellite can be obtained by comparing the reputation value recommended by the recommending satellite with the score of the ground relay base station given by the trading satellite after the transaction is completed. In this embodiment, it is considered that if the gap between the reputation value obtained by the recommending satellite based on historical transactions and the reputation value obtained by the trading satellite after the transaction is too large, it means that the recommendation of the recommending satellite is unreliable, and it is necessary to significantly reduce the recommendation accuracy of the recommending satellite, and at the same time reduce the recommendation weight of the recommending satellite, so that the trading satellite can select a more reliable recommended satellite for recommendation. Therefore, the calculation method of the recommendation accuracy is as follows:

[0148]

[0149] in, Rep represents the accuracy of the recommendation opinion of satellite j for the i-th ground relay base station in the current transaction, si (t) is the transaction reputation value of the transaction satellite s to the i-th ground relay base station in the current transaction, Rep ji (t) is the credibility value of the recommended satellite j for recommending the i-th ground relay base station in this transaction according to historical transaction conditions, that is, the historical transaction credibility value between the j-th recommended satellite and the i-th ground relay base station. m is the coefficient for evaluating the accuracy of the recommendation opinion, which is also used as the recommendation opinion deviation threshold. When the recommendation deviation is greater than this threshold, the recommendation opinion of the recommended satellite is not credible.

[0150] The accuracy of historical recommendation opinions is the accuracy of the recommendation opinions of satellite j to the i-th ground relay base station in historical transactions.

[0151] It should be noted that after each transaction is completed and the accuracy of the recommendation opinion of the recommended satellite is obtained, the accuracy of the recommendation opinion of the recommended satellite for the i-th ground relay base station needs to be updated in combination with the historical situation of the recommendation to obtain the historical recommendation opinion accuracy ω ji (t), the exponential moving average (EMA) method is used for updating.

[0152]

[0153] Among them, λ is the exponential moving average update parameter, ω ji (t-1) is the accuracy of the recommendation opinion of the historical transaction recommendation satellite to the i-th ground relay base station, t represents the number of historical transactions, and t=0 represents no historical transaction.

[0154] Furthermore, after the transaction is completed, the transaction reputation value can more intuitively display the service status of the ground relay base station in the transaction with the satellite, which affects the satellite's trust in the ground relay base station, as well as the size of the time factor allocated for the satellite's transaction with the ground relay base station in subsequent transactions. The satellite evaluates the service quality of the ground relay base station transaction by real-time monitoring of the service status of the ground relay base station, mainly considering whether the ground relay base station provides the transmission power as agreed in the transaction. The transmission power of the ground relay base station obtained by monitoring during the transaction is averaged and variance is taken, and the obtained transmission power is converted into a standard normal distribution to obtain the probability that the overall transmission power reaches the agreed transmission power. This probability is used as the transaction reputation value of the satellite to the ground relay base station in the current transaction, thereby intuitively evaluating the service status of the ground relay base station. The specific calculation process is as follows:

[0155] (1) Calculate the average transmission power of the ground relay base station

[0156] First, calculate the average transmission power of the ground relay base station obtained by monitoring during service. The calculation formula is as follows:

[0157]

[0158] in, is the average power transmitted by the ground relay base station i to the ground user of the trading satellite s, L is the number of times the transmission power of the ground relay base station is monitored, and P si (l) is the transmission power of the i-th ground relay base station serving the ground users of the trading satellite s obtained from the l-th monitoring.

[0159] (2) Calculate the variance of the transmission power sequence

[0160] The variance σ of the transmission power series is calculated by calculating the average transmission power si , calculated as follows:

[0161]

[0162] (3) Calculate the probability that the ground relay base station transmits power according to the transaction agreement

[0163] The transmission power of the monitored ground relay base station is converted into a standard normal distribution through the obtained mean and variance. The probability distribution function of the normal distribution is used to calculate the probability that the ground relay base station service reaches the transaction agreement: The calculation formula is as follows:

[0164]

[0165] in, The transmission power sequence of the monitored ground relay base station is converted into a normal distribution.

[0166] (4) Obtaining transaction reputation

[0167] The reputation value of the transaction satellite s for the ground relay base station is The calculation is as follows:

[0168]

[0169] (5) Update the credit value of the current transaction to obtain the credit value of the historical transaction.

[0170] After the transaction is completed, the satellite not only scores the service quality of the ground relay base station in the current transaction, but also updates the evaluation of the ground relay base station based on the historical transaction with the ground relay base station, and obtains the satellite's score evaluation of the relay before the next transaction. This evaluation can also be recommended to other satellites as a recommended reputation value with the ground relay base station. Before the first transaction between the satellite and the relay, the satellite gives the ground relay base station an initial reputation value evaluation, and the update method also uses EMA for update. The calculation method is as follows:

[0171]

[0172] Wherein, t represents the number of historical transactions, t=0 represents no historical transactions, and λ represents the exponential moving average update coefficient.

[0173] S2: Constructing a cognitive satellite-to-ground network spectrum trading game model according to the reputation value, wherein the game model includes a revenue optimization model for satellite users and a revenue optimization model for ground relay base stations.

[0174] This embodiment constructs a game model based on a cognitive satellite-to-ground network spectrum resource trading scenario. The scenario is constructed as a one-to-many Stackelberg game model, in which the satellite is the leader and the ground relay base station is the follower. In order to derive the equilibrium of the game, it is necessary to calculate the benefits of satellite users and each relay base station user separately. For satellite users, its main purpose is to improve the transmission rate of its own users. In most studies on assisted transmission, it is often believed that assisting ground relay base stations is selfless, that is, ground relay base stations will willingly participate in collaboration and assist in transmission without any incentives. However, this selfless behavior is impossible in real transactions, and ground relay base stations need to obtain corresponding rewards. The satellite purchases the power of the ground relay base station for cooperative transmission. Considering the lack of spectrum of the relay base station users, the satellite obtains the assistance of the ground relay base station in turn by allocating spectrum usage time. In order for satellite users to obtain higher transmission rates, the revenue optimization model for satellite users is constructed as follows:

[0175]

[0176] in, represents the transmission rate obtained by the satellite user, α i is the spectrum usage time factor allocated by the trading satellite to the i-th ground relay base station, represents the service power of the ground relay base station to the satellite user, B is the satellite spectrum bandwidth, T is the idle time of the satellite spectrum, N is the number of ground relay base stations participating in the transaction, SNR S is the signal-to-noise ratio of the signal received by the satellite user, It represents the transmission rate obtained by the relay base station user, and the constraint condition is that the sum of the spectrum usage time factors allocated by the satellite to all ground relay base stations is less than 1.

[0177] For each ground relay base station, the ground relay base station hopes to obtain more satellite spectrum usage time and leave more power to serve its own users. Therefore, the ground relay base station must not only consider the benefits of its own users, but also the consumption of assisting satellite users in transmitting and serving its own users. In order to ensure the enthusiasm of honest ground relay base stations to participate in transactions, the reputation value of the ground relay base station calculated in the previous article is added to the effect function as part of its benefits. Therefore, the revenue optimization model of the ground relay base station is constructed as follows:

[0178]

[0179] in, is the transmission rate obtained by the user of the i-th ground relay base station, Indicates the power of the ground relay base station to provide services to base station users. represents the channel coefficient from the i-th ground relay base station to the base station user, n0 is Gaussian white noise, C1 is the energy pricing of the ground relay base station assisting satellite users in transmission, C2 is the energy pricing of the ground relay base station serving its own users, C3 is the parameter of the reputation value affecting the transaction, Rep si (t) is the overall reputation value of the i-th ground relay base station calculated by the transaction satellite, P max is the maximum service power of the ground relay base station, and the constraint condition is that the power consumed by the ground relay base station is less than the power consumed in the second and third phases under the maximum power. In this embodiment, C1=5, C2=20, and C3=15.

[0180] S3: using the reverse induction method to solve the revenue optimization model of the ground relay base station, and obtain the optimal service power of the ground relay base station to the satellite user.

[0181] As mentioned above, the traditional solution to the one-to-many Stackelberg game problem uses reverse induction and iteration. The iteration method has a high actual time complexity and is prone to privacy leakage, affecting transaction fairness. The reverse induction method is difficult to solve, and both methods cannot cope with changes in channel states. This embodiment first uses the reverse induction method to solve the optimization problem of the ground relay base station, and then uses the deep reinforcement learning method to solve the optimization problem of the satellite user.

[0182] Specifically, the expression of the revenue optimization model of the ground relay base station is first inverted, and the transformed inequality constraint is Obtain the modified revenue optimization model:

[0183]

[0184] Use the Lagrange multiplier method to solve the optimization problem, introduce a slack variable v, and transform the constraints into equality constraints:

[0185]

[0186] According to the modified profit optimization model (Formula (26)), the Lagrangian function is constructed

[0187] Let the partial differential equations of the Lagrangian function be 0, and we can get:

[0188]

[0189] Solving the above formula, we get:

[0190]

[0191] According to the above formula, we can get λ′v=0, and discuss the following cases:

[0192] (1) When λ′=0, Constraints on the revenue optimization model for satellite users Simplifying, we can get λ′=C1. Since C1≠0, λ≠0.

[0193] (2) When υ = 0, substituting λ′ = C1 into the equation yields the optimal solution:

[0194]

[0195] The optimal strategy for ground relay base stations is thus solved That is, the optimal service power provided by the ground relay base station to satellite users.

[0196] S4: Using a deep reinforcement learning method (DQN) to solve the revenue optimization model of the satellite user, and obtain the optimal satellite spectrum time allocation factor.

[0197] Step S4 of this embodiment specifically includes:

[0198] S4.1: Before using deep reinforcement learning to solve the game model, the optimal service power of the ground relay base station to the satellite user Substituting the constraint condition into the satellite user's revenue optimization model, formula (24) is transformed into:

[0199]

[0200] S4.2: The satellite user is modeled as a single agent, and a centralized control center collects global information and controls each ground relay base station and satellite. The global information includes the overall reputation value calculated above and the optimal service power of each ground relay base station to the satellite user obtained by the reverse induction method. Figure 5 , Figure 5 It is a schematic diagram of a deep reinforcement learning network provided by an embodiment of the present invention.

[0201] S4.3: Construct a single-agent based deep reinforcement learning model, which includes state, action and reward functions.

[0202] 1) Status: Status s t ={α1,α2,...,α i ,...,α N} is the spectrum usage time allocated by the trading satellite to the ground relay base station, where α i is the time factor allocated by the satellite to each ground relay. DQN can obtain the user’s current state s t The vector of Q values ​​for all actions taken is as follows:

[0203] y=[Q(s t ,a 0 ),Q(s t ,a 1 ),...,Q(s t ,a i ),...,Q(s t ,a N )](33)

[0204] Among them, a i An action is selected.

[0205] 2) Action: In order to converge faster, the satellite is allowed to adjust the size of multiple time allocation factors in each action adjustment. The action taken by the satellite in step t is The action of the agent for each time allocation factor can be defined as When the time allocation factor α i = 0, only rising and unchanged actions can be selected. At the same time, the selected action is affected by the greedy strategy, and a greedy factor is set that decays with the number of training times. When the random number taken is greater than the decaying greedy factor, the optimal action is selected, otherwise the action is randomly selected, that is:

[0206]

[0207] Among them, random is the set random number, eplison is the decay greed factor, and random(a) is the randomly selected action.

[0208] 3) Reward function: In each action change, the reward function will encourage the agent to take actions that increase the benefits of satellite users. At the same time, actions that do not meet the constraints (I) and (II) in formula (32) will also be punished. The reward function is defined as:

[0209]

[0210] in, is the satellite user's profit function, U0=0, and β is the penalty factor. When the inequality constraints (I) and (II) of the game problem are not satisfied, the agent is punished. t Take action a t The long-term return G t It is the weighted sum of each current reward. The specific formula is:

[0211]

[0212] Among them, θ (0≤θ≤1) is the discount factor, which indicates the importance of the subsequent reward to the long-term return at the current moment. The smaller the θ value, the lower the importance.

[0213] S4.4: Use the single-agent based deep reinforcement learning model to solve the satellite user's revenue optimization model and obtain the optimal satellite spectrum time allocation factor.

[0214] Specifically, after starting training, the initially set time allocation factor is used as the initial state, and the Q values ​​of all actions in the current state action space are output. Then, according to the random number and greed factor, the action with the largest Q value or a random action is selected to execute, the current state is changed, the Q network is updated, the reward of the selected action is recorded, the rewards are accumulated and summed, and the experience is saved and put into the experience replay pool. The action is selected in the current state and the process is repeated until the number of steps reaches the set number of steps per round, and the target network is updated. Reset the environment state and repeat the previous process until the set number of training rounds is reached. The training ends and the optimal time allocation factor is output. The specific process is as follows:

[0215] Step 1: Set the number of training rounds train_eps = 1

[0216] Step 2: Set the initial spectrum time allocation factors α1 = 0.1, α2 = 0.1, α3 = 0.1, α4 = 0.1

[0217] Step 3: Set the reward per round and ep_reward = 0

[0218] Step 4: Set step=1, done=false

[0219] Step 5: Generate a random number

[0220] Step 6: If random>epsilon, select the action a with the largest Q value t ; otherwise choose a random action t

[0221] Step 7: Calculate the reward value of the current state next_state according to formula (35). Update the overall reward value ep_reward = reward + ep_reward.

[0222] Step 8: If the experience memory is less than the memory capacity of the experience pool, store (state, action, reward, next_state, done) into the experience pool, memory = memory + 1; otherwise, replace the old experience. Update the Q value network Q_net.

[0223] Step 9: If done≠true, step+1 and jump to step 5.

[0224] Step 10: If train_eps%target_update=0, update the target network target_net.

[0225] Step 11: train_eps=train_eps+1. If train_eps>total_eps, the algorithm ends, otherwise jump to step 2.

[0226] S5: Perform signal transmission according to the optimal satellite spectrum time allocation factor and the optimal service power for satellite users of the ground relay base station.

[0227] S6: After the current transaction is completed, the historical transaction reputation value is updated. Please refer to step S1 and formula (23) for the specific update process, which will not be repeated here.

[0228] The effect of the trusted spectrum trading method based on Stackelberg game in the cognitive satellite-to-ground network of the embodiment of the present invention is further explained below through simulation experiments.

[0229] See also Figure 6 and Figure 7 , Figure 6 and Figure 7 The scenario of the recommendation satellite recommending a ground relay base station was simulated. In order to prove that the recommendation mechanism can fight against malicious recommendations, reduce the weight of the recommendation satellite that cannot provide accurate recommendations, and accurately evaluate the reputation of multiple ground relay base stations, the scenario selects 4 recommendation satellites (1, 2, 3, 4) and recommends 4 ground relay base stations (A, B, C, D) that show different malicious behaviors in transactions. The malicious behavior probabilities of the 4 ground relay base stations are [0.4, 0.3, 0.2, 0.1]. The recommendation satellite recommends the ground relay base station 50 times, and the recommendation reputation values ​​of different ground relay base stations are [0.6, 0.7, 0.8, 0.9], the transaction times of different ground relay base stations are [6, 3, 4, 5], and the transaction time of different ground relay base stations is [4, 5, 4, 5]. The current time is 8. Recommendation satellite 3 will have malicious recommendation behavior in the 15th recommendation, and will resume normal recommendation in the 35th recommendation.

[0230] from Figure 6 It can be seen from the figure that after multiple recommendations, the recommendation weight is stable. The recommendation weights obtained by the recommended satellite using the same recommendation reputation value to recommend different ground relay base stations are different. The recommendation weight of the recommended satellite whose recommendation reputation value is close to the actual reputation value of the ground relay base station is high. Under the same recommendation reputation value, the recommendation weight of the recommended satellite with more transactions and closer transaction time with the ground relay base station is high. When the recommended satellite 3 has a malicious recommendation, the recommendation weight of the recommended satellite decreases rapidly, indicating that the recommendation reputation value mechanism designed in this paper can effectively prevent malicious recommendation behavior. Figure 7In the example, due to malicious behavior of the recommended satellite 3, the recommended reputation value when recommending different ground relay base stations is rapidly reduced, affecting the reliability of the recommended reputation value. The recommended reputation value mechanism designed in this embodiment quickly reduces the recommended weight of the recommended satellite and weakens its influence on the recommended reputation value calculation. After the recommended satellite 3 returns to normal, the recommended weight of the recommended satellite is quickly changed to ensure the stability of the recommended reputation value.

[0231] Figure 8 The scenario of the transaction satellite calculating the transaction reputation value of the ground relay base station with different transaction behaviors in the transaction is simulated. In order to illustrate the accuracy of the transaction reputation value designed in the embodiment of the present invention in describing the performance of the ground relay base station in the transaction, four ground relay base stations with different behaviors in the transaction are set, and the probability of malicious transactions is [0.75, 0.5, 0.25, 0], and the transaction is repeated 50 times. Figure 8 It can be seen that after 5 transactions, the satellite's reputation evaluation of the relay transaction performance is stable, and the transaction reputation value of the ground relay base station evaluated by the satellite is consistent with the behavior of the ground relay base station itself. Therefore, this embodiment is set to use the transaction reputation value to evaluate the ground relay base station completely after the satellite has traded with an unfamiliar ground relay base station 5 times.

[0232] See also Fig. 9 Fig. 9 A scenario was simulated to compare the transaction reputation value designed by the present invention with the method adopted by the existing method 1 (M. Zaeri-Amirani, F. Afghah and S. Zeadally, "A Hierarchical Spectrum Access Scheme for TV WhiteSpace Coexistence in Heterogeneous Networks," in IEEE Access, vol. 6, pp. 78992-79004, 2018.). In order to verify that the designed transaction reputation value can resist the ground relay base station of malicious transactions, the ground relay base station D set in the previous article was set for simulation, and the number of transactions was set to 50 times. In the 15th transaction, the ground relay base station D had an 80% probability of not transmitting power services in accordance with the transaction agreement. In the 30th transaction, the ground relay base station D resumed normal behavior. Existing method 1 uses a weighted summation method to process the transmission power obtained through monitoring. The purpose of this method is to reduce the impact of service failures in the transaction relay at the beginning of the transaction, but it also leads to fluctuations in the calculation of the transaction reputation value, and cannot accurately describe the transaction behavior of the ground relay base station D. From Fig. 9 It can be seen that the transaction reputation value used in the embodiment of the present invention can more accurately describe the behavior of the ground relay base station D and is more stable than the solution proposed in the existing method 1.

[0233] See also Fig.10 , Fig.10 A scenario was simulated in which the overall reputation value designed in the present invention was compared with the reputation value designed in the existing method 2 (Huang Yijie. Research on secure collaborative spectrum sensing based on reputation mechanism in cognitive radio networks [D]. Fujian Normal University, 2017.). In order to show the advantage of the overall reputation value designed in the present invention in dealing with malicious ground relay base stations, four ground relay base stations recommended by four recommended satellites were selected for verification, and a satellite that was unfamiliar with the four ground relay base stations traded with them. After the 20th transaction, the four ground relay base stations exhibited malicious behavior. Fig.10 It can be seen that when the satellite and the unfamiliar ground relay base station just started to trade, the reputation value calculation method designed by the present invention more accurately describes the reputation value of the ground relay base station than the method set in the existing method 2, which shows that the reputation value method designed by the present invention better solves the cold start problem between the satellite and the unfamiliar ground relay base station. As the transaction proceeds, the transaction reputation value designed by the present invention accurately calculates the transaction reputation value of the ground relay base station, and the overall reputation value can accurately describe the behavior of the ground relay base station. After the 20th transaction, the ground relay base station involved in the transaction exhibited malicious behavior. The overall reputation values ​​designed in the present invention and the existing method 2 can quickly reduce the evaluation of the relay and accurately describe the malicious behavior of the ground relay base station. After the 30th transaction, the ground relay base station returned to normal and the overall reputation value recovered quickly.

[0234] See also Fig.11 and Fig.12 , Fig.11 and Fig.12 The influence of the change of the reputation value of the ground relay base station in the transaction on the satellite user's income and the income of other honest ground relay base stations is simulated. In order to describe the influence of the reputation value on the satellite user's income, the satellite conducts transactions with four ground relay base stations, one of which, the reputation value of the ground relay base station A, changes from 0 to 1, and the reputation values ​​of the other ground relay base stations are set as above. As shown in the figure, as the reputation value of the ground relay base station A increases, the spectrum usage time allocated to the ground relay base station increases. Therefore, the incentive mechanism designed by the present invention can effectively incentivize the ground relay base station to trade honestly and encourage the ground relay base station to serve satellite users. The ground relay base station A has obtained more benefits because of its honest transactions. In order to reward the trustworthy behavior of the relay base station, the satellite user chooses to allocate more spectrum usage time to the ground relay base station A.

[0235] See also Fig.13 , Fig.13The scenario of a satellite trading with four ground relay base stations was simulated, and a comparison was made between the reputation value and the one without reputation value. In order to illustrate the impact of reputation value on transactions, the satellite traded with four ground relay base stations, one of which had a different reputation value, while the other remained unchanged. The transaction results with and without reputation value were compared. As shown in the figure, when the reputation value was not considered, ground relay base station D obtained more spectrum usage time and higher revenue due to good channel conditions. As the reputation value of relay base station D decreased, the time allocation factor allocated to relay base station D remained unchanged, so the revenue of the ground relay base station increased instead, which greatly reduced the enthusiasm of other ground relay base stations to conduct honest transactions. After considering the reputation value, the time allocation factor of ground relay base station D decreased significantly as its reputation value decreased, and its revenue also decreased rapidly. By introducing the reputation value model into transactions, the revenue of satellite users is increased, while the revenue of malicious ground relay base stations with low reputation values ​​is reduced, encouraging ground relay base stations to participate in transactions honestly and ensuring the enthusiasm of satellites to participate in transactions.

[0236] See also Fig.14 , Fig.14 It is a comparison chart of the benefits of satellite users obtained by using the method of the embodiment of the present invention and the existing iterative method as the number of training times increases. Fig.14 It can be seen that as the training continues, the benefits of satellite users continue to increase and finally stabilize. At the beginning of training, the DQN network used in the present invention continuously records the benefits of actions selected in different states, and conducts extensive exploration accompanied by a continuously decaying greed factor, learning to select the optimal action according to the state of the satellite user. When the above learning is completed, DQN tends to converge, and deep reinforcement learning has also obtained the optimal route, so the benefits of satellite users tend to stabilize. It can be observed that compared with the results of the existing iterative method, the benefits obtained by the convergence of the method based on deep reinforcement learning of the present invention are 4.7% lower. Although the solution results of reinforcement learning are slightly worse than those of the iterative method, the privacy of the transaction is guaranteed and the time for online solution is saved.

[0237] The present invention provides a trusted spectrum trading method based on Stackelberg game under cognitive satellite-to-ground network. First, a one-to-many Stackelberg game model is established in the cognitive satellite-to-ground network spectrum trading scenario to allow more ground relay base stations to participate in the transaction and solve the spectrum shortage problem; a ground relay base station reputation model is established according to the transaction model, and other satellite recommendation reputation values ​​and satellite historical transaction reputation values ​​are considered to solve the impact of malicious ground relay base stations on satellite revenue, while encouraging ground relay base stations to actively participate in transactions; finally, reverse induction and deep reinforcement learning are used to solve the game model to avoid privacy leakage and reduce the difficulty of solving. The reputation model constructed by the present invention modifies the weighting method of the recommended reputation value and the transaction reputation value, modifies the recommended reputation value calculation method, adds the recommendation accuracy and the recommender's recent transaction time, accurately describes the behavior of malicious ground relay base stations, and gives a more accurate evaluation. The present invention uses deep reinforcement learning to solve the problem of unfair transactions caused by the difficulty of solving traditional solution optimization algorithms and privacy leakage, protects the privacy of ground relay base stations, and ensures fair transactions.

[0238] Another embodiment of the present invention provides a storage medium, in which a computer program is stored, and the computer program is used to execute the steps of the trusted spectrum trading method based on Stackelberg game under the cognitive satellite-to-ground network described in the above embodiment. Another aspect of the present invention provides an electronic device, including a memory and a processor, in which a computer program is stored, and when the processor calls the computer program in the memory, the steps of the trusted spectrum trading method based on Stackelberg game under the cognitive satellite-to-ground network described in the above embodiment are implemented. Specifically, the above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including several instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0239] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A trusted spectrum trading method based on Stackelberg game in a cognitive satellite-to-ground network, characterized in that: include: S1: constructing a reputation value model based on cognitive satellite-to-ground network spectrum trading, and obtaining the reputation value of each ground relay base station according to the reputation value model; including: S1.1: Obtain the recommended reputation value of each ground relay base station; S1.2: Obtain the historical transaction reputation value of each ground relay base station; S1.3: Obtaining the overall reputation value of each ground relay base station according to the recommended reputation value and the historical transaction reputation value: Rep si (t)=βRept si (t)+(1-β)Rep rec-i (t), Among them, Rep. si (t) represents the overall reputation value of the ground relay base station i obtained by trading satellite s before the current transaction, Rept si (t) represents the historical transaction reputation value of the transaction satellite s to the ground relay base station i, Rep rec-i (t) represents the recommended reputation value of other satellites to the ground relay base station i, β is the weighting factor of the historical transaction reputation value of the transaction satellite s, and the expression is: Among them, t N represents the historical transaction times between the transaction satellite s and the ground relay base station i, μ represents the weighting factor coefficient; S2: constructing a cognitive satellite-to-ground network spectrum trading game model according to the reputation value, wherein the game model includes a revenue optimization model for satellite users and a revenue optimization model for ground relay base stations; S3: solving the revenue optimization model of the ground relay base station by using the reverse induction method to obtain the optimal service power of the satellite users of the ground relay base station; S4: solving the revenue optimization model of the satellite user by using a deep reinforcement learning method to obtain an optimal satellite spectrum time allocation factor; S5: performing signal transmission according to the optimal satellite spectrum time allocation factor and the optimal service power of the satellite user of the ground relay base station; S6: After the current transaction is completed, the historical transaction reputation value is updated.

2. The trusted spectrum trading method based on Stackelberg game in the cognitive satellite-to-ground network according to claim 1 is characterized in that: The expression of the recommended reputation value is: Among them, η ji (t) represents the recommendation weight of the jth recommended satellite to the ith ground relay base station, M is the number of recommended satellites, Rep ji (t) represents the historical transaction reputation value between the j-th recommended satellite and the i-th ground relay base station; The expression of the recommendation weight is: Among them, ρ ji represents the historical transaction times between the jth recommended satellite and the ith ground relay base station, ω ji (t) represents the historical recommendation accuracy of the jth recommended satellite to the ith ground relay base station, γ is the transaction time coefficient, t x is the current time, t ji Indicates the most recent transaction time between the j-th recommended satellite and the i-th ground relay base station.

3. The trusted spectrum trading method based on Stackelberg game in the cognitive satellite-to-ground network according to claim 2 is characterized in that: The calculation process of the accuracy of the historical recommendations includes: Calculate the accuracy of recommendations in the current transaction: in, Rep represents the accuracy of the recommendation opinion of satellite j for the i-th ground relay base station in the current transaction, si (t) is the transaction reputation value of the transaction satellite s to the i-th ground relay base station in the current transaction, Rep ji (t) is the historical transaction reputation value between the jth recommended satellite and the ith ground relay base station, and m is the coefficient for evaluating the accuracy of the recommendation opinion; The accuracy of the recommendations in the current transaction is used to update the accuracy of the historical recommendations: Among them, λ is the exponential moving average update coefficient, ω ji (t-1) is the accuracy of the recommendation opinion of the historical transaction recommendation satellite to the i-th ground relay base station, t represents the number of historical transactions, and t=0 represents no historical transaction.

4. The trusted spectrum trading method based on Stackelberg game in the cognitive satellite-to-ground network according to claim 1 is characterized in that: The S1.2 includes: S1.21: Calculate the average transmission power of the ground relay base station: in, is the average power transmitted by the ground relay base station i to the ground user of the trading satellite s, L is the number of times the transmission power of the ground relay base station is monitored, and P si (l) is the transmission power of the i-th ground relay base station serving the ground users of the trading satellite s obtained by the l-th monitoring; S1.22: Calculate the variance of the transmission power of the ground relay base station: S1.23: Calculate the probability that the ground relay base station transmits power according to the transaction agreement: in, represents the transmission power of the i-th ground relay base station; S1.24: Obtain the credit value of this transaction S1.25: Update the reputation value of this transaction to obtain the historical transaction reputation value: Wherein, t represents the number of historical transactions, t=0 represents no historical transactions, and λ represents the exponential moving average update coefficient.

5. The trusted spectrum trading method based on Stackelberg game in the cognitive satellite-to-ground network according to claim 1 is characterized in that: The expression of the satellite user's revenue optimization model is: in, represents the transmission rate obtained by the satellite user, α i is the spectrum usage time factor allocated by the trading satellite to the i-th ground relay base station, represents the service power of the ground relay base station to the satellite user, B is the satellite spectrum bandwidth, T is the idle time of the satellite spectrum, N is the number of ground relay base stations participating in the transaction, SNR S is the signal-to-noise ratio of the signal received by the satellite user, Indicates the transmission rate obtained by the relay base station user; The expression of the revenue optimization model of the ground relay base station is: in, is the transmission rate obtained by the user of the i-th ground relay base station, Indicates the power of the ground relay base station to provide services to base station users. represents the channel coefficient from the i-th ground relay base station to the base station user, n0 is Gaussian white noise, C1 is the energy pricing of the ground relay base station assisting satellite users in transmission, C2 is the energy pricing of the ground relay base station serving its own users, C3 is the parameter of the reputation value affecting the transaction, Rep si (t) is the overall reputation value of the i-th ground relay base station calculated by the transaction satellite, P max It is the maximum service power of the ground relay base station.

6. The trusted spectrum trading method based on Stackelberg game in the cognitive satellite-to-ground network according to claim 5 is characterized in that: The S3 includes: S3.1: Reverse the expression of the revenue optimization model of the ground relay base station to obtain a modified revenue optimization model: S3.2: Introduce the slack variable υ and transform the constraints into equality constraints: S3.3: Constructing Lagrangian function using the modified revenue optimization model S3.4: Set the partial differential equations of the Lagrangian function to 0, and calculate the optimal service power for satellite users of the ground relay base station.

7. The trusted spectrum trading method based on Stackelberg game in the cognitive satellite-to-ground network according to claim 6 is characterized in that: The S4 includes: S4.1: Optimal service power for satellite users at ground relay base stations Substituting the constraints into the satellite user's revenue optimization model, we get: Among them, n0 is Gaussian white noise; S4.2: The satellite user is modeled as a single agent, and a centralized control center collects global information and controls the satellite and each ground relay base station, wherein the global information includes the calculated overall reputation value and the optimal service power of each ground relay base station to the satellite user; S4.3: Build a single-agent based deep reinforcement learning model; S4.4: Use the single-agent based deep reinforcement learning model to solve the satellite user's revenue optimization model and obtain the optimal satellite spectrum time allocation factor.

8. The trusted spectrum trading method based on Stackelberg game in the cognitive satellite-to-ground network according to claim 7 is characterized in that: The single-agent based deep reinforcement learning model includes state, action and reward functions, wherein: The status t ={α1,α2,...,α i ,...,α N } is the spectrum usage time allocated by the trading satellite to the ground relay base station, α i is the time factor allocated by the satellite to each ground relay. The user is in the current state s t The vector composed of the Q values ​​of all actions is: y=[Q(s t ,a 0 ),Q(s t ,a 1 ),...,Q(s t ,a i ),...,Q(s t ,a N )], Among them, a i is an action selected; The actions are: Among them, random is the set random number, eplison is the decay greed factor, a is the action, and random(a) is the randomly selected action; The reward function is: in, β′ is the penalty factor.

9. The trusted spectrum trading method based on Stackelberg game in the cognitive satellite-to-ground network according to claim 8 is characterized in that: The S4.3 includes: Take the initially set time allocation factor as the initial state, and output the Q values ​​of all actions in the current state action space; select the action with the largest Q value or a random action according to the random number and greed factor, change the current state, update the Q network, record the reward of the selected action, accumulate and sum the rewards, and save the experience into the experience replay pool; Continue to select actions in the current state until the number of steps reaches the set number of steps per round, and update the target network; Reset the environment state and repeat the steps until the set number of training rounds is reached, and output the optimal satellite spectrum time allocation factor.

Images