Multi-user access channel method and system based on Thompson sampling for 5G communication system

Abstract

The invention discloses a 5G communication system-oriented multi-user channel access method and system based on Thompson sampling. The method comprises the following steps of: trying to access a channel by a user; initialization: distributing a unique index number for a user and initializing a channel reward parameter model; selecting a channel: performing distributed channel selection by a user through an LBT (Listen Before Talk) mechanism; accessing a channel: selecting a channel of a utility value calculated by the award sampling value of the corresponding index; updating model parameters: updating the parameters according to a transmission result and calculating the network throughput; the system comprises an initialization module, a selection channel module, an access channel module and an optimization module. The multi-arm machine model adopting reinforcement learning can dynamically make a decision according to a real-time environment and a network state, a strategy can be autonomously explored and adjusted in the learning process to continuously optimize selection of a transmission rate, a channel is selected through a kth large value, resource allocation is optimized, and therefore the overall network capacity is improved.

Description

Technical Field

[0001] The present invention relates to the field of multi-user access channel selection under 5G wireless channels, and in particular to a multi-user access channel method and system based on Thompson sampling for a 5G communication system. Background Art

[0002] Channel is the passage for data transmission. Networks can be divided into single channel and multi-channel according to the number of channels used for data transmission. Single channel scheduling only transmits data at a fixed frequency, and the algorithm design is relatively simple. However, in the face of large-scale networks, a single channel cannot handle scheduling in a timely manner, and conflicts are prone to occur. In order to avoid conflicts when transmitting data, multi-channel scheduling allows nodes to select one channel from multiple channels in the same time slot for transmission. For example, the IEEE 802.15.4 protocol can use 16 channels for scheduling at the physical layer.

[0003] With the continued rapid growth of mobile Internet and the Internet of Things, huge traffic has posed a major challenge to network capacity. The dynamic and uncertain nature of unlicensed spectrum makes it difficult for users to grasp the channel status in a timely manner, making it impossible to make the best access choice. In addition, the limited spectrum resources intensify the competition among multiple users for channels, making access efficiency low and affecting the overall network capacity. When multiple users try to access the same channel at the same time, it is easy to cause serious interference and network congestion. Interference between users will not only lead to a decrease in transmission rate, but also affect the stable transmission of data and reduce user experience. In high-density multi-user scenarios, due to fierce competition for spectrum resources, some users may not be able to access the channel for a long time, resulting in a decrease in the quality of data transmission service and affecting the user experience.

[0004] In the prior art, there are various methods to improve spectrum utilization. Among them, based on the LBT mechanism, the network environment information is optimized by adjusting the size of the contention window, which improves the network access performance and user experience. The exponential backoff algorithm can also be improved according to the transmission feedback, reducing the retransmission waiting time and the probability of data collision, and enhancing the spectrum utilization. In addition to optimizing the LBT mechanism, the researchers also analyzed the performance of unlicensed networks. The Markov chain model was used to analyze the network capacity and delay to optimize the channel resource allocation. Among them, although the exponential backoff algorithm optimizes the backoff mechanism to reduce the waiting time and the probability of collision, in high-density user scenarios, the backoff mechanism may cause too much retransmission waiting, prolong the access time, and reduce the performance of real-time applications. In addition, the Markov chain model requires a high computational overhead for the analysis of channel resource allocation. In complex wireless environments, the accuracy and applicability of the model may be limited, which is particularly evident in edge devices or real-time systems with limited resources. The comprehensive application of the above technologies only involves centralized channel access management in a static environment, but ignores the impact of dynamic time-varying environments on access strategies. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a multi-user access channel method and system based on Thompson sampling for 5G communication systems, so as to cope with the impact of conflicts when multiple users access unlicensed frequency bands.

[0006] Technical solution: The multi-user access channel method based on Thompson sampling for 5G communication system described in the present invention comprises the following steps:

[0007] (1) A user attempts to access a channel;

[0008] (2) Initialization: assign a unique index number to the user and initialize the channel reward parameter model;

[0009] (3) Channel selection: Users select channels in a distributed manner through the LBT mechanism;

[0010] (4) Access channel: select the channel with the utility value calculated by the reward sampling value corresponding to the index;

[0011] (5) Update model parameters: Update parameters based on the transmission results and calculate the network throughput.

[0012] Furthermore, the step (1) includes assuming that each NR-U user can only select one channel for monitoring and transmission in each decision time slot. At the beginning of the decision time slot, the user accesses the channel according to the selected strategy and occupies the channel through the LBT mechanism to complete data transmission. After the decision time slot ends, the user will collect relevant transmission information and use the learning algorithm to update its channel selection strategy.

[0013] Furthermore, the step (2) includes: for the dynamic access system under consideration, defining a n Channel selection for NUE n, To select the set of users for channel m, the normalized transmission rate of user n in the decision time slot t is given by:

[0014]

[0015] Among them, r mn (t) is the instantaneous transmission rate of user equipment n associated with channel m during decision hour interval t; and The transmission efficiency function of NUE is as follows: The time for successful transmission of the time slice

[0016]

[0017] in represents the number of NUE on channel m, p fis the probability that the channel is idle, is the probability that the channel is occupied by NUE successfully transmitting, is the probability that the channel is occupied by NUE transmission conflicts, is the duration of successful NUE transmission, T θ is the duration of the idle time slot, δ and DIFS are other time parameters,

[0018] The channel reward parameter model initialization is expressed as:

[0019] α m，n =1,β m，n =1, c m,n =0

[0020] Where m is the gNB broadcast channel index and n is the number of users.

[0021] Furthermore, the step (3) of distributed channel selection includes:

[0022]

[0023] Where a=(a1,a2,......,a n ) is the channel selection of NUE, I(a j , a n ) indicates whether the user selects the same channel as user n,

[0024] The expected rate at which user n chooses m is expressed as:

[0025]

[0026] Furthermore, the channel representation of the utility value in step (4) is:

[0027] The rate set on channel m is denoted as R m = {r m1 , r m2 ,......,r mK}, the corresponding rate state probability is P m ={p m1 , p m2 ,......,p mK Then the expected utility of user n is expressed as:

[0028]

[0029] For each user l, sample the number of users k in each channel from its reward distribution and select the lth largest normalized utility The transmission is carried out through the channel, where

[0030] Furthermore, the network throughput of step (5) is expressed as:

[0031]

[0032] Where a=(a1,a2,......,a n ) is the channel selection of NUE, I(a j , a n ) indicates whether the user selects the same channel as user n,

[0033]

[0034] The goal of the system is to find the optimal selection vector a opt To maximize the system capacity,

[0035] a opt = argmaxU s (a).

[0036] Furthermore, the optimal selection vector a opt The search to maximize system capacity includes the following steps:

[0037] The properties of the study problem, where the channel selection game in a dynamic environment can be expressed as:

[0038]

[0039] Where NUE n∈N is the player selected in the non-cooperative game, and the strategy space of each user can be expressed as And μ n is the utility of user n, defined as its throughput,

[0040] The utility of user n in the decision hour interval t can be expressed as:

[0041]

[0042] All channels provide the same bandwidth and the same transmission rate to each user, while a finite rate channel model is used to represent the temporal fluctuations, where the achievable transmission rate at each decision hour interval can only be selected from the relevant set.

[0043] The multi-user access channel system based on Thompson sampling for a 5G communication system according to the present invention comprises:

[0044] An initialization module, used to assign a unique index number to the user and initialize the channel reward parameter model;

[0045] Channel selection module, used by users to select channels in a distributed manner through the LBT mechanism;

[0046] An access channel module, used to select a channel corresponding to the utility value calculated by the reward sampling value of the index;

[0047] The optimization module is used to update parameters and calculate network throughput according to the transmission results.

[0048] Furthermore, when the processor executes the program, a multi-user access channel method based on Thompson sampling for a 5G communication system is implemented.

[0049] Furthermore, the computer program is designed to implement a multi-user access channel method based on Thompson sampling for a 5G communication system when running.

[0050] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: for unknown time-varying channel environment constraints, the multi-armed tiger machine model of reinforcement learning can dynamically make decisions according to the real-time environment and network status, and can autonomously explore and adjust strategies to continuously optimize the selection of transmission rate during the learning process. In an unknown time-varying channel environment, multiple users can select channels through the kth maximum value based on the distributed learning strategy of multi-arm theory, optimize resource allocation and thus improve the overall network capacity. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of the present invention;

[0052] Figure 2 It is a diagram of the network structure of multi-user equipment;

[0053] Figure 3 This is the channel time slot division diagram. DETAILED DESCRIPTION

[0054] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0055] like Figure 1 As shown, the multi-user access channel method based on Thompson sampling for a 5G communication system according to the present invention comprises the following steps:

[0056] (1) A user attempts to access a channel;

[0057] (2) Initialization: assign a unique index number to the user and initialize the channel reward parameter model;

[0058] (3) Channel selection: Users select channels in a distributed manner through the LBT mechanism;

[0059] (4) Access channel: select the channel with the utility value calculated by the reward sampling value corresponding to the index;

[0060] (5) Update model parameters: Update parameters based on the transmission results and calculate the network throughput.

[0061] The algorithm flow is shown in Table 1. Assuming that the player can know the number of users to select a channel by monitoring the channel, each user runs a distributed algorithm to select a channel and exchange information in the access phase, which can be regarded as the cost of the algorithm. It should be noted that under the constraints of the Thompson algorithm, The value of is supported within [0, 1], so Defined as normalized transfer rate, in ascending order is θ1, θ2, ..., θ k As shown in the algorithm, the user stores two 1×MN vectors, namely and If the user plays the Kth arm, he will choose the corresponding channel In order to clearly evaluate the system performance under strategy π, the regret loss at time n is defined as:

[0062]

[0063] Specific process: Each user is assigned a unique number. At the beginning of each time step t, all users will select channels and sample the probability modulus according to the Thompson Sampling strategy. The sampling value reflects the user's estimate of the success rate of transmission on the channel. The higher the value, the more likely the success rate of the channel is under the current conditions. Each channel is sorted according to the calculated utility value so that users can select the channel with the corresponding sequence number. Then the LBT (Listen-Before-Talk) mechanism is used to check whether the channel is idle. If the channel is idle, the user can transmit immediately. If multiple users select the same channel, it will cause a conflict. Only one user can transmit successfully, and the other users will fail to transmit. The user who successfully transmits sends data on the channel, and the unsuccessful user reselects the channel in the next time step. After each time step, the user updates the probability distribution parameters of the channel based on the transmission results:

[0064] Table 1: Algorithm flow of the multi-user access channel mechanism based on Thompson sampling implemented in this scheme

[0065]

[0066]

[0067] When users choose to access the same unlicensed spectrum, they need to use the LBT mechanism to seize the channel for data transmission. Figure 3When a user successfully accesses the channel according to the LBT mechanism, the channel enters the Figure 3 The Transmission time shown has a maximum transmission time limit to ensure that other users have a chance to use the channel. When the user's transmission time on the channel reaches the specified maximum channel occupancy time, the channel will be actively released, allowing other users to have a chance to successfully access.

[0068] When a base station attempts to access an unlicensed channel for data transmission, it must first perform a clear channel assessment (CCA) and use the energy detection (ED) mechanism to determine whether there are other user signals on the channel, thereby determining whether the channel is idle and available. If the energy received by the base station is greater than the standard threshold, it is considered that the channel is temporarily occupied by other users and the base station needs to continue waiting.

[0069] like Figure 2 As shown in the figure, specify multiple user tags (A to F) and follow the steps below: Figure 3 The user time slots shown are divided into time slots. When an example is used to illustrate the algorithm later, refer to Figure 2 , Figure 3 .

[0070] like Figure 2 In the multi-user multi-channel network shown, each user (A~F) can choose one of the three unlicensed channels (C1, C2 and C3) for data transmission. This channel selection is based on a distributed learning algorithm. In the initial stage of the algorithm, each user is randomly assigned a unique number l, and the reward parameters (α and β) of the channel are initialized. Represents the initial transmission effect of each channel. In each time slot t, users A to F use the LBT mechanism to select channels respectively. For example, user A estimates the effect of each channel by sampling from the reward distribution of channels C1, C2 and C3, and selects the channel with the lth largest normalized utility after calculating the sampling results. (e.g. C2 blue channel) for transmission. Other users such as B and C will perform the same operation. Each user will select the channel with the lth largest normalized utility corresponding to its own number to transmit. After the transmission is completed, the base station will feedback the transmission results of each user. If the transmission is successful, the user updates the α parameter of the channel by 1; if it fails, the β parameter is updated by 1. The user stores two 1×MN vectors: and At the end, the user will update the number of channel selections. In this way, the user gradually has a more accurate estimate of the transmission effect of each channel. Over time, the user's selection of different channels tends to be stable by updating the α and β parameters. In the frequent transmission process, the user gradually learns to select the most suitable channel for transmission, thereby maximizing network capacity and efficiently utilizing the spectrum.

[0071] The multi-user access channel system based on Thompson sampling for a 5G communication system according to the present invention is characterized by comprising:

[0072] An initialization module, used to assign a unique index number to the user and initialize the channel reward parameter model;

[0073] Channel selection module, used by users to select channels in a distributed manner through the LBT mechanism;

[0074] An access channel module, used to select a channel corresponding to the utility value calculated by the reward sampling value of the index;

[0075] The optimization module is used to update parameters and calculate network throughput according to the transmission results.

Claims

1. A multi-user access channel method based on Thompson sampling for 5G communication system, characterized in that: The steps include: (1) A user attempts to access a channel; (2) Initialization: assign a unique index number to the user and initialize the channel reward parameter model; (3) Channel selection: Users select channels in a distributed manner through the LBT mechanism; (4) Access channel: select the channel with the utility value calculated by the reward sampling value corresponding to the index; (5) Update model parameters: Update parameters based on the transmission results and calculate the network throughput.

2. The multi-user access channel method based on Thompson sampling for 5G communication system according to claim 1, characterized in that: The step (1) includes assuming that each NR-U user can only select one channel for monitoring and transmission in each decision time slot. At the beginning of the decision time slot, the user accesses the channel according to the selected strategy and occupies the channel through the LBT mechanism to complete data transmission. After the decision time slot ends, the user will collect relevant transmission information and use the learning algorithm to update its channel selection strategy.

3. The multi-user access channel method based on Thompson sampling for 5G communication system according to claim 1, characterized in that: The step (2) comprises: for the dynamic access system under consideration, defining a n Channel selection for NUE n, To select the set of users for channel m, the normalized transmission rate of user n in the decision time slot t is given by: Among them, r mn (t) is the instantaneous transmission rate of user equipment n associated with channel m during decision hour interval t; and The transmission efficiency function of NUE is as follows: The time for successful transmission of the time slice in represents the number of NUE on channel m, p f is the probability that the channel is idle, is the probability that the channel is occupied by NUE successfully transmitting, is the probability that the channel is occupied by NUE transmission conflicts, is the duration of successful NUE transmission, T θ is the duration of the idle time slot, δ and DIFS are other time parameters, The channel reward parameter model initialization is expressed as: α m，n ＝1，β m，n ＝1，c m,n ＝0 Where m is the gNB broadcast channel index and n is the number of users.

4. The multi-user access channel method based on Thompson sampling for 5G communication system according to claim 1, characterized in that: The step (3) of distributed channel selection comprises: Where a=(a1,a2,......,a n ) is the channel selection of NUE, I(a j , a n ) indicates whether the user selects the same channel as user n, The expected rate at which user n chooses m is expressed as:

5. The multi-user access channel method based on Thompson sampling for 5G communication system according to claim 1, characterized in that: The channel representation of the utility value in step (4) is: The rate set on channel m is denoted as R m = {r m1 , r m2 ,......,r mK }, the corresponding rate state probability is P m ={p m1 ,p m2 ,......,p mK Then the expected utility of user n is expressed as: For each user l, sample the number of users k in each channel from its reward distribution and select the lth largest normalized utility The transmission is carried out through the channel, where 6. The multi-user access channel method based on Thompson sampling for 5G communication system according to claim 1, characterized in that: The network throughput of step (5) is expressed as: Where a=(a1,a2,......,a n ) is the channel selection of NUE, I(a j , a n ) indicates whether the user selects the same channel as user n, The goal of the system is to find the optimal selection vector a opt To maximize the system capacity, a opt =argmaxU s (a)。 7. The multi-user access channel method based on Thompson sampling for 5G communication system according to claim 6, characterized in that: The optimal selection vector α opt The search to maximize system capacity includes the following steps: The properties of the study problem, where the channel selection game in a dynamic environment can be expressed as: Where NUE n∈N is the player selected in the non-cooperative game, and the strategy space of each user can be expressed as And μ n is the utility of user n, defined as its throughput, The utility of user n in the decision hour interval t can be expressed as: All channels provide the same bandwidth and the same transmission rate to each user, while a finite rate channel model is used to represent the temporal fluctuations, where the achievable transmission rate at each decision hour interval can only be selected from the relevant set.

8. A multi-user access channel system based on Thompson sampling for 5G communication system, characterized in that: include: An initialization module, used to assign a unique index number to the user and initialize the channel reward parameter model; Channel selection module, used for users to select channels in a distributed manner through the LBT mechanism; An access channel module, used to select a channel corresponding to the utility value calculated by the reward sampling value of the index; The optimization module is used to update parameters and calculate network throughput according to the transmission results.

9. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the multi-user access channel method based on Thompson sampling for 5G communication systems according to any one of claims 1-7.

10. A storage medium storing a computer program, characterized in that: The computer program is designed to implement the Thompson sampling-based multi-user access channel method for a 5G communication system according to any one of claims 1 to 7 when running.

Images