Gibbs sampling heuristic covert communication dynamic spectrum access method

Through the combination of Gibbs sampling-inspired rejection mechanism and Q-Learning algorithm, a DSA strategy based on probability rejection is built, which solves the problem of users being easily attacked in the existing methods and achieves better spectrum utilization and hidden communication effects.

CN120568336AActive Publication Date: 2025-08-29TONGJI UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202510840190.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-06-23
Publication Date
2025-08-29
Estimated Expiration
2045-06-23

AI Technical Summary

Technical Problem

The existing dynamic spectrum access method based on reinforcement learning weighs the trade-off between adaptability, exploration efficiency and communication overhead, and ignores the importance of improving users' hidden communication capabilities, making users vulnerable to attackers to detect and track.

Method used

Using Gibbs sampling-inspired rejection mechanism, the overused channel is accessed through probabilistic rejection, combined with the Q-Learning algorithm, a DSA strategy based on probability rejection is constructed to promote the diversity and concealment of channel selection.

Benefits of technology

It reduces the risk of users being detected by attackers, improves the balance of spectrum utilization and covert communication performance, reduces the probability of long-term access of users on the same channel, and enhances the user's covert communication capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120568336A_ABST
    Figure CN120568336A_ABST
Patent Text Reader

Abstract

In the invention, a dynamic spectrum access (DSA) method for covert communication, which is inspired by Gibbs sampling, is provided. The method comprises the following steps: firstly, defining a covert communication index as KL divergence between user channel access distribution and uniform distribution; the lower the KL divergence is, the higher the difficulty of detecting or attacking the transmission channel by the attacker is. Secondly, inspired by Gibbs sampling, a rejection mechanism for rejecting access to an overused channel is provided; through the mode, the user is promoted to dynamically access a plurality of channels, and the probability that the user continuously accesses the same channel is reduced. And finally, combining a rejection mechanism with a Q-Learning reinforcement learning algorithm to form a new aperiodic frequency hopping DSA framework for covert communication. And the channel selection is continuously adjusted, so that the covert communication performance is improved. A simulation result shows that compared with a standard Q-Learning algorithm, under different user densities, the provided frequency hopping DSA method has the advantages that the KL divergence is remarkably reduced and the covert communication performance is enhanced while the good channel capacity is maintained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of wireless communications and networks, and particularly relates to cognitive radio technology based on reinforcement learning. Background Art

[0002] Global wireless technology usage continues to grow rapidly. This unprecedented surge in wireless connectivity presents new challenges for ensuring secure and private data transmission. Due to the open and broadcast nature of wireless channels, communication systems are inherently vulnerable to various forms of interference and malicious attacks. In particular, attackers can eavesdrop on ongoing transmissions or disrupt them by transmitting high-power signals into active communication channels. Such attacks can severely degrade the signal-to-interference-and-noise ratio (SINR) of legitimate signals, resulting in data failure or incomprehensibility at the receiving end. These risks highlight the growing need for covert communication technologies to reduce the likelihood of detection or attack during spectrum access. This issue is particularly acute in sectors with high communication security requirements, such as military operations, financial systems, and government communications, where ensuring data integrity, confidentiality, and availability are paramount. Therefore, protecting wireless transmissions from attackers—including those seeking to eavesdrop, inject interference, or tamper with signals—remains a fundamental challenge in wireless security. With the development of extremely complex radio services, the need for adaptive, intelligent, and covert communication technologies is more pressing than ever.

[0003] Frequency hopping (FH) technology has been widely adopted to reduce the risk of attack. A FH system first communicates with the receiver to predefine a channel switching pattern, allowing users to transmit on rapidly changing channels, thereby reducing the possibility of an attacker continuously disrupting transmissions on a specific frequency band. FH systems have proven effective against non-adaptive attackers, who detect and attack transmission channels based on static or preset patterns without real-time feedback from the radio environment. However, with the advancement of spectrum sensing and adaptive signal processing technologies, attackers have developed adaptive attack strategies that can detect, track, and adapt to the channel selection strategies of target users. Due to the periodic nature of FH system channel selection, they generally lack the sensitivity required to respond to rapidly changing radio environments.

[0004] At the same time, the complexity and variability of the radio environment have also promoted the development of dynamic spectrum access (DSA) technology. DSA enables users to observe and respond to changes in the radio environment in real time, promoting efficient and flexible spectrum utilization. Although DSA was originally designed to alleviate spectrum shortages, its core advantages—namely, spectrum sensing and adaptive decision-making—make it particularly suitable for covert communications. Among various DSA methods, reinforcement learning (RL) has become one of the most effective and widely adopted solutions. RL-based DSA users can autonomously track optimal channel selection strategies by continuously interacting with the radio environment and learning from feedback. In recent years, the integration of covert communication and intelligent learning has become a hot topic. Many studies have explored how to use RL to enhance the performance of covert communications in dynamic radio environments.

[0005] However, existing methods often face trade-offs between adaptability, exploration efficiency, and communication overhead. Furthermore, many of these methods focus primarily on throughput or interference avoidance, while neglecting the importance of enhancing users' inherent covert communication capabilities, i.e., reducing the likelihood of detection or tracking by attackers. These limitations necessitate a more balanced and effective anti-attack strategy that promotes diverse channel access without relying on frequent inter-user communication or user mobility. Summary of the Invention

[0006] In order to address the inherent limitations of existing RL-based DSA methods, the present invention proposes a Gibbs sampling-inspired dynamic spectrum access method for covert communication, which essentially enhances the flexibility and concealment of the spectrum access strategy.

[0007] Specifically, the present invention designs a rejection mechanism inspired by Gibbs sampling that probabilistically denies access to overused channels. This method evaluates the access probability of each channel and selectively denies access to overused channels, leading users to more diverse and unpredictable channel selection outcomes.

[0008] This approach not only reduces the risk of users being easily detected by attackers due to long-term access to the same channel, but also promotes more balanced utilization of the available spectrum. As a result, users can achieve stronger covert communication performance without having to frequently communicate with the receiver or move around to avoid interference from attackers.

[0009] Technical Solution A Gibbs sampling-inspired covert communication dynamic spectrum access method comprises the following steps: Step 1: Each SU creates its own Q-Table. In each time slot, after sensing the channel state, it searches the Q-Table for the Q value vector corresponding to the state, that is, the Q values ​​corresponding to all optional actions.

[0010] Step 2: SU collects statistics on the action selection results in the period before the current time slot, calculates the historical channel access frequency, and designs a probability rejection mechanism based on the idea of ​​Gibbs sampling.

[0011] Step 3: Integrate the rejection mechanism into the Q-Learning reinforcement learning framework to build a DSA strategy based on probabilistic rejection and decide the final channel selection.

[0012] Through the above steps, after multiple access iterations, the optimal spectrum access strategy for the cognitive user is obtained, and the covert communication capability is optimized.

[0013] Beneficial effects Inspired by Gibbs sampling, this paper designs a rejection mechanism for channel access decisions in dynamic spectrum access. Unlike the standard Q-Learning algorithm, which always selects the optimal channel, this rejection mechanism probabilistically denies access to the optimal channel, reducing the probability of persistent access to the same channel. This prevents users from continuously accessing a single channel, effectively reducing the probability of being detected and attacked by attackers, and improving the user's ability to communicate covertly. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Schematic diagram of the processing flow of the method of the present invention; Figure 2 A schematic diagram of a dynamic spectrum access scenario according to an embodiment of the present invention; Figure 3 This is a DSA framework diagram for covert communication based on a rejection mechanism and Q-Learning according to an embodiment of the present invention; FIG4 is an embodiment of the present invention =4 and =8 algorithm performance comparison chart (a) =6(b) =8(c) =12). DETAILED DESCRIPTION

[0015] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] The present invention is a dynamic spectrum access method for covert communication inspired by Gibbs sampling. The complete process of a user performing dynamic spectrum access is as follows: Figure 1 shown.

[0017] In the dynamic spectrum access scenario, there is Primary Users (PUs), Secondary Users (SUs) and Independent channels, such as Figure 2 shown.

[0018] First, the user creates their own Q-Table (Q-value state-action table, where states correspond to different channel state vectors and actions represent the selection of different channels). In each time slot, when the user senses the channel state, they search the Q-Table for the Q-value vector corresponding to that state, that is, the Q values ​​corresponding to all optional actions.

[0019] At the same time, the user will collect statistics on the action selection results in the period before the current time slot, calculate the historical channel access frequency to approximate the current channel access probability, and design a probability rejection mechanism based on the idea of ​​Gibbs sampling.

[0020] Gibbs sampling is a Markov Chain Monte Carlo method for extracting samples from high-dimensional joint distributions. Its core concept is to iteratively conditionally sample each variable, thus avoiding the curse of dimensionality that comes with directly sampling from the joint distribution. Inspired by this Gibbs sampling mechanism, this paper introduces a rejection mechanism into the action selection strategy. Specifically: (1) Drawing on the idea of ​​"sampling each variable conditionally in turn" in Gibbs sampling, a sequential traversal conditional selection process is designed, that is, starting from the action with the highest priority, gradually trying to decide whether to adopt the action; (2) Similar to the construction of conditional distribution based on other variables in Gibbs sampling, in each selection step, the probability of the action being rejected is calculated based on the historical access pattern of the channel. This probabilistic rejection mechanism is analogous to the conditional sampling of a variable under a joint distribution in Gibbs sampling; (3) Each time an action is selected, if the current high-priority channel is rejected due to overuse, the system will try the next alternative action instead. This method retains the Markov chain property of Gibbs sampling's "update according to variable conditions" and introduces perturbations while controlling the spatial search range, thereby avoiding falling into a suboptimal local solution.

[0021] Then, users will jointly decide the final channel selection based on the rejection mechanism and the Q-value ranking in the Q-value vector.

[0022] Finally, the access point receives the user's message and, based on the reward function, provides feedback on the actual channel capacity obtained by the user. The user then updates the Q-Table based on the feedback and senses and accesses the next time slot, thus completing the user's dynamic spectrum access.

[0023] A dynamic spectrum access method for covert communication inspired by Gibbs sampling is proposed. The specific steps are as follows: Step 1: Each SU creates its own Q-Table. In each time slot, after sensing the channel state, it searches the Q-Table for the Q value vector corresponding to the state, that is, the Q values ​​corresponding to all optional actions.

[0024] The details are as follows: Step (11): Each SU constructs its own Q-Table (Q value state action table), where the state of the Q-Table corresponds to different channel state vectors, and the action represents the selection of different channels.

[0025] The Q-Table is used to calculate the action selection for channel access and update the Q value after the reward is obtained.

[0026] An example of a Q-Table is Figure 3 , where the state is represented by , the action is represented by .

[0027] Step (12): In the time slot , obtain channel state perception results After that, each SU looks up the corresponding Q value vector in its own Q-Table in Indicates the total number of accessible channels, the last element Indicates the Q value corresponding to the action "do not access". represents the Q value of the mth action, , corresponding to the Q value of selecting access to the mth channel.

[0028] Then sort the Q values ​​in the Q value vector: in Indicates the instructions for the corresponding actions after sorting by Q value. Indicates the action corresponding to the maximum Q value, Indicates the action corresponding to the minimum Q value.

[0029] Step 2: SU collects statistics on the action selection results in the period before the current time slot, calculates the historical channel access frequency, and designs a probability rejection mechanism based on the idea of ​​Gibbs sampling.

[0030] The details are as follows: Step (21): Each SU maintains a channel access probability vector For channel The channel access probability is defined as the channel access probability in a period of time before the current time slot. The number of accesses and the time The ratio of the total number of channel access times to the total number of channel access times.

[0031] The ideal channel access probability follows a uniform distribution, which can be expressed as A sliding window is introduced to calculate the channel access probability. The sliding window size of each SU is ,in is the sliding window size and channel access probability update period The proportional constant of , in the embodiment, is 5. Time slots are updated periodically. , SU for channel The channel access probability is defined as in Indicates that SU is in time slot Whether to access the channel Specifically, if SU is in time slot Select access channel ,So ;otherwise, .

[0032] Each SU independently updates its own channel access probability vector and uses it as the basis for subsequent channel selection decisions.

[0033] Step (22): The difference in channel access probability between two consecutive candidate actions is defined as in Indicates action The probability of selection, Indicates action The probability of selection. The calculation refers to the above formula ( ).parameter The larger the value, the more SU chooses the action. The probability of choosing an action is SU is more likely to fall into the dilemma of accessing only a single channel and is more likely to be detected and attacked by attackers.

[0034] Define the rejection probability as , The parameters Defined as the level of covert communication performance, in this embodiment, the value is 3. A smaller parameter means a higher probability of channel access and a higher probability of action rejection (corresponding to a higher Q value, the probability of action rejection is higher). This enhances the SU's covert communication capabilities and prevents excessive access to the same channel.

[0035] if The last candidate action , and its rejection probability is Here, the rejection probability of the last candidate action ensures that the action is rejected only when its channel access probability is greater than the uniform distribution threshold.

[0036] Introduce a uniformly distributed random variable ,when If , the current action is selected; otherwise, the algorithm moves to the next candidate action and determines again whether to reject the action. If all actions are rejected, SU will select the action "Do not access".

[0037] Step 3: Integrate the rejection mechanism into the Q-Learning reinforcement learning framework to build a DSA strategy based on probability rejection and decide the final channel selection. Figure 3 As shown. It includes the following steps: Step (31): In the time slot , the actual channel state is ,in Indicates the The actual status of each channel. Indicates channel Not occupied, Indicates channel It is occupied. First, SU performs channel sensing and the sensing result is ,in Indicates SU's The perception results of the channels. In actual scenarios, perception is often not perfect and there are perception errors. Assume that the probability of perception error is , the probability that SU correctly perceives the channel state is Perception results As the input of the Q-Learning algorithm framework.

[0038] Step (32): SU based on the perception results Select the corresponding Q value vector, and then select the action according to the Q value sorting and probability rejection mechanism described in steps 1 and 2.

[0039] Step (33): After the SU accesses the channel according to the selected action, the environment feeds back the corresponding reward according to the reward function. The reward function is expressed as follows: in Indicates the penalty value when a collision occurs between SU ​​and PU. is the channel capacity when SU ​​accesses.

[0040] Step (34): The Q value matrix is ​​calculated based on the reward function Update the Q value: in, is the learning rate, . is the discount factor, which indicates the impact of future rewards on current decisions. Step (35): After multiple iterations, the KL divergence reaches convergence. The KL divergence is defined as the channel access probability distribution With uniform distribution The degree of closeness. It is expressed as The above technical solution has the following main contributions: 1) KL divergence is defined as a measure of concealment: The present invention introduces the KL divergence between the user channel access distribution and the uniform distribution as a measure to quantify the concealment performance. The lower the KL divergence, the more balanced and unpredictable the channel access behavior, making it more difficult for attackers to detect or attack. 2) Gibbs sampling-inspired rejection mechanism: This mechanism probabilistically rejects access to overused channels. This mechanism encourages users to dynamically explore multiple channels and reduce long stays on any single channel. 3) Frequency-hopping DSA framework for covert communication: The present invention integrates the rejection mechanism into the Q-Learning reinforcement learning framework to form a new DSA strategy based on non-periodic frequency hopping. This strategy improves covert communication performance by reducing the KL divergence.

[0041] As shown in Figure 4 =4 and =8 when the algorithm of the present invention and the standard Q-Learning algorithm have different Comparison of covert communication performance of (Random is the case of completely random access), where Figure 4(a) =6; Figure 4 (b) =8; Figure 4(c) = 12. Simulation results show that compared with the standard Q-Learning algorithm, the proposed algorithm can reduce the KL divergence by 17.8%, 18.9%, and 15.4%, respectively, which means it has better covert communication performance.

[0042] The above description is only a description of the preferred embodiments of the present application and does not limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical content should be regarded as equivalent valid embodiments and fall within the scope of protection of the technical solution of the present application.

Claims

1. A Gibbs sampling-inspired covert communication dynamic spectrum access method, characterized in that: The following steps are involved: Step 1: Each SU creates its own Q-Table. In each time slot, when the user senses the channel state, it searches the Q-Table for the Q value vector corresponding to the state, that is, the Q values ​​corresponding to all optional actions. Step 2: The SU collects statistics on the action selection results in the period before the current time slot, calculates the historical channel access frequency, and designs a probability rejection mechanism based on the idea of ​​Gibbs sampling. Step 3: Integrate the rejection mechanism into the Q-Learning reinforcement learning framework to build a DSA strategy based on probabilistic rejection and determine the final channel selection; Through the above steps, after multiple access iterations, the optimal spectrum access strategy for the cognitive user is obtained, and the covert communication capability is optimized.

2. The Gibbs sampling-inspired covert communication dynamic spectrum access method according to claim 1, characterized in that: Step 1 is as follows: Step (11): Each SU constructs its own Q-value state action table Q-Table, where the state of the Q-Table corresponds to different channel state vectors, and the action represents the selection of different channels; Step (12): In the time slot , obtain channel state perception results After that, each SU looks up the corresponding Q value vector in its own Q-Table in Indicates the total number of accessible channels, the last element Indicates the Q value corresponding to the action "no access"; represents the Q value of the mth action, , corresponding to the Q value of selecting access to the mth channel; Then sort the Q values ​​in the Q value vector: in Indicates the instructions for the corresponding actions after sorting by Q value; Indicates the action corresponding to the maximum Q value, Indicates the action corresponding to the minimum Q value.

3. The Gibbs sampling-inspired covert communication dynamic spectrum access method according to claim 1, characterized in that: Step 2 is as follows: Step (21): Each SU maintains a channel access probability vector ; For the channel The channel access probability is defined as the channel access probability in a period of time before the current time slot. The number of accesses and the time The ratio of the total number of channel access times; A sliding window is introduced to calculate the channel access probability. The sliding window size of each SU is ,in is the sliding window size and channel access probability update period The proportional constant of the channel access probability is Time slots are updated periodically; Time Slot , SU for channel The channel access probability is defined as in Indicates that SU is in time slot Whether to access the channel ; Specifically, if SU is in time slot Select access channel ,So ; otherwise, . Each SU independently updates its own channel access probability vector and uses it as the basis for subsequent channel selection decisions. Step (22): The difference in channel access probability between two consecutive candidate actions is defined as in Indicates action The probability of selection, Indicates action The probability of selection; Define the rejection probability as , The parameters Defined as the level of covert communication performance; if The last candidate action , and its rejection probability is Introduce a uniformly distributed random variable ,when , the current action is selected; otherwise, the algorithm moves to the next candidate action to determine whether to reject the action again; if all actions are rejected, SU will select the action "do not access".

4. The Gibbs sampling-inspired covert communication dynamic spectrum access method according to claim 3, characterized in that: The value is 5.

5. The Gibbs sampling-inspired covert communication dynamic spectrum access method according to claim 3, characterized in that: The value is 3.

6. The Gibbs sampling-inspired covert communication dynamic spectrum access method according to claim 1, characterized in that: Step 3 is as follows: Step (31): In the time slot , the actual channel state is ,in Indicates the The actual status of each channel, Indicates channel Not occupied, Indicates channel has been occupied; First, SU performs channel sensing, and the sensing result is ,in Indicates SU's The perception results of the channels; As the input of the Q-Learning algorithm framework; Step (32): SU based on the perception results Select the corresponding Q-value vector, and then select the action based on the Q-value sorting and probability rejection mechanism; Step (33): After the SU accesses the channel according to the selected action, the environment feeds back the corresponding reward according to the reward function; the reward function is expressed as follows: in Indicates the penalty value when a collision occurs between SU ​​and PU; is the channel capacity when SU ​​accesses; Step (34): The Q value matrix is ​​calculated based on the reward function Update the Q value: in, is the learning rate, ; is the discount factor, which indicates the influence of future rewards on current decisions; Step (35): After multiple iterations, the KL divergence reaches convergence; the KL divergence is defined as the channel access probability distribution With uniform distribution The degree of closeness is expressed as 。

Citation Information

Patent Citations

  • Sparse channel estimation method based on approximate sampling and underwater acoustic communication system

    CN118041727A

  • Anti-attack method of electromagnetic spectrum allocation intelligent decision-making model

    CN118101100A

  • Fast convergence dynamic induction spectrum access method based on LSTM and Q-Learning fusion

    CN118764114A

  • Probabilistic sampling using search trees constrained by heuristic bounds

    US20120259807A1