Low-power wide-area-network integrated sensing and communications method based on channel sensing and reinforcement learning
Through channel perception and deep reinforcement learning technology, channels are reconstructed and network configuration are optimized, which solves the challenge of channel information utilization in synesthesia fusion in low-power wide area networks, improves communication performance and decoding capabilities, and realizes dynamic adaptive optimal communication transmission .
Patent Information
- Application Number
- PCT/CN2024/086862
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-08
- Filing Date
- 2024-04-09
- Publication Date
- 2024-11-14
AI Technical Summary
In the synesthesia fusion of low-power wide area networks, there are still challenges in how to effectively use wireless signals to mine wireless communication channel information and reconstruct channels based on this information to assist network communication.
A soft demodulation algorithm based on channel perception is used to receive the uplink signal of the terminal node, reconstruct the channel through a deep reinforcement learning model, and calculate the optimal network configuration based on the reconstructed channel to allocate it to each terminal node during the next downlink transmission. The specific steps include calculating the bit log likelihood ratio using Bayesian formula, filtering key frequency points for channel estimation of the deep reinforcement learning model, rebuilding the complete channel through interpolation method, and conducting online reinforcement learning training in actual communication to optimize the model.
It improves the decoding capability and communication performance of the gateway, realizes the dynamic adaptive optimal configuration of the network environment, and improves the overall communication transmission performance of the LPWAN network.
Smart Images

Figure CN2024086862_14112024_PF_FP_ABST
Abstract
Description
Synaesthesia fusion method for low-power wide area networks based on channel perception and reinforcement learning Technical Field
[0001] The present invention belongs to the field of communication technology, and in particular relates to a low-power wide area network synaesthesia fusion method based on channel perception and reinforcement learning. Background Art
[0002] With the continuous advancement of communication technology, both wireless communications and wireless sensing are moving towards higher frequency bands, larger antenna arrays, and smaller devices. The hardware architectures, channel characteristics, and signal processing in these two fields are becoming increasingly similar, creating the ideal conditions for integrated sensing and communications (ISAC).
[0003] Synaesthesia fusion involves using the same set of devices, or sharing some components, within the same network environment to achieve both communication and perception functions. This reduces device cost, size, and power consumption, allowing communication and perception to complement each other. Traditional Internet of Things (IoT) terminal devices use various sensors to sense the physical environment and then transmit this data via IoT uplink to a gateway node for processing. Synaesthesia fusion, as a new IoT technology, integrates communication and perception functions on the same platform, reducing the number and complexity of devices, improving device flexibility and real-time performance, and reducing the number and cost of devices. It provides new solutions for the Industrial Internet of Things ecosystem, such as the Internet of Vehicles, smart factories, and smart healthcare. LPWAN (Low-Power Wide-Area Network) technology is widely used in IoT applications requiring long distances, low power consumption, and a large number of connections. Synaesthesia fusion technology is even more urgently needed for the Internet of Everything, which demands a balance between power consumption, distance, and communication throughput.
[0004] Artificial intelligence (AI) has made tremendous progress in many fields in recent years, and has also been applied to wireless communications to address various technical challenges. AI can learn or explore appropriate strategies from vast and complex data sets, dynamically adjusting them to the environment. Therefore, leveraging AI's dynamic and powerful exploration capabilities can greatly facilitate the realization of interawareness integration in low-power wide-area networks. However, significant challenges remain. For example, how to extract information about wireless communication channels from wireless signals, how to enable AI models to fully utilize this information to reconstruct channels, and how to assist network communications based on the reconstructed channels remain significant issues.
[0005] Summary of the Invention
[0006] The purpose of the embodiments of this specification is to provide a low-power wide area network synaesthesia fusion method based on channel perception and reinforcement learning.
[0007] To solve the above technical problems, the embodiments of the present application are implemented in the following ways:
[0008] The present application provides a low-power wide area network synaesthesia fusion method based on channel perception and reinforcement learning, the method comprising:
[0009] The gateway receiver receives the uplink signal sent by the terminal node, demodulates the uplink signal using a soft demodulation algorithm, and obtains a bit log-likelihood ratio;
[0010] The deep reinforcement learning model completes the channel reconstruction of the corresponding link in the communication process based on the bit log-likelihood ratio and obtains the reconstructed channel;
[0011] Based on the reconstructed channel, the current optimal network configuration is calculated and allocated to each terminal node during the next downlink transmission.
[0012] In one embodiment, the uplink signal is demodulated using a soft demodulation algorithm to obtain a bit log likelihood ratio, including:
[0013] The bit log likelihood ratio calculated based on the uplink signal is given by the Bayesian formula Determine, and the probability of each symbol being sent p(x) is equal, then the bit log likelihood ratio of the kth bit is:
[0014] Among them, Ω k,0 ,Ω k,1 Respectively represent the symbol set where the kth bit is equal to 0 and 1, Ω={A 1 ,A 2 ,···,A W} is the set of all possible symbols received;
[0015] Among them, under the model of additive white Gaussian noise channel, the noise obeys the zero-mean Gaussian distribution N(0,σ 2 ), based on the maximum a posteriori probability criterion, the demodulation result has a conditional probability
[0016] Where Y is the actual signal received by the receiver, Symbol A ω ∈Ω modulation signal.
[0017] In one embodiment, the deep reinforcement learning model completes channel reconstruction of the corresponding link during the communication process based on the bit log likelihood ratio to obtain the reconstructed channel, including:
[0018] The bit log-likelihood ratio is input into the deep reinforcement learning model to select key frequency points as pilot frequencies.
[0019] According to the modulation channel model, the channel estimation of the key frequency point is performed based on the pilot signal to obtain the estimated channel of the key frequency point;
[0020] The estimated channel of the key frequency point is used to perform the original estimation of the complete channel through the interpolation method to obtain the original estimated channel;
[0021] The original estimated channel is estimated N times through N symbols collected in the continuous time slice window, and the complete channel of the corresponding link in the communication process is reconstructed to obtain the reconstructed channel.
[0022] In one embodiment, before actual communication, the deep reinforcement learning model is trained offline to obtain an initial deep reinforcement learning model, which is used for the first round of communication during actual communication;
[0023] The training process of the initial deep reinforcement learning model includes:
[0024] Acquire a simulation data set, where the simulation data set includes a real channel generated by simulation, a transmitted signal, a received signal, and a bit log-likelihood ratio obtained by demodulation at the receiver;
[0025] The deep reinforcement learning model is trained using the simulation data set until the end condition is reached to obtain the initial deep reinforcement learning model.
[0026] In one embodiment, during the actual communication process, online reinforcement learning training is performed on the deep reinforcement learning model, including:
[0027] After completing a round of communication, collect the communication performance parameters of the current round and the communication performance parameters of the previous round;
[0028] According to the communication performance parameters of the current round and the communication performance parameters of the previous round, an online training reward function is constructed to perform online reinforcement learning training on the deep reinforcement learning model.
[0029] In one embodiment, the communication performance parameters include: bit error rate, bit error rate, global throughput in the regional network, and communication delay of the terminal node.
[0030] In one embodiment, the agent of the deep reinforcement learning model includes:
[0031] The experience replay pool is used to store the result data generated by the interaction between the intelligent agent and the low-power wide area network environment; the result data includes the bit log likelihood ratio of the current state, the bit log likelihood ratio of the next state, the action output of the intelligent agent, and the reward for executing the action;
[0032] Sample batches, which are formed by randomly sampling a number of result data from the experience replay pool at regular intervals and used for training the estimation network and the target network;
[0033] Estimate the network, output the state-action value after each iteration, and update the parameters of the estimation network after each iteration;
[0034] The target network outputs the state value after each iteration, and after a preset time, receives the parameters of the estimated network to update the parameters of the target network;
[0035] An optimizer is used to calculate the loss loss based on the reward, state-action value, and state value of the executed action to adjust the parameters of the estimation network through the back propagation of the loss loss.
[0036] In one embodiment, the reward for performing an action is determined according to an offline reward function:
[0037] in, To reconstruct the channel, H is the real channel simulated in the data set, p is the frequency point position, ||H||=N is the total number of frequency points, is the reconstructed channel corresponding to the frequency point p, and H(p) is the real channel corresponding to the frequency point p.
[0038] In one embodiment, the method further includes: during decoding, the gateway uses a soft decision iterative decoding algorithm based on a bit log likelihood ratio to perform decoding.
[0039] In one embodiment, decoding is performed using a soft decision iterative decoding algorithm based on a bit log likelihood ratio, including:
[0040] Constructing factor graph structure based on the check equation constraint of channel coding;
[0041] Initializing an iteration variable based on a bit log-likelihood ratio to obtain an initial iteration variable;
[0042] Perform message passing based on the factor graph structure and initial iteration variables to update the variable nodes;
[0043] The variable node obtains the judgment results of all bits according to the judgment rules;
[0044] The decoding result is determined according to the judgment result.
[0045] It can be seen from the technical solution provided in the above embodiments of this specification that this solution: uses a deep reinforcement learning model to mine key environmental information, so that the gateway can allocate the optimal network configuration to different terminal nodes according to the current network environment conditions, and improve the gateway's decoding capabilities. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0047] FIG1 is a flow chart of a low-power wide area network synaesthesia fusion method based on channel perception and reinforcement learning provided by this application;
[0048] FIG2 is another flow chart of the low-power wide area network synaesthesia fusion method based on channel perception and reinforcement learning provided by the present application;
[0049] FIG3 is a schematic diagram of the training of an agent in the deep reinforcement learning model provided by this application;
[0050] FIG4 is a schematic diagram of the structure of the factor graph provided in this application. DETAILED DESCRIPTION
[0051] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments derived by those skilled in the art based on the embodiments in this specification without creative effort shall fall within the scope of protection of this specification.
[0052] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.
[0053] It will be apparent to those skilled in the art that various modifications and variations may be made to the specific embodiments described herein without departing from the scope or spirit of the present application. Other embodiments will be apparent to those skilled in the art from the present description. The present description and examples are intended to be illustrative only.
[0054] The words “include,” “including,” “have,” “contain,” etc. used in this document are open-ended terms, meaning including but not limited to.
[0055] Unless otherwise specified, "parts" in this application are calculated by mass.
[0056] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.
[0057] 1 and 2 , which illustrate a flow chart of a low-power wide area network synaesthesia fusion method based on channel perception and reinforcement learning applicable to an embodiment of the present application.
[0058] As shown in Figures 1 and 2, the LPWAN synaesthesia fusion method based on channel perception and reinforcement learning may include:
[0059] S110 : The gateway receiver receives an uplink signal sent by the terminal node, demodulates the uplink signal using a soft demodulation algorithm, and obtains a bit log-likelihood ratio.
[0060] Specifically, during actual communication, the terminal node of the LPWAN communicates with the gateway through the uplink (i.e., the terminal node sends an uplink signal to the gateway). The transmitted signal reaches the gateway receiver through the wireless channel; the gateway receiver receives the wireless uplink signal from the terminal node, demodulates the uplink signal using a soft demodulation algorithm, and calculates the bit log-likelihood ratio (Bit LLR) as the demodulation result. The absolute value of the bit log-likelihood ratio can represent the confidence level of the demodulation result.
[0061] Among them, under the model of additive white Gaussian noise channel, the noise obeys the zero-mean Gaussian distribution N(0,σ 2 ), assuming that the set of all possible symbols received is Ω={A 1 ,A 2 ,···,A W}; Based on the maximum a posteriori probability criterion, the demodulation result has a conditional probability
[0062] Where Y is the actual signal received by the receiver, Symbol A ω ∈Ω modulation signal;
[0063] The bit log likelihood ratio (Bit LLR) calculated based on the received uplink signal is given by the Bayesian formula. Since the probability of sending each symbol p(x) is equal, the bit log likelihood ratio of the kth bit is:
[0064] Among them, Ω k,0 ,Ω k,1 Represents the set of symbols where the kth bit is equal to 0 or 1 respectively.
[0065] Substituting into the above formula, we can find a set of bit log-likelihood ratios [l1,l2,...,l k ], and its absolute value can represent the confidence level of the demodulation result.
[0066] S120. The deep reinforcement learning model completes the channel reconstruction of the corresponding link in the communication process according to the bit log-likelihood ratio to obtain a reconstructed channel.
[0067] In one embodiment, S120 may include:
[0068] The bit log-likelihood ratio is input into the deep reinforcement learning model to select key frequency points as pilot frequencies.
[0069] According to the modulation channel model, the channel estimation of the key frequency point is performed based on the pilot signal to obtain the estimated channel of the key frequency point;
[0070] The estimated channel of the key frequency point is used to perform the original estimation of the complete channel through the interpolation method to obtain the original estimated channel;
[0071] The original estimated channel is estimated N times through N symbols collected in the continuous time slice window, and the complete channel of the corresponding link in the communication process is reconstructed to obtain the reconstructed channel.
[0072] Specifically, the gateway actively and passively collects signals during communication, including uplink signals and demodulation results from terminal nodes. It then constructs data records from the continuous time series of received signals and log-likelihood ratios, organized into time-slice windows. The time-slice window stores a sliding window of N time slices, which slides over time to maintain the temporal continuity of the data within the window. A certain number of symbols are collected for iterative reconstruction of the link channel.
[0073] The absolute value of the bit log-likelihood ratio is the confidence probability of decoding. The bit log-likelihood ratio within the same modulation symbol is normalized and then input into the deep reinforcement learning model (Deep Q-Network, DQN) as soft information. The neural network (deep reinforcement learning model) selects the key frequency points for channel reconstruction as pilots, and obtains a set of high-quality pilot sets (p, X k (p)), where p∈S k ,k=1,…,N,S k is the set of all pilots of the kth symbol, p is the frequency point position, X k (p) is the modulated transmitted signal corresponding to the frequency point p in the k-th symbol.
[0074] According to the modulation channel model Y=HX+N, the channel estimation of the key frequency point is performed to obtain the estimated channel of the key frequency point
[0075] Among them, Y k (p) is the received signal corresponding to the frequency point p in the kth symbol, N k (p) is the additive noise corresponding to the frequency point p in the k-th symbol, and H(p) is the multiplicative effect of the channel corresponding to the frequency point p.
[0076] Finally, the estimated key frequency channel Through the interpolation method, the original estimation of the complete channel is performed to obtain the original estimated channel
[0077] Here, interp() is an interpolation algorithm, for example, a linear interpolation algorithm or other high-order interpolation algorithms.
[0078] Estimated channel obtained by key frequency points The original estimation results To rebuild:
[0079] Among them, ω k is the weight coefficient of the kth symbol, β k is the weight coefficient for key frequency point estimation.
[0080] Then after N symbol iterations, the channel of the link is finally reconstructed.
[0081] Among them, the DQN model needs to be trained offline before actual communication for the first round of communication during actual communication. During the actual communication process, the deep reinforcement learning model needs to be trained for online reinforcement learning.
[0082] In one embodiment, before actual communication, the deep reinforcement learning model is trained offline to obtain an initial deep reinforcement learning model, which is used for the first round of communication during actual communication.
[0083] The training process of the initial deep reinforcement learning model includes:
[0084] Acquire a simulation data set, where the simulation data set includes a real channel generated by simulation, a transmitted signal, a received signal, and a bit log-likelihood ratio obtained by demodulation at the receiver;
[0085] The deep reinforcement learning model is trained using the simulation data set until the end condition is reached to obtain the initial deep reinforcement learning model.
[0086] Specifically, the simulation data set can be constructed in a high-performance server by fully simulating the LPWAN communication process based on MATLAB, and using various channel models such as Rayleigh channel and Rice channel.
[0087] The termination condition can be reaching a preset number of training times, or it can be when the offline reward function converges stably and no longer fluctuates violently. There is no restriction on this. The preset number of training times can be set according to actual needs.
[0088] In another embodiment, during the actual communication process, online reinforcement learning training is performed on the deep reinforcement learning model, including:
[0089] After completing a round of communication, collect the communication performance parameters of the current round and the communication performance parameters of the previous round;
[0090] According to the communication performance parameters of the current round and the communication performance parameters of the previous round, an online training reward function is constructed to perform online reinforcement learning training on the deep reinforcement learning model.
[0091] Specifically, according to the communication performance parameter X of the current round t t and the communication performance parameter X of the previous round t-1 t-2 , construct the reward function reward(X t ,X t-1 ):
[0092] Among them, X t =(x t,i ),0≤i≤||X|| includes communication performance indicators (parameters) such as the bit error rate of the terminal node, the bit error rate, the global throughput in the regional network, and the communication delay, ω i and α i are all weight coefficients.
[0093] In another embodiment, as shown in FIG3 , the agent of the deep reinforcement learning model includes:
[0094] The experience replay pool is used to store the result data generated by the interaction between the agent and the LPWAN network environment. The result data includes the bit log-likelihood ratio of the current state, the bit log-likelihood ratio of the next state, the action output of the agent, and the reward for executing the action. The reward for executing the action is determined according to the offline reward function:
[0095] in, To reconstruct the channel, H is the real channel simulated in the data set, p is the frequency point position, ||H||=N is the total number of frequency points, is the reconstructed channel corresponding to the frequency point p, and H(p) is the real channel corresponding to the frequency point p.
[0096] Sample batches, which are formed by randomly sampling a number of result data from the experience replay pool at regular intervals and used for training the estimation network and the target network;
[0097] Estimate the network, output the state-action value after each iteration, and update the parameters of the estimation network after each iteration;
[0098] The target network outputs the state value after each iteration, and receives the parameters of the estimated network after a preset time to update the parameters of the target network;
[0099] An optimizer is used to calculate the loss loss based on the reward, state-action value, and state value of the executed action to adjust the parameters of the estimation network through the back propagation of the loss loss.
[0100] Specifically, each interaction between the agent and the LPWAN network environment generates a record (s, a, r, s') that is inserted into the experience replay pool (or simply the experience pool). Here, state s represents the Bit LLR at the current moment; s' represents the Bit LLR at the next moment; a represents the agent's action output, i.e., the pilot filter result output by the deep reinforcement learning model (or simply the model); and r represents the reward for executing the action, calculated using the offline reward function:
[0101] in To reconstruct the channel, H is the real channel simulated in the data set, p is the frequency point position, ||H||=N is the total number of frequency points, is the reconstructed channel corresponding to the frequency point p, and H(p) is the real channel corresponding to the frequency point p.
[0102] When the number of records reaches a certain level, the model will periodically randomly sample a small batch of records from the experience pool to form a mini-batch for training through experience pool playback. The gradient descent optimizer will then calculate the loss loss, and the weights of the estimated network will be adjusted through the back propagation of the loss loss to complete a round of training. The calculation formula for the training loss loss is:
[0103] Among them, Q(s,a) is the state-action value corresponding to the current state s and the executed action a. The estimation network passes Q(s,a) to the optimizer; For the next state s′, select the execution action a′ that maximizes the Q value among all actions i, the target network passes the selected maximum Q value to the optimizer; θ is the weight coefficient of the estimation network, θ' is the weight coefficient of the target network; γ is the discount coefficient, which is used to quantify the impact of the future on the present.
[0104] Finally, after a certain number of iterations (the number of rounds can be set according to actual needs), the intelligent body will grasp the channel characteristics corresponding to different bit log-likelihood ratios and be able to select key frequency points as pilot signals to reconstruct the channel.
[0105] In this embodiment, a mini-batch training method is used to construct samples using periodic random sampling from an experience replay pool (or simply the experience pool). This breaks the correlation between samples and makes them independent. The purpose of setting up the estimation network and the target network is to reduce the random jitter and interference caused by training. The estimation network uses the calculated Q value to update the network weights (i.e., network parameters) after each iteration, while the target network regularly receives the weights of the estimation network for updates, maintaining a relatively slow update speed, ultimately ensuring the stability and convergence of training.
[0106] S130: Calculate the current optimal network configuration based on the reconstructed channel, and allocate it to each terminal node during the next downlink transmission.
[0107] Specifically, based on each terminal node e∈E={ed1,...,ed m}Reconstruct the complete reconstruction channel of each link After the action output layer of the DQN agent, the network configuration of the terminal node in the next round of communication is obtained:
[0108] in Indicates the configuration parameters for the next round of communication of terminal e It includes data rate, transmission power, communication band selection, coding rate CR, etc., where ||A|| represents the length of A.
[0109] During downlink transmission, the gateway sends configuration information to each terminal node to achieve overall optimal transmission of the local network.
[0110] In one embodiment, during decoding, the gateway uses a soft decision iterative decoding algorithm based on a bit log likelihood ratio to perform decoding, including:
[0111] Constructing factor graph structure based on the check equation constraint of channel coding;
[0112] Initializing an iteration variable based on a bit log-likelihood ratio to obtain an initial iteration variable;
[0113] Perform message passing based on the factor graph structure and initial iteration variables to update the variable nodes;
[0114] After the update, the variable node obtains the judgment results of all bits according to the judgment rules;
[0115] The decoding result is determined according to the judgment result.
[0116] Specifically, the factor graph structure is constructed based on the check equation constraints of the channel coding, including:
[0117] For the check matrix, each row corresponds to a check node f i (i.e. a check equation constraint), each column corresponds to a variable node b j ; Check constraint: P = Hc T .
[0118] For example, for the check matrix H:
[0119] The corresponding factor graph structure is shown in Figure 4.
[0120] Initialize variable node b j (for bit b j The corresponding iteration variable in the soft decision iterative decoding algorithm for: Among them, LLR(b j ) represents bit b j Bit LLR, It is the message passing from variable node to check node.
[0121] Then, according to the factor graph structure, the message passing from the check node to the variable node is:
[0122] where f i →b j Represents the check node f i To variable node b j message passing, b k ∈N(f i )\b j Indicates removal of b j After the other and f i The connected variable nodes, F is the function tanh(·).
[0123] The message transmission from variable node to check node is:
[0124] Finally, based on the previously passed message, update the variable node:
[0125] After updating, the variable node is based on the decision rule Get the decision results of all bits c=[c0,c1,...,c M ], when the constraint Hc of the check equation is satisfied T =0 (H is the check matrix, c is the bit vector obtained by judgment, and 0 is the all-zero vector) or when the iteration round reaches the set maximum upper limit, the decoding result is output, otherwise the iteration continues.
[0126] The low-power wide area network (LPWAN) synaesthesia fusion method based on channel perception and reinforcement learning, provided in embodiments of this application, is based on LPWAN synaesthesia fusion at the physical layer. This method enables LPWAN gateways to extract key environmental information and more accurately reconstruct and predict the channel, thereby assisting their communications. Traditional LPWAN modulation and demodulation technologies can only utilize preambles for perception, but this approach is insensitive to network perturbations and has poor stability. This application designs a synaesthesia-integrated soft demodulation algorithm, combined with deep reinforcement learning technology, to enable LPWAN gateways to perceive wireless signals and improve the overall communication transmission performance of local LPWAN networks. In actual LPWAN communications, the gateway receiver receives wireless signals from terminal nodes and demodulates them using an LPWAN synaesthesia-integrated soft demodulation algorithm based on maximum likelihood estimation. It extracts valid Bit LLR representation confidence levels from these signals to achieve wireless channel perception. The DQN model then analyzes and learns the Bit LLRs, selecting key frequency points as pilots for more accurate channel reconstruction and prediction. Finally, based on the channel information, the optimal communication configuration for each terminal node in the current LPWAN network is calculated. LPWAN synaesthesia fusion technology, based on the physical layer, enables the gateway to leverage its current perception of the network environment, significantly improving its decoding capabilities and enabling optimal communication configuration for each terminal node. Through continuous model iteration within the network system, it ultimately achieves optimal overall communication transmission across a dynamically adaptive local LPWAN network.
[0127] It should be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.
[0128] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.
Claims
1. A low power wide area network synaesthesia fusion method based on channel perception and reinforcement learning, characterized in that: The method comprises: The gateway receiver receives the uplink signal sent by the terminal node, and demodulates the uplink signal using a soft demodulation algorithm to obtain a bit log likelihood ratio; The deep reinforcement learning model completes the channel reconstruction of the corresponding link in the communication process according to the bit log likelihood ratio to obtain a reconstructed channel; According to the reconstructed channel, the current optimal network configuration is calculated to be allocated to each terminal node during the next downlink transmission.
2. The method according to claim 1, characterized in that The method of demodulating the uplink signal by using a soft demodulation algorithm to obtain a bit log likelihood ratio includes: The bit log likelihood ratio calculated based on the uplink signal is given by the Bayesian formula Determine, and the probability of each symbol being sent p(x) is equal, then the bit log-likelihood ratio of the kth bit is: Among them, Ω k,0 ,Ω k,1 represents the symbol set whose kth bit is equal to 0 and 1 respectively, Ω={A 1 ,A 2 ,…,A W } is the set of all possible symbols received; In the additive white Gaussian noise channel model, the noise follows a zero-mean Gaussian distribution N(0,σ 2 ), based on the maximum a posteriori probability criterion, the demodulation result has a conditional probability Where Y is the actual signal received by the receiver, Symbol A ω ∈Ω modulation signal.
3. The method according to claim 1, characterized in that The deep reinforcement learning model completes the channel reconstruction of the corresponding link in the communication process according to the bit log likelihood ratio to obtain the reconstructed channel, including: Inputting the bit log-likelihood ratio into the deep reinforcement learning model, and selecting key frequency points as pilot frequencies; According to the modulation channel model, the channel estimation of the key frequency point is performed based on the pilot signal to obtain the key frequency point The estimated channel of The estimated channel of the key frequency point is used to perform an original estimation of the complete channel by an interpolation method to obtain an original estimated channel; The original estimated channel is iterated N times by using N symbols collected in a continuous time slice window to reconstruct a complete channel of a corresponding link during the communication process, thereby obtaining the reconstructed channel.
4. The method according to claim 1, characterized in that: Before actual communication, the deep reinforcement learning model is trained offline to obtain an initial deep reinforcement learning model for the first round of communication during actual communication; The training process of the initial deep reinforcement learning model includes: Acquire a simulation data set, wherein the simulation data set includes a real channel generated by simulation, a transmission signal, a reception signal, and a bit log-likelihood ratio obtained by demodulation at a receiving party; The deep reinforcement learning model is trained using the simulation data set until an end condition is reached to obtain the initial deep reinforcement learning model.
5. The method according to claim 1, characterized in that In the actual communication process, the deep reinforcement learning model is trained through online reinforcement learning, including: After completing a round of communication, collect the communication performance parameters of the current round and the communication performance parameters of the previous round; According to the communication performance parameters of the current round and the communication performance parameters of the previous round, a reward function for online training is constructed to perform online reinforcement learning training on the deep reinforcement learning model.
6. The method according to claim 5, characterized in that The communication performance parameters include: bit error rate, bit error rate, global throughput in the regional network, and communication delay of the terminal node.
7. The method according to any one of claims 1 to 5, characterized in that: The agent of the deep reinforcement learning model includes: An experience replay pool is used to store the result data generated by the interaction between the agent and the low-power wide area network environment; the result data includes the bit log likelihood ratio of the current state, the bit log likelihood ratio of the next state, the action output of the agent, and the reward for executing the action; Sample batches, which are formed by periodically randomly sampling a number of the result data from the experience replay pool, and are used for training the estimation network and the target network; An estimation network is provided, which outputs a state-action value after each iteration and updates parameters of the estimation network after each iteration; The target network outputs a state value after each iteration, and receives the parameters of the estimation network after a preset time to update the parameters of the target network; An optimizer is used to calculate the Loss loss according to the reward of the executed action, the state-action value and the state value, so as to adjust the parameters of the estimation network through Loss loss back propagation.
8. The method according to claim 7, characterized in that The reward for executing the action is determined according to the offline reward function: in, To reconstruct the channel, H is the real channel simulated and generated in the data set, p is the frequency point position, ∥H∥=N is the total number of frequency points, is the reconstructed channel corresponding to the frequency point p, and H(p) is the real channel corresponding to the frequency point p.
9. The method according to claim 1, characterized in that: The method further includes: during decoding, the gateway uses a soft decision iterative decoding algorithm to perform decoding based on the bit log likelihood ratio.
10. The method according to claim 9, characterized in that The decoding is performed using a soft decision iterative decoding algorithm based on a bit log-likelihood ratio, including: Constructing factor graph structure based on check equation constraints of channel coding; Initializing an iteration variable based on the bit log-likelihood ratio to obtain an initial iteration variable; Perform message passing according to the factor graph structure and the initial iteration variables to update the variable nodes; After the update, the variable node obtains the judgment results of all bits according to the judgment rules; A decoding result is determined according to the judgment result.
Citation Information
Patent Citations
Combined equipment detection and channel estimation method based on deep learning
CN111641570A
Simulation method and system for channel estimation and iterative detection of large-scale MIMO system
CN113517941A
Low-power wide area network communication fusion method based on channel perception and reinforcement learning
CN116264704A
Apparatus and method of calculating channel frequency domain correlation
US20110194633A1
Cited By
Wi-Fi-oriented intelligent performance sensing and predicting method and system
CN120676385A
Quantitative distributed online near-end gradient descent optimization method based on Bandit feedback
CN121078462A
